Naked-eye 3D display method and device for in-vehicle meters

Abstract

To provide a method and device for naked eye 3D displaying a vehicle instrument that solves the risk of operator dizziness caused when the naked eye 3D display cannot be refreshed in a timely manner to improve the display effect of the naked eye 3D.SOLUTION: A method for naked eye 3D displaying a vehicle instrument, includes: collecting a real-time eye position of an operator in a real time, generating a corresponding real-time visual interlace image based on the real-time eye position, and displaying the real-time visual interlace image on a display interface; identifying whether the operator is engaged in human behavior, if so, obtaining human behavior information of the operator; predicting a deviation interval of the operator based on the human behavior information, and generating and buffering a viewing angle image set corresponding to the deviation interval; and compensating in response to the real-time visual interlace image on the display interface based on the viewing angle image set.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure belongs to the field of naked-eye 3D technology, and particularly relates to a method and apparatus for naked-eye 3D display of in-vehicle instruments.

Background Art

[0002] Compared with the conventional 2D display, 3D display is more suitable for human visual characteristics and has more stereoscopic and immersive feelings when observing. The naked-eye 3D technology is realized based on the parallax of human eyes. That is, when the left and right eyes of a human observe the same target, there are differences in the images. After performing some processing on the screen and mapping the images with parallax to the left and right eyes of a human respectively, the human brain synthesizes a three-dimensional screen with a sense of depth.

[0003] When naked-eye 3D is applied to in-vehicle instruments, it has advantages such as a sense of reality, stereoscopic feeling, and no need to wear special glasses. It can display road condition information and vehicle operation condition information more intuitively and in real time, and can bring a better driving experience. However, the naked-eye 3D display has many restrictions on the observation position. If the observer does not fix at the optimal position, the optimal display effect cannot be seen. When naked-eye 3D is applied to in-vehicle instruments, when the operator observes the road condition, the head shifts, so the display effect of naked-eye 3D is affected. For example, the display screen cannot be refreshed in time. If the refresh of the naked-eye 3D display screen cannot respond in time, the operator may get dizzy, which is extremely dangerous in the driving state.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Embodiments of the present disclosure provide a method and apparatus for naked-eye 3D display of in-vehicle instruments that can solve the problem that there is a risk of the operator getting dizzy when the naked-eye 3D display cannot be refreshed in time.

Means for Solving the Problems

[0005] According to a first aspect, embodiments of the present disclosure are Collect the real-time human eye position of the operator in real time, generate a corresponding real-time visual interlace image based on the real-time human eye position, and display the real-time visual interlace image on a display interface; Identify whether a human body behavior has occurred to the operator, and if a human body behavior has occurred, obtain the human body behavior information of the operator; Pre-determine the displacement interval of the operator based on the human body behavior information, generate a set of perspective images corresponding to the displacement interval, and buffer them; Based on the set of perspective images, perform response compensation on the real-time visual interlace image on the display interface. Provide a method for in-vehicle instrument naked-eye 3D display.

[0006] Exemplarily, the step of generating a corresponding real-time visual interlace image based on the real-time human eye position includes: Determine a viewing area interval based on the real-time human eye position, and generate a 3D model scene of the viewing area interval in a graphics library; Arrange a virtual viewpoint position corresponding to the real-time human eye position in the graphics library, and perform real-time rendering based on the virtual viewpoint position to generate a real-time visual interlace image corresponding to the virtual viewpoint position in the 3D model scene.

[0007] In addition, the step of determining a viewing area interval based on the real-time human eye position includes: Obtain a standard viewing area where a predetermined operation center is located, where the standard viewing area is an angular range of several unit intervals on the left and right of the viewing angle of the predetermined operation center in the horizontal direction, and each unit interval within the standard viewing area is the viewing area interval; Calculate the viewing area interval where the real-time human eye position is located based on the deflection angle between the real-time human eye position and the predetermined operation center.

[0008] According to a second aspect, an embodiment of the present disclosure is a collection module that collects the position information of an operator, the collection module including an eye position measurement unit that collects the real-time human eye position and a behavior identification unit that acquires human body behavior information; a calculation processing module including a first calculation processing unit that receives the real-time human eye position, calculates a current viewing area section, and generates a first interlace image screen corresponding to the current viewing area section, and a second calculation processing unit that predicts a displacement section of the operator based on the human body behavior information and generates a second interlace image screen corresponding to the displacement section; a buffer module that buffers the second interlace image screen; a display module that displays the first interlace image screen and extracts and refreshes the second interlace image screen; and provides a vehicle instrument naked-eye 3D display device. Provide a vehicle instrument naked-eye 3D display device.

[0009] According to a third aspect, an embodiment of the present disclosure is including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein when the processor executes the computer program, the vehicle instrument naked-eye 3D display method according to any one of the first aspects is realized, and provides a terminal device. Provide a terminal device.

[0010] According to a fourth aspect, an embodiment of the present disclosure is a computer-readable storage medium storing a computer program that, when executed by a processor, realizes the vehicle instrument naked-eye 3D display method according to any one of the first aspects. Provide a computer-readable storage medium.

[0011] According to a fifth aspect, an embodiment of the present disclosure is when executed on a terminal device, causing the terminal device to execute the vehicle instrument naked-eye 3D display method according to any one of the first aspects. Provide a computer program product.

[0012] In addition, for the effects of the second to fifth aspects, reference can be made to the relevant descriptions in the first aspect, and the description is omitted here.

Advantages of the Invention

[0013] The embodiments of the present disclosure have the following effects as compared with the prior art.

[0014] In the embodiments of the present disclosure, by collecting the human body behavior information of the operator, the displacement interval of the operator is determined in advance, and a set of perspective images corresponding to the displacement interval is generated, so as to perform response compensation on the real-time visual interlace image. When the operator has a head rotation or other behaviors that deviate from the original position, the refresh rate of the displayed naked-eye 3D screen decreases from 100 ms to 30 ms or less, reducing the dizziness when viewing the naked-eye 3D screen and significantly improving the experience effect of the 3D screen.

[0015] In the embodiments of the present disclosure, the real-time human eye position is collected by the human eye positioning unit, a first interlace image screen corresponding to the current viewing area interval is generated and displayed, the human body behavior information is obtained using the behavior identification unit, the displacement interval of the operator is predicted, and a second interlace image screen corresponding to the displacement interval is generated and buffered, thereby significantly reducing the burden on a single camera and increasing the refresh rate of the naked-eye visual interlace image display.

[0016] Needless to say, the products of any one of the present disclosures do not necessarily have to achieve all the above-mentioned advantages at the same time.

[0017] The above description is only an overview of the technical solution of the present disclosure, and can be implemented based on the content of the specification in order to more clearly understand the technical solution of the present invention. In addition, in order to more clearly understand the purpose, features and advantages of the present disclosure, specific embodiments of the present disclosure are given below.

[0018] To more clearly explain the technical solutions in the embodiments of the present disclosure, the drawings that need to be used in the following description of the embodiments or the prior art will be briefly introduced. Needless to say, the drawings in the following description are only some embodiments of the present disclosure, and those skilled in the art can also obtain other drawings based on these drawings without creative efforts.

Brief Description of the Drawings

[0019]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0020] In the following description, specific details such as specific system configurations and technologies are described for the purpose of explanation rather than limitation to fully understand the embodiments of the present disclosure. However, those skilled in the art can also implement the present disclosure in other embodiments without these specific details. In other cases, the details of well-known systems, devices, circuits, and methods are omitted to prevent the description of the present disclosure from being hindered by unnecessary details.

[0021] Note that, when the term "comprising" is used in the specification and claims of the present disclosure, it indicates the presence of the features, wholes, steps, operations, elements and / or assemblies to be described, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, assemblies and / or their combinations.

[0022] Also, when the term "and / or" is used in the specification and claims of the present disclosure, it refers to any combination and all possible combinations of one or more of the relatedly listed items, and includes these combinations.

[0023] The term "case" may be construed, as used in the specification and claims of the present disclosure, based on the context, as "when...", or "with...", or "in response to having determined...", or "in response to having detected...". Similarly, the description that "the described condition or event" is "when determined" or "when detected" may, based on the context, be construed as "when the described condition or event is determined" or "in response to the described condition or event being determined", or "when the described condition or event is detected" or "in response to the described condition or event being detected".

[0024] Also, in the specification and claims of the present disclosure, terms such as "first", "second", "third", etc. are merely for the purpose of distinguishing and describing, and are not to be understood as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" described in this disclosure specification mean that one or more embodiments of the present disclosure include specific features or structures described in connection with that embodiment. Therefore, "in one embodiment", "in some embodiments", "in some other embodiments", "in some other alternative embodiments", etc. described at different positions in this specification do not necessarily refer to the same embodiment, and unless otherwise specified, it means "one or more embodiments, but not all embodiments". The terms "include", "have" and their variants all mean "include but are not limited to" unless otherwise specified.

[0026] When naked-eye 3D is applied to in-vehicle instruments, there is a risk of the operator getting dizzy when the head shifts while the operator observes the road conditions and when the refresh of the display screen of the naked-eye 3D cannot respond in a timely manner. The embodiments of the present disclosure can pre-determine the displacement interval of the operator, generate a set of perspective images corresponding to the displacement interval, and perform response compensation on the real-time visual interlace image, so as to provide an in-vehicle instrument naked-eye 3D method that can improve the display effect of the naked-eye 3D.

[0027] The in-vehicle instrument naked-eye 3D display method according to the embodiments of the present disclosure can be applied to terminal devices such as in-vehicle devices. By way of example and not limitation, when the above terminal device is an in-vehicle device, the in-vehicle device may be a general term for devices with a display function intelligently designed and developed for a locomotive by in-vehicle device technology, for example, an instrument panel, a display, etc.

[0028] Taking the above terminal device being an in-vehicle instrument as an example. FIG. 1 is a block diagram showing a partial structure of an in-vehicle instrument according to an embodiment of the present disclosure. As shown in FIG. 1, the in-vehicle instrument includes components such as a display device 100, a camera group 200, and a processor 300, and the processor 300 and the display device 100 transmit data signals wired / wirelessly.

[0029] Specifically, the display device 100 includes a backlight for providing illumination light, a grating for 3D imaging, and a display module for displaying a screen.

[0030] By way of example and not limitation, the display device 100 includes a cover plate 101, a 3D diffraction grating 102, a TFT (Thin Film Transistor) module 103, a backlight module 104, and a case 105. Here, the cover plate 101, the 3D diffraction grating 102, the TFT module 103, and the backlight module 104 are all enclosed within the case 105, and the camera group 200 is installed on the upper surface of the case 105.

[0031] Specifically, the camera group 200 may be a dual camera module, and the camera group 200 includes any one or a combination of an RGB camera, a structured light camera, and an IR camera. The operation mechanism of the camera is that the light emitted to the object is reflected on the object surface, and the reflected light is transmitted through the lens to the image sensor. The image sensor receives the reflected light and converts the optical signal into an electrical signal and transmits it to the analog-digital conversion circuit. The analog-digital conversion circuit converts the received analog electrical signal into a digital electrical signal and transmits it to the digital signal processing chip for processing. The signal after the final processing is transmitted to the computer through the USB port, so that the original image can be displayed by the display. Therefore, the camera group 200 can collect the image data of the interior of the vehicle, especially the face and body of the operator.

[0032] Optionally, considering the implementation cost and photographing in a dark environment, the camera group 200 may be a combination of an RGB camera and an IR camera.

[0033] Specifically, the processor 300 may include one or more of processors such as a central processing unit, an application processor (AP), and a baseband processor. The processor 300 may be the nerve center and command center of the wireless router. The processor 300 can generate operation control signals based on instruction operation codes and time-series signals, and complete the control of instruction acquisition and instruction execution. The memory can store computer-executable program codes, and the executable program codes include instructions. The processor 300 executes the instructions stored in the memory to execute various functional applications and data processing of the network device. The memory may include a program storage area and a data storage area, and stores, for example, a visual interface image screen. For example, the memory may be a double data rate synchronous dynamic random access memory DDR or a flash memory Flash, etc.

[0034] Although not shown in the figure, the in-vehicle instrument may further include a power source for supplying power to each component. Preferably, the power source is logically connected to the processor 300 via a power management system, and the power management system manages functions such as charging, discharging, and power consumption management.

[0035] It can be understood by those skilled in the art that the structure of the in-vehicle instrument shown in FIG. 1 does not limit the in-vehicle instrument. It may include more or fewer components than shown, or some components may be combined, or different component arrangements may be used.

[0036] Hereinafter, the in-vehicle instrument naked-eye 3D display method according to the present disclosure will be exemplarily described in relation to specific embodiments. FIG. 2 is a flowchart of the in-vehicle instrument naked-eye 3D display method according to an embodiment of the present disclosure. By way of example and not limitation, the method can be applied to the above-mentioned in-vehicle instrument naked-eye 3D display device.

[0037] Step S201: Collect the real-time human eye position of the operator in real time, generate a corresponding real-time visual interlace image based on the real-time human eye position, and display the real-time visual interlace image on the display interface.

[0038] In an embodiment, the collected real-time human eye position is obtained by an image collection device with a human eye tracking system collecting an image or video. The image collection device includes any one or a combination of an infrared waterproof gun-type camera, an infrared hemispherical camera, an isokinetic spherical camera, a high-speed spherical camera, a wide dynamic camera, a recording terminal, a video recording card, and a DVR (Digital Video Recorder) recorder. The image collection device can make different selections according to the differences in the environment and collection needs.

[0039] For example, the image collection device with a human eye tracking system includes an infrared camera, an infrared light illuminator, and an eye tracking algorithm for pupil center detection and artifact elimination. Image processing and data collection are processed by dedicated hardware or by a computer or software. Illumination by infrared has advantages such as not being easily visible and being able to filter artifacts from artificial light sources by wavelength. The real-time human eye position collected by the image collection device with a human eye tracking system includes the three-dimensional coordinates of the human eye with respect to the earth coordinate system.

[0040] In one embodiment, FIG. 3 shows the process of a method for generating a corresponding real-time visual interlace image based on the real-time human eye position in the embodiment of the present disclosure. By way of example and not limitation, the specific steps are as follows.

[0041] S301: Obtain a standard visual field where the viewing angle of a predetermined operation center is located.

[0042] Specifically, the standard visual field is the angular range of several unit intervals on the left and right of the viewing angle of the predetermined operation center in the horizontal direction, and each unit interval within the standard visual field is a visual field interval.

[0043] Exemplarily, the central point at the operation position is set as the predetermined operation center, the connection line between the predetermined operation center and the center of the connection line of the optical centers of the dual cameras is set as the viewing angle of the predetermined operation center, the distance between the centers of the connection lines of the optical centers of the dual cameras and the operator's both eyes at the operation position is set as the observation distance, with the center of the connection line of the optical centers of the dual cameras as the vertex and the observation distance as the side length, a sector interval that covers the operator's operation range, that is, the standard viewing field, can be demarcated. The optimal observation distance is set as 60 cm to 90 cm, the range of ±25° of the viewing angle of the predetermined operation center is set as the standard viewing field, and the standard viewing field is divided into 10 unit intervals, that is, one viewing field interval every 5°.

[0044] S302: Calculate the viewing field interval where the real-time human eye position is located based on the deviation angle between the real-time human eye position and the viewing angle of the predetermined operation center.

[0045] Extract the earth coordinate data at the real-time human eye position, calculate the earth coordinate data of the center of the connection line of the optical centers of the dual cameras based on the positioning information of the dual cameras, and further calculate the distance between the real-time human eye position and the center of the connection line of the optical centers of the dual cameras, and the deviation angle between the real-time human eye position and the viewing angle of the predetermined operation center by the trigonometric method.

[0046] Determine the viewing field interval where the real-time human eye position is located based on the degree of the deviation angle and the demarcation of the unit intervals within the standard viewing field.

[0047] S303: Generate a 3D model scene of the viewing field interval in the graphics library.

[0048] Exemplarily, the graphics library may be a cross-platform, cross-language application programming interface for rendering 2D and 3D vector images, such as OpenGL (Open Graphics Library), Direct3D (a 3D graphics programming interface based on the general-purpose mode of Microsoft).

[0049] As an example, the Open Graphics Library (OpenGL for short in English) is a cross-language and cross-platform application programming interface used to render 2D and 3D vector images. In this embodiment, OpenGL is adopted as the graphics library, the OpenGL engine environment is initialized, rendering data is prepared, and based on the viewing area where the real-time human eye position is located, a corresponding 3D model scene is generated within OpenGL.

[0050] S304: Place a virtual viewpoint position corresponding to the real-time human eye position in the graphics library, and perform real-time rendering based on the virtual viewpoint position to generate a real-time visual interlace image corresponding to the virtual viewpoint position in the 3D model scene.

[0051] In one embodiment, based on the real-time human eye position, in OpenGL, a virtual viewpoint position corresponding to the actual human eye observation viewpoint is placed, and further real-time rendering is performed to generate a screen of the corresponding viewing angle, which is pushed to the video memory and further displayed on the liquid crystal display.

[0052] As an example, Framebuffer drives one video display device from one memory buffer containing complete frame data on one video output device. In this embodiment, Framebuffer is adopted as the video memory, and the screen of the corresponding viewing angle generated by real-time rendering is output to the display.

[0053] As an example, a liquid crystal display (LCD for short in English) arranges a liquid crystal cell between two parallel glass substrates. A TFT (thin film transistor) is installed on the lower substrate glass, and a color filter is installed on the upper substrate glass. By controlling the rotation direction of liquid crystal molecules according to the change of signals and voltages in the TFT, the presence or absence of polarized light emission of each pixel point is controlled to achieve the display purpose. In this embodiment, an LCD is adopted as the display to display the screen corresponding to the viewing angle.

[0054] In addition, when the change in the virtual viewpoint position is small, in order to improve the refresh efficiency, local refresh is performed using OpenGL. When the current frame is being rendered, the nodes whose states have changed are counted, the refresh area generated by all the nodes whose states have changed is calculated, the screen area in the screen corresponding to the refresh area is calculated, the projection matrix is set based on the screen area, and further the viewport and the cutting area are set. The obtained model data and texture data are submitted to the renderer for rendering, and further the corresponding real-time visual interlace image is obtained.

[0055] Step S202: Identify whether a human behavior has occurred to the operator. If it has occurred, obtain the human behavior information of the operator.

[0056] In one embodiment, during driving, if the movement of the operator's human behavior is too large, it will be a factor that the refresh of the naked-eye 3D screen cannot respond in a timely manner. Since the possible human behaviors of the operator are mainly the rotation and movement of the head, if the posture information of the operator's face and the main body of the human body is captured by the camera, the screen of the buffer can be extracted based on the operator's human behavior, the refresh speed of the screen can be improved, and the dizziness when viewing the naked-eye 3D screen can be reduced. FIG. 4 shows the flow of a method for identifying whether a human behavior has occurred to the operator in the embodiment of the present disclosure. By way of example and not limitation, the specific steps are as follows.

[0057] S401: Collect the operator's pose key points in real time, and construct the human main body pose vector and the motion vector of the face corner points based on the pose key points.

[0058] Use a camera to collect the two-dimensional RGB image information of the human body, obtain more than 18 human body key points from the two-dimensional RGB image based on the human body pose estimation algorithm, and compare with the standard human body pose to obtain pose key points that can accurately represent the rotation of the human head or the movement of the upper body. Use the human body pose estimation algorithm to obtain the raw data of the human body pose key points in the image coordinate system, and construct the main body pose vector of the human body and the motion vector representing the face corner points.

[0059] As an example, OpenPose is an open-source library of a human body pose recognition algorithm based on convolutional neural networks and supervised learning, and can realize pose estimation of human body movements, facial expressions, finger movements, etc. In this embodiment, OpenPose is used to obtain human body key points from a two-dimensional RGB image. By way of example and not limitation, the numbers are pose key points such as 1 (representing the nose), 2 (representing the neck), 3 (representing the right shoulder), 6 (representing the left shoulder), 15 (representing the right eye), 16 (representing the left eye), etc.

[0060] S402: Calculate the displacement angles of the operator's main body and head based on the human main body pose vector and the motion vector of the face corner points.

[0061] S403: Determine the human behavior based on the displacement angles of the main body and head. When the displacement angles of the main body and head exceed a predetermined value, it is determined that a human behavior has occurred to the operator, and further, the displacement angles of the main body and head are used as human behavior information representing the human body's inclined posture.

[0062] Step S203: Predetermine the displacement interval of the operator based on the human behavior information, generate a set of perspective images corresponding to the displacement interval, and buffer them.

[0063] In one embodiment, a human body posture algorithm is used to integrate and identify human body behavior information to obtain the behavior tendency probability value of the operator.

[0064] By way of example and not limitation, human body behavior information is extracted, a movement feature vector is constructed with the displacement angle of the operator's main body, a head rotation feature vector is constructed with the displacement angle of the operator's head, and a trained SVM (Support Vector Machine) is used to perform a behavior determination on the movement feature vector, and the first classification result is output as x.

[0065]

Number

[0066] Similarly, the trained SVM is used to perform a behavior determination on the head rotation feature vector, and the second classification result is output as y.

[0067]

Number

[0068] The behavior tendency probability value of the operator is the sum of the first classification result x and the second classification result y.

[0069] Based on the human body behavior tendency probability value, the eye position of the operator within the next unit time can be predicted. For example, when the tendency probability value is -1, it is predicted that the eye position of the operator within the next unit time is within one visual field interval on the left side of the current visual field interval, that is, the displacement interval of the operator is one visual field interval on the left side of the current visual field interval.

[0070] By way of example and not limitation, OpenGL can be adopted as the graphics library, and a set of perspective view images corresponding to the displacement interval can be generated and buffered based on the displacement interval. The specific steps are as follows.

[0071] (1) Generate a 3D model scene of the displacement interval in the graphics library.

[0072] (2) Render and generate buffer images corresponding to each of the predetermined viewing area predetermined viewpoints in the displacement interval.

[0073] (3) Use the buffer images corresponding to all the predetermined viewing area predetermined viewpoints as a viewing angle image set corresponding to the displacement interval.

[0074] Specifically, a plurality of viewpoints are preset in each viewing area, the predetermined viewpoints of the viewing area are uniformly distributed within each viewing area along the circumferential direction, with the center of the connection line of the optical centers of the dual cameras as the vertex, the included angle between the predetermined viewpoints of adjacent viewing areas is equal, the deviation angle between the viewing angle of the predetermined viewpoint of the viewing area and the predetermined operation center is used as the position information of the predetermined viewpoint of the viewing area, the view corresponding to the predetermined viewpoint of the viewing area is stored in the memory in advance, and through the continuous real-time refresh processing of the viewing angle image set corresponding to the subsequent displacement interval, the observer can always view the optimal 3D screen and further reduce the dizziness. For example, 24 viewpoints may be preset in each viewing area.

[0075] Step S204: Based on the viewing angle image set, perform response compensation on the real-time visual interlace image in the display interface.

[0076] In one embodiment, FIG. 5 shows the flow of a method for performing response compensation on the real-time visual interlace image in the embodiment of the present disclosure. By way of example and not limitation, the specific steps are as follows.

[0077] S501: Based on the real-time human eye position of the operator, determine whether the real-time human eye position of the operator and the human eye position of the operator collected in the previous frame are located in the same viewing area.

[0078] In this embodiment, for the geodetic coordinate data of the human eye position of the operator in each frame collected by the camera, by means of triangulation, the distance between the human eye position of the operator and the center of the connection line between the optical centers of the dual cameras, and the deflection angle between the human eye position of the operator and the viewing angle of a predetermined operation center are calculated. Further, based on the division of the unit interval in the standard viewing area of the degree of the deflection angle between the human eye position of the operator and the viewing angle of the predetermined operation center, the viewing area interval where the human eye position of the operator is located can be determined.

[0079] Buffer the viewing area interval where the human eye position of the operator in the previous frame is located, compare the calculated viewing area interval where the real-time human eye position is located with the viewing area interval where the human eye position of the operator in the previous frame is located, and determine whether the two are the same. If the two are the same, the real-time human eye position of the operator and the human eye position of the operator collected in the previous frame are located in the same viewing area interval.

[0080] S502: When the real-time human eye position of the operator and the human eye position of the operator collected in the previous frame are not located in the same viewing area interval, determine whether the real-time human eye position of the operator is located in the displacement interval.

[0081] S503: When the real-time human eye position of the operator is located in the displacement interval, set the predetermined viewing point of the viewing area interval closest to the real-time human eye position of the operator in the displacement interval as the predetermined viewing point of the target viewing area interval, and obtain the target buffer image matched from the viewing angle image set corresponding to the predetermined viewing point of the target viewing area interval.

[0082] In this embodiment, based on the deflection angle between the real-time human eye position and the viewing angle of the predetermined operation center, the predetermined viewing point of the viewing area interval with the position information closest to the real-time human eye position is specified, and the closest predetermined viewing point of the viewing area interval is set as the target predetermined viewing point, and the buffer image corresponding to the target predetermined viewing point is extracted from the viewing angle image set as the target buffer image.

[0083] S504: Perform response compensation on the real-time visual interlace image based on the target buffer image.

[0084] In addition, when the real-time human eye position of the operator is not within the displacement interval, a view corresponding to a predetermined viewing point in the viewing area interval closest to the real-time human eye position is acquired from the memory, and response compensation is performed on the real-time visual interlace image.

[0085] In another embodiment, considering that when the brightness of the driving environment is too low, the naked-eye 3D effect is poor, and the operator is more likely to get tired when viewing the naked-eye 3D screen, the in-vehicle instrument naked-eye 3D display method according to the present disclosure also includes a method of automatically turning off the naked-eye 3D effect. By way of example and not limitation, the specific steps of automatically turning off the naked-eye 3D effect are as follows.

[0086] (1) Perform data frame average processing on the brightness captured by the RGB camera for the screen, and calculate the median of the data frame segment of the current environment.

[0087] (2) Determine the working environment based on the median of the data frame segment of the current environment. If the median of the data frame segment of the current environment is smaller than a predetermined reference threshold, it is determined that the ambient light is insufficient, and frame measurement is performed using the IR camera to supplement the situation when the light is insufficient.

[0088] (3) If the data measured by the IR camera for frame measurement is smaller than a predetermined threshold, automatically turn off the naked-eye 3D effect and convert it to 2D display. At this time, the eye tracking system of the RGB camera is turned off.

[0089] Note that the magnitudes of the numbers of the steps in the above embodiments do not mean the order of execution before and after. The execution order of each process should be determined by its function and internal logic, and does not limit the implementation process of the embodiments of the present disclosure in any way.

[0090] FIG. 6 shows a structural block diagram of an in-vehicle instrument naked-eye 3D display device according to an embodiment of the present disclosure. For the sake of convenience of description, only the parts related to the embodiments of the present disclosure are shown.

[0091] Referring to FIG. 6, the apparatus includes the following steps.

[0092] A collection module for collecting the operator's position information, the collection module including a human eye positioning unit for collecting the real-time human eye position and a behavior identification unit for obtaining the human body behavior information; A calculation processing module including a first calculation processing unit for receiving the real-time human eye position, calculating the current viewing area interval, and generating a first interlaced image screen corresponding to the current viewing area interval, and a second calculation processing unit for predicting the displacement interval of the operator based on the human body behavior information and generating a second interlaced image screen corresponding to the displacement interval; A buffer module for buffering the second interlaced image screen; A display module for displaying the first interlaced image screen and extracting and refreshing the second interlaced image screen.

[0093] It should be noted that the content such as the information interaction and execution process between the above-mentioned apparatus / units is based on the same concept as the embodiments of the method of the present disclosure. Therefore, for its specific functions and the resulting technical effects, reference can be made to the specific embodiments of the method, and the description is omitted here.

[0094] For the convenience and brevity of description, only the division of the above functional units and modules is taken as an example for illustration. However, in actual applications, if necessary, the above function allocation can be completed by different functional units and modules, that is, the internal structure of the above device can be divided into different functional units or modules to realize all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit. The above integrated unit may be implemented as hardware or as a software functional unit. Also, the specific names of each functional unit and module are for distinguishing from each other and do not limit the protection scope of the present disclosure. For the specific operation process of the units and modules in the above system, reference may be made to the corresponding process in the embodiment of the above method, and the description is omitted here.

[0095] Embodiments of the present disclosure further provide a network device, which includes at least one processor, a memory, and a computer program stored in the memory and executable by the at least one processor. When the processor executes the computer program, it implements the steps in the embodiments of the above methods.

[0096] Embodiments of the present disclosure provide a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps in the embodiments of the above methods.

[0097] Embodiments of the present disclosure provide a computer program product which, when executed on a mobile terminal, implements the steps in the embodiments of the above methods.

[0098] When the above integrated unit is realized as a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Thus, in the present disclosure, the realization of all or part of the processes in the above method embodiments is completed by instructing the relevant hardware with a computer program, and the computer program may be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be realized. The computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may at least include any entity or device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, electrical communication signal, and software distribution medium that can carry the computer program code to the imaging device / terminal device. For example, it may be a USB disk, a mobile hard disk, a magnetic disk, or an optical disk. In a certain jurisdiction, according to legislation and patent practice, the computer-readable medium shall not be an electrical carrier signal and an electrical communication signal.

[0099] In the above embodiments, the description of each embodiment has its own focus. For the parts that are not detailed or not described in a certain embodiment, the relevant descriptions of other embodiments can be referred to.

[0100] Each example unit and algorithm step described in connection with the embodiments disclosed in this specification can be implemented in combination with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application of the technical solution and design constraints. Those skilled in the art can implement the functions described using different methods for each specific application, but such implementation should not be considered as exceeding the scope of this disclosure.

[0101] In the embodiments according to this disclosure, the disclosed apparatus / network device and method can be implemented in other ways. For example, the embodiments of the apparatus / network device described above are merely illustrative. For example, the division of the above modules or units is a division of logical functions, and in actual implementation, there may be other division methods. For example, a plurality of units or assemblies may be combined or integrated into another system. Some features may be omitted or not executed. Also, the described or considered couplings, direct couplings, or communication connections may be indirect couplings or communication connections through some ports, devices, or units, and may be in electrical, mechanical, or other forms.

[0102] The units described as separate members may or may not be physically separated. The members shown as units may or may not be physical units, that is, they may be located in one place or distributed among a plurality of network units. Depending on the actual needs, some or all of the units can be selected to achieve the purpose of the solution means of this embodiment.

[0103] The above embodiments are merely for explaining the technical solutions of the present disclosure and are not intended to be limiting. Although the present disclosure has been described in detail with reference to the above embodiments, the technical solutions described in each of the above embodiments can be modified, or some of the technical features therein can be equivalently replaced. Any of these modifications or replacements should be included within the protection scope of the present disclosure without departing from the spirit and scope of the technical solutions of each embodiment of the present disclosure.

Description of Reference Numerals

[0104] 101 Cover Plate 102 3D Diffraction Grating 103 TFT Module 104 Backlight Module 105 Case 200 Camera Group 300 Processor S201 Step S202 Step S203 Step S204 Step S301 Step S302 Step S303 Step S304 Step S401 Step S402 Step S403 Step S501 Step S502 Step S503 Step S504 Step

Claims

1. Collecting the real-time human eye position of the operator in real time, generating a corresponding real-time visual interlace image based on the real-time human eye position, and displaying the real-time visual interlace image on a display interface; Identifying whether a human body behavior has occurred to the operator, and if a human body behavior has occurred, obtaining the human body behavior information of the operator; Pre-determining the displacement interval of the operator based on the human body behavior information, generating a set of perspective images corresponding to the displacement interval, and buffering them; Performing response compensation on the real-time visual interlace image on the display interface based on the set of perspective images, including: The step of generating a corresponding real-time visual interlace image based on the real-time human eye position includes: Determining a viewing area interval based on the real-time human eye position, and generating a 3D model scene of the viewing area interval in a graphics library; Arranging a virtual viewpoint position corresponding to the real-time human eye position in the graphics library, and rendering and generating a real-time visual interlace image corresponding to the virtual viewpoint position in the 3D model scene in real time based on the virtual viewpoint position; The step of determining a viewing area interval based on the real-time human eye position includes: Obtaining a standard viewing area where the viewing angle of a predetermined operation center is located, where the standard viewing area is an angular range of several unit intervals on each of the left and right sides of the viewing angle of the predetermined operation center in the horizontal direction, and each unit interval in the standard viewing area is the viewing area interval; Calculating a viewing area interval where the real-time human eye position is located based on the deflection angle between the real-time human eye position and the predetermined operation center; An in-vehicle instrument naked-eye 3D display method.

2. Collecting the real-time human eye position of the operator in real time, generating a corresponding real-time visual interlace image based on the real-time human eye position, and displaying the real-time visual interlace image on a display interface; Identifying whether a human body behavior has occurred to the operator, and if a human body behavior has occurred, obtaining the human body behavior information of the operator; Determining in advance the displacement interval of the operator based on the human body behavior information, generating a set of perspective images corresponding to the displacement interval, and buffering them; Based on the set of perspective images, performing response compensation on the real-time visual interlace image in the display interface; The step of identifying whether a human body behavior has occurred to the operator; Collecting the posture key points of the operator in real time, and constructing a human body main body posture vector and an operation vector of the face corner points based on the posture key points; Calculating the displacement angles of the main body and the head of the operator based on the human body main body posture vector and the operation vector of the face corner points, and when the displacement angles of the main body and the head exceed a predetermined value, determining that a human body behavior has occurred to the operator, and further using the displacement angles of the main body and the head as human body behavior information representing the human body inclination posture; In-vehicle instrument naked-eye 3D display method.

3. Collecting the real-time human eye position of the operator in real time, generating a corresponding real-time visual interlace image based on the real-time human eye position, and displaying the real-time visual interlace image on a display interface; Identifying whether a human body behavior has occurred to the operator, and when a human body behavior has occurred, obtaining the human body behavior information of the operator; Determining in advance the displacement interval of the operator based on the human body behavior information, generating a set of perspective images corresponding to the displacement interval, and buffering them; Based on the set of perspective images, performing response compensation on the real-time visual interlace image in the display interface; The step of determining in advance the displacement interval of the operator based on the human body behavior information; Integrating and identifying human body behavior information using a human body posture algorithm to obtain an action tendency probability value of the operator; Predicting the human eye position of the operator within the next unit time based on the action tendency probability value, and calculating the displacement interval of the operator based on the human eye position of the operator within the next unit time; In-vehicle instrument naked-eye 3D display method.

4. The step of generating a set of perspective images corresponding to the displacement interval and buffering them; A step of generating a 3D model scene of the displacement interval in the graphics library Based on several predetermined viewing area intervals and predetermined viewpoints in the displacement interval, rendering and generating buffer images corresponding to each of the viewing area intervals and predetermined viewpoints, and using the buffer images corresponding to all the viewing area intervals and predetermined viewpoints as a viewing angle image set corresponding to the displacement interval, including The in-vehicle instrument naked-eye 3D display method according to any one of claims 1 to 3

5. Based on the viewing angle image set, the step of performing response compensation on the real-time visual interlace image in the display interface is Based on the real-time human eye position, determining whether the real-time human eye position and the human eye position of the operator collected in the previous frame are located in the same viewing area interval When the real-time human eye position and the human eye position of the operator collected in the previous frame are not located in the same viewing area interval, determining whether the real-time human eye position is located in the displacement interval When the real-time human eye position is located in the displacement interval, taking the viewing area interval and predetermined viewpoint closest to the real-time human eye position in the displacement interval as the target viewing area interval and predetermined viewpoint, and obtaining a target buffer image matched corresponding to the target viewing area interval and predetermined viewpoint from the viewing angle image set Based on the target buffer image, performing response compensation on the real-time visual interlace image, including The in-vehicle instrument naked-eye 3D display method according to claim 4

6. A collection module for collecting the position information of the operator, including a human eye positioning unit for collecting the real-time human eye position and a behavior identification unit for obtaining the human body behavior information A first calculation processing unit that receives the real-time human eye position, calculates the current viewing area interval, and generates a first interlace image screen corresponding to the current viewing area interval, and a second calculation processing unit that predicts the displacement interval of the operator based on the human body behavior information and generates a second interlace image screen corresponding to the displacement interval, including A buffer module for buffering the second interlace image screen A display module that displays the first interlace image screen and extracts and refreshes the second interlace image screen, including The first calculation processing unit: determines a viewing area based on the real-time human eye position, generates a 3D model scene of the viewing area in a graphics library, arranges a virtual viewpoint position corresponding to the real-time human eye position in the graphics library, and generates in real time a real-time visual interlace image corresponding to the virtual viewpoint position in the 3D model scene based on the virtual viewpoint position, Determining the viewing area based on the real-time human eye position includes: obtaining a standard viewing area where the viewing angle of a predetermined operation center is located, the standard viewing area being an angular range of several unit intervals on each of the left and right sides of the viewing angle of the predetermined operation center in the horizontal direction, and each unit interval in the standard viewing area being the viewing area, calculating a viewing area where the real-time human eye position is located based on a deviation angle between the real-time human eye position and the predetermined operation center, An in-vehicle instrument naked-eye 3D display device.

7. A collection module that collects the position information of the operator, the collection module including a human eye positioning unit that collects the real-time human eye position and a behavior identification unit that obtains human body behavior information, a first calculation processing unit that receives the real-time human eye position, calculates the current viewing area, and generates a first interlace image screen corresponding to the current viewing area, and a second calculation processing unit that predicts a displacement interval of the operator based on the human body behavior information and generates a second interlace image screen corresponding to the displacement interval, a buffer module that buffers the second interlace image screen, a display module that displays the first interlace image screen and extracts and refreshes the second interlace image screen, The behavior identification unit: collects the posture key points of the operator in real time, constructs a human body main body posture vector and an operation vector of the face corner points based on the posture key points, calculates the displacement angles of the main body and head of the operator based on the human body main body posture vector and the operation vector of the face corner points, and determines that a human body behavior has occurred to the operator when the displacement angles of the main body and head exceed a predetermined value, and further uses the displacement angles of the main body and head as human body behavior information representing the human body inclination posture, An in-vehicle instrument naked-eye 3D display device.

8. A collection module for collecting the position information of an operator, the collection module including an eye position measurement unit for collecting the real-time human eye position and a behavior identification unit for obtaining human body behavior information, a first calculation processing unit that receives the real-time human eye position, calculates the current viewing area section, and generates a first interlace image screen corresponding to the current viewing area section, and a second calculation processing unit that predicts the displacement section of the operator based on the human body behavior information and generates a second interlace image screen corresponding to the displacement section, a buffer module for buffering the second interlace image screen, a display module for displaying the first interlace image screen and extracting and refreshing the second interlace image screen, wherein predicting the displacement section of the operator based on the human body behavior information in the second calculation processing unit includes integrating and identifying human body behavior information using a human body posture algorithm to obtain a behavior tendency probability value of the operator, predicting the human eye position of the operator within the next unit time based on the behavior tendency probability value, and calculating the displacement section of the operator based on the human eye position of the operator within the next unit time, an in-vehicle instrument naked-eye 3D display device.

9. A terminal device including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein when the processor executes the computer program, the in-vehicle instrument naked-eye 3D display method according to any one of Claims 1 to 3 is realized, a terminal device.

10. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the in-vehicle instrument naked-eye 3D display method according to any one of Claims 1 to 3 is realized, a computer-readable storage medium.

Images