Display device and image generation method thereof
By acquiring the rendering information of the eye position, determining the virtual image point, and unifying the rendering information of the sub-pixels, the problem of insufficient image clarity in 3D display products is solved, and a clearer image display effect is achieved.
Patent Information
- Application Number
- CN202611471788.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-20
- Publication Date
- 2026-11-17
AI Technical Summary
In 3D display products, insufficient image clarity leads to poor display effects, especially in 3D head-up display devices, where optical path parallax causes serious problems such as image blurring and crosstalk.
By acquiring the rendering information of the eye position, the first virtual image point is determined, and multiple sub-pixels of the display device are rendered according to the image information of the virtual image point. This ensures that the central ray of the first sub-pixel and the non-central ray of the second sub-pixel present the same content, avoids crosstalk, and improves clarity.
It effectively improves image blurring, enhances the clarity and display effect of display products, reduces image crosstalk, and improves user experience.
Smart Images

Figure CN123113158A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of display technology, and more particularly to a display device and an image generation method thereof. Background Technology
[0002] With the development of display technology, the application of 3D display technology is becoming increasingly widespread. For example, 3D HUD (Head-Up Display) technology is an in-vehicle display technology based on the principle of stereoscopic vision. It not only projects information onto the windshield but also utilizes realistic stereoscopic vision technology to give virtual images different depths and distances in three-dimensional space, blending them with real road conditions. Drivers can obtain key information without looking down at the instrument panel, improving driving safety and convenience. However, in some 3D display products, image clarity still needs improvement.
[0003] Therefore, how to improve the above problems has become one of the urgent technical issues to be addressed at this stage. Summary of the Invention
[0004] To address the aforementioned technical problems, this disclosure provides a display device and an image generation method thereof.
[0005] In a first aspect, this disclosure provides an image generation method for a display device, comprising: Rendering information is obtained based on the eye position; wherein, the rendering information includes a first virtual image point; the multiple sub-pixels of the display device include a first sub-pixel and a second sub-pixel, the center ray of the first sub-pixel corresponds to the first virtual image point, and at least one non-center ray of the second sub-pixel corresponds to the first virtual image point; Rendering is performed based on the rendering information to generate an image.
[0006] Secondly, based on the same inventive concept, this disclosure provides a display device that generates images using the image generation method of the display device in the first aspect.
[0007] The technical solution provided in this disclosure has the following advantages compared with the prior art: This disclosure provides a display device and an image generation method thereof. The image generation method includes acquiring rendering information based on eye position and rendering based on the rendering information to generate an image. The rendering information includes a first virtual image point, a central ray of a first sub-pixel corresponding to the first virtual image point, and a non-central ray of a second sub-pixel corresponding to a second virtual image point. The central ray of the first sub-pixel corresponding to the first virtual image point means that the intersection of the backward extension of the central ray of the first sub-pixel with the virtual image plane is the first virtual image point; the non-central ray of the second sub-pixel corresponding to the second virtual image point means that the intersection of the backward extension of at least one non-central ray of the second sub-pixel with the virtual image plane is the first virtual image point. In this disclosure, both the first and second sub-pixels are rendered according to the image information of the first virtual image point. With this configuration, the content presented by the central ray of the beam emitted by the first sub-pixel and the non-central ray of the beam emitted by the second sub-pixel received by the user's eye is the same, thus helping to avoid crosstalk between them, improving the problem of blurry generated images, enhancing clarity, and thus improving the display effect of the display product. Attached Figure Description
[0008] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0009] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 The diagram shown is a schematic representation of a 3D head-up display device provided in an embodiment of this disclosure. Figure 2 The diagram shown is a schematic representation of the imaging principle of a display device provided in an embodiment of this disclosure. Figure 3 The diagram shown is a schematic representation of an image generation method for a display device provided in an embodiment of this disclosure. Figure 4 The diagram shown is a schematic representation of a sub-pixel beam, a virtual image plane, and a viewing area provided in an embodiment of this disclosure. Figure 5 The diagram shown is a schematic representation of step S100 provided in an embodiment of this disclosure; Figure 6 The diagram shown is a schematic representation of a method for generating a first data lookup table according to an embodiment of this disclosure. Figure 7 The diagram shown is a schematic representation of step S320 provided in an embodiment of this disclosure; Figure 8 The diagram shown is a schematic representation of step S322 provided in an embodiment of this disclosure; Figure 9 The diagram shown is another schematic representation of step S322 provided in an embodiment of this disclosure; Figure 10 The diagram shown is another schematic representation of step S100 provided in an embodiment of this disclosure; Figure 11 The diagram shown is a schematic of a setting sub-lookup table provided in an embodiment of this disclosure. Detailed Implementation
[0011] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.
[0012] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.
[0013] Before describing the technical solutions provided in the embodiments of this disclosure, this disclosure first illustrates the applicable scenarios and system architecture of the technical solutions. This technical solution can be applied in the field of 3D display, such as in automotive 3D head-up display technology. Figure 1 The diagram shown is a schematic representation of a 3D head-up display device provided in an embodiment of this disclosure. Please refer to it. Figure 1 The in-vehicle system includes an image processor (not shown in the diagram), an image source, and an optical system. The image processor acquires 3D scene information at the viewpoint sampling point and renders it into a 2D image. The image processor then transmits the rendered 2D image to the image source. Light emitted from the image source travels to the optical system, such as a reflecting optical system. The light is reflected and amplified by the reflecting optical system before entering the windshield and then the driver's eyes. The backward extensions of the light rays converge to form a virtual image surface. The direction of the light rays perceived by the human eye appears to originate from this virtual image surface. Two 2D images, corresponding to the left and right eye viewing angles, are formed on the virtual image surface. These two images exhibit a parallax shift in spatial depth. After being received by the driver's eyes, they are fused by the brain and perceived as a 3D virtual image with depth.
[0014] Figure 2 The diagram shown is a schematic representation of the imaging principle of a display device provided in an embodiment of this disclosure. Please refer to it. Figure 2The image source may include the display panel 210, and the optical system may include, but is not limited to, a prism 220. The prism 220 is located on the light-emitting surface side of the display panel 210. Light emitted from the display panel 210 is redirected by the prism 220, changing the propagation direction of the corresponding light rays. This causes the planar image rays, originally emitted in the same direction, to be guided to different spatial directions, thereby creating two images with parallax at the observer's left and right eyes—a left-eye image and a right-eye image. The right-eye image and the right-eye image exhibit parallax shift; binocular parallax causes the brain to fill in a 3D spatial image, i.e., a virtual image surface 10 with a 3D display effect. It should be noted that... Figure 2 This is only for illustrative purposes and does not mean that the virtual image surface 10 is necessarily directly in front of the display panel 210.
[0015] The inventors discovered during their research that parallax exists in the optical path of 3D head-up display devices. This means that the position of the same sub-pixel on the virtual image plane changes when viewed from different positions. Therefore, light rays not directly facing the eye are perceived at a different position than expected, resulting in blurred images, reduced clarity, and negatively impacting the display performance.
[0016] In view of this, the present disclosure provides a display device and an image generation method thereof to improve the clarity of display products, thereby improving the display effect of display products.
[0017] Figure 3 The diagram shown is a schematic representation of an image generation method for a display device provided in this disclosure. Please refer to it. Figure 3 This disclosure provides an image generation method for a display device, including but not limited to steps S100 and S200, wherein: Step S100: Obtain rendering information based on eye position; wherein, the rendering information includes a first virtual image point; the multiple sub-pixels of the display device include a first sub-pixel and a second sub-pixel, the center ray of the first sub-pixel corresponds to the first virtual image point, and at least one non-center ray of the second sub-pixel corresponds to the first virtual image point. It should be noted that, please refer to... Figure 1 "The central ray of the first sub-pixel corresponds to the first virtual image point" means that the intersection of the backward extension of the central ray of the first sub-pixel with the virtual image plane is the first virtual image point; "The non-central ray of the second sub-pixel corresponds to the second virtual image point" means that the intersection of the backward extension of at least one non-central ray of the second sub-pixel with the virtual image plane is the first virtual image point.
[0018] Step S200: Render according to the rendering information to generate an image.
[0019] Through the embodiments of this disclosure, it is possible to achieve the following: when a user's eye is located at a certain position and simultaneously receives the central ray of the beam emitted by the first sub-pixel and the non-central ray of the beam emitted by the second sub-pixel, the content presented by the central ray of the beam emitted by the first sub-pixel and the non-central ray of the beam emitted by the second sub-pixel is the same. Therefore, it is beneficial to avoid crosstalk between the two, improve the problem of blurry generated images, enhance clarity, and thus improve the display effect of display products.
[0020] Furthermore, the inventors discovered that the light emitted by different sub-pixels is not completely independent. The light emitted by a sub-pixel spreads out from the pixel, similar to the floodlight pattern of a flashlight, with multiple non-central rays surrounding the central ray. At least one non-central ray from the second sub-pixel falls at the point where the central ray of the first sub-pixel falls. Therefore, when the human eye moves to the position of viewing the first virtual image point, it should be viewing the first virtual image point formed by the central ray of the first sub-pixel. However, the first virtual image point actually observed is affected by the light from at least the non-central rays of the second sub-pixel. This causes the human eye to observe both the image presented by the first sub-pixel and the image presented by the second sub-pixel. The difference between the two images creates crosstalk, resulting in a blurred image with low clarity for the user.
[0021] Specifically, Figure 4 The diagram shown is a schematic representation of a sub-pixel beam, virtual image plane, and view area provided in an embodiment of this disclosure. Please refer to it. Figure 3 and Figure 4 To clearly illustrate the image generation method of the display device provided in this disclosure, the following will be combined with... Figure 4 This disclosure will be explained. It should be noted that this disclosure... Figure 4 This is only to illustrate the relationship between the sub-pixel beam, the virtual image plane 10, and the view area Eyebox, and does not represent the actual positions of the three.
[0022] It should also be noted that, Figure 1 In the 3D head-up display shown, light emitted from the image source is directed towards a reflective optical system. The light is reflected and magnified by the reflective optical system before entering the car's windshield and then reflected again into the driver's eyes. The reflective optical system is used to change the direction of light propagation, ensuring that the light enters the eye along a predetermined path.
[0023] refer to Figure 1In this case, the virtual image surface can be considered as formed by the convergence of the backward extensions of several light rays, and the intersection of the backward extension of a light ray with the virtual image surface is considered as the virtual image point corresponding to that light ray. The statement in this disclosure that "the center ray of the first sub-pixel corresponds to the first virtual image point" can be considered as the intersection of the backward extension of the center ray of the first sub-pixel with the virtual image surface 10 being the first virtual image point; similarly, "the non-center ray of the second sub-pixel corresponds to the second virtual image point" can be considered as the intersection of the backward extension of at least one non-center ray of the second sub-pixel with the virtual image surface 10 being the first virtual image point.
[0024] Figure 4 Compared to Figure 1 Since the reflective optical system is omitted, it can be considered that several rays converge to form the virtual image surface 10, and the intersection of a ray with the virtual image surface 10 is considered to be the virtual image point corresponding to that ray. Similarly, the statement in this disclosure that "the center ray of the first sub-pixel corresponds to the first virtual image point" can be considered to mean that the intersection of the center ray of the first sub-pixel with the virtual image surface 10 is the first virtual image point; and the statement that "the non-center ray of the second sub-pixel corresponds to the second virtual image point" can be considered to mean that the intersection of at least one non-center ray of the second sub-pixel with the virtual image surface 10 is the first virtual image point.
[0025] Specifically, Figure 4 The image shows a virtual image plane 10, the viewport Eyebox, a light beam A emitted from sub-pixel PA, and a light beam B emitted from sub-pixel PB. The virtual image plane 10 is not a physically existing plane, but rather a virtual image plane perceived by the human eye, formed by the convergence of light rays or their backward extensions. Unlike a real image, a virtual image cannot be displayed on a screen and can only be perceived through human observation. Figure 1 For example, virtual image plane 10 refers to the light emitted from the image source that is directed toward an optical system, such as a reflecting optical system. The light is reflected by the reflecting optical system and then projected and magnified before entering the windshield of a car. After being reflected by the windshield, the light enters the driver's eyes, and the backward extensions of the light converge to form virtual image plane 10. Figure 4 The reflection optical system is omitted, and the virtual image surface can be considered as a surface formed by the light emitted from the sub-pixel.
[0026] The eyebox refers to the area that the driver's eyes are looking at.
[0027] The image source includes multiple sub-pixels, and the light beam emitted by each sub-pixel includes several rays. The accompanying drawings of this disclosure show the light beams emitted by two sub-pixels (sub-pixel PA and sub-pixel PB), wherein region A represents the projection range of the light beam emitted by sub-pixel PA, and region B represents the projection range of the light beam emitted by sub-pixel PA.
[0028] The point corresponding to the central ray A1 of sub-pixel PA on the virtual image plane 10 is virtual image point 11, and the coordinates of virtual image point 11 in the virtual image plane 10 are (X1, Y1). The point where the central ray A1 of sub-pixel PA falls within the viewport Eyebox is S1. It should be noted that the point corresponding to the central ray A1 of sub-pixel PA on the virtual image plane 10 refers to the intersection of the backward extension of the central ray A1 of sub-pixel PA and the virtual image plane 10.
[0029] The point corresponding to the center ray B1 of sub-pixel PB on the virtual image plane 10 is virtual image point 12, and the coordinates of virtual image point 12 in the virtual image plane 10 are (X2, Y2). The point where the center ray B1 of sub-pixel PB falls within the viewport Eyebox is S2. It should be noted that the point corresponding to the center ray B1 of sub-pixel PB on the virtual image plane 10 refers to the intersection of the backward extension of the center ray B1 of sub-pixel PB and the virtual image plane 10.
[0030] In related technologies, sub-pixel PA is rendered according to the ray information of virtual image point 11, and sub-pixel PB is rendered according to the ray information of virtual image point 12. However, the light beams emitted by sub-pixel PA and sub-pixel PB overlap. When the user's eye is positioned at S1, they can observe both the image rendered by sub-pixel PA according to the image information of virtual image point 11 and the image rendered by sub-pixel PB according to the image information of virtual image point 12. The image information corresponding to virtual image point 11 and the image information corresponding to virtual image point 12 differ, creating crosstalk between them, which in turn causes the user to observe a blurry image with low clarity.
[0031] To address the aforementioned issues, the image generation method provided in this disclosure renders the sub-pixels that generate crosstalk and the sub-pixels that originally displayed a clear image according to the ray information of the same virtual image point.
[0032] Specifically, taking the user's eye located at point S1 in the Eyebox as an example, sub-pixel PA is the first sub-pixel, and sub-pixel PB is the second sub-pixel. In step S100, rendering information is obtained based on the eye position. With the user's eye at S1, the user's eye will receive the central ray A1 emitted by the first sub-pixel (sub-pixel PA) and the non-central ray B2 emitted by the second sub-pixel (sub-pixel PB). In this embodiment, the obtained rendering information includes a first virtual image point, which is used for subsequent rendering.
[0033] In step S200, when rendering based on the rendering information, for the first sub-pixel and the second sub-pixel, rendering is performed according to the image information corresponding to the first virtual image point. Specifically, both the first sub-pixel and the second sub-pixel are rendered according to the image information of the first virtual image point, and the content of the beam emitted by the first sub-pixel is the same as the content of the beam emitted by the second sub-pixel.
[0034] It should be noted that when the user's eye is at position S1, if only the central ray of the first sub-pixel is received, the image is clear. However, the non-central ray B2 of the second sub-pixel also falls on S1. In related technologies, the second sub-pixel is rendered according to the image of virtual image point 12. The image information of virtual image point 11 and virtual image point 12 are different, and they will cause crosstalk, which will work together on S1, resulting in an unclear image observed by the user. In this embodiment, virtual image point 11 is used as the first virtual image point, and both the first and second sub-pixels are rendered according to the image information of the first virtual image point (virtual image point 11). With this setting, when the user's eye is at position S1, the central ray A1 of the beam emitted by the first sub-pixel and the non-central ray B2 of the beam emitted by the second sub-pixel present the same content, which helps to avoid crosstalk between the two, improves the problem of blurry generated images, enhances clarity, and thus improves the display effect of the display product.
[0035] For further information, please continue to refer to [link / reference]. Figure 3 and Figure 4 In one optional embodiment of this disclosure, the non-center ray of the second sub-pixel overlaps with the center ray of the first sub-pixel.
[0036] Specifically, taking sub-pixel PA as the first sub-pixel and sub-pixel PB as the second sub-pixel as an example, the projection range of the light beam emitted by the first sub-pixel overlaps with that of the light beam emitted by the second sub-pixel. Specifically, the central ray A1 of the first sub-pixel overlaps with the non-central ray B2 of the second sub-pixel, and both fall at point S1 within the eyebox, corresponding to the same position in the virtual image plane 10, i.e., both corresponding to virtual image point 11. If the content displayed by the central ray A1 of the first sub-pixel and the non-central ray B2 of the second sub-pixel differs, crosstalk will occur between them, resulting in image blurring.
[0037] The image generation method provided in this embodiment renders both the first sub-pixel and the second sub-pixel according to the image information of the first virtual image point. With this setting, when the user's eye is in S1, the light received (the center light ray A1 of the first sub-pixel and the non-center light ray B2 of the second sub-pixel) corresponds to the same display content. Therefore, it is beneficial to reduce the occurrence of crosstalk problems, improve the problem of blurry generated images, enhance clarity, and thus improve the display effect of display products.
[0038] Please continue to refer to this. Figure 3 and Figure 4 In one optional embodiment of this disclosure, when the eye position changes, the first virtual image point corresponding to at least one sub-pixel changes.
[0039] It should be noted that a virtual image is not a physically existing entity on a plane, but rather a perceptual result deduced by the brain based on the direction of light received by the human eye. Please refer to... Figure 1 The backward extensions of light rays converge to form a virtual image surface 10. The direction of the light rays perceived by the human eye appears to originate from this virtual image surface 10. In other words, the spatial position of the virtual image is essentially determined by the directional distribution of light rays entering the eye. When the eye position changes, the direction of light rays emitted from the same sub-pixel in the image source changes after reflection by the optical system and the windshield. This change in the direction of the light rays directly causes a shift in the perceived position of the virtual image by the brain.
[0040] For a given sub-pixel, this means that the virtual image point corresponding to that sub-pixel should change. If the eye position is not considered, and the corresponding sub-pixel is rendered according to the image information of a fixed virtual image point regardless of the eye position, then when the eye position deviates from the preset position, the image seen will be blurry.
[0041] In this embodiment, when the eye position changes, the first virtual image point corresponding to at least one sub-pixel is adjusted. That is, when the eye position changes, the content displayed by at least some sub-pixels also changes. For example, please refer to... Figure 4 When the eye position is at S1, virtual image point 11 is used as the first virtual image point, and the first and second sub-pixels are rendered according to the image information of the first virtual image point. When the eye position changes, for example, from S1 to S2, if the image is still rendered according to the image information of virtual image point 11, the image will be blurry.
[0042] This embodiment adjusts the first virtual image point corresponding to at least one sub-pixel to improve image clarity. For example, when the user's eye moves from S1 to S2, sub-pixel PA is an interfering pixel. The non-center ray A2 of sub-pixel PA crosstalks with the center ray B1 of sub-pixel PB. At this time, sub-pixel PB is considered the first sub-pixel, and sub-pixel PA is considered the second sub-pixel. The center ray B1 of the first sub-pixel (sub-pixel PB) corresponds to virtual image point 12, and the non-center ray A2 of the second sub-pixel (sub-pixel PA) also corresponds to virtual image point 12. Therefore, virtual image point 12 is used as the first virtual image point, and the first sub-pixel (sub-pixel PB) and the second sub-pixel (sub-pixel PA) are rendered according to the image information of virtual image point 12. With this setting, when the user's eye is at S2, the light rays that can be received include ray B1 and ray A2. The intersection of the backward extension lines of these two rays with the virtual image surface 10 is virtual image point 12. At this time, using virtual image point 12 as the first virtual image point helps to improve the clarity of the image seen by the user, thereby improving the display effect of the display product. That is, through this application, when the human eye position corresponds to the first virtual image point, by calibrating at least one sub-pixel of the first virtual image point where non-central light rays fall, the clarity of the image seen by the user is improved, thereby improving the display effect of the display product.
[0043] Please refer to Figure 1 and Figure 2 In one optional embodiment of this disclosure, the display device is a 3D head-up display device. The display device includes a display panel 210 and a plurality of prisms 220, the prisms 220 being located on the light-emitting side of the display panel 210; the display panel 210 is used to display planar images, and the prisms 220 are used to transform the planar images of the display panel 210 into stereoscopic images, and to project part of the stereoscopic image to the left eye and part to the right eye.
[0044] Optionally, the display panel 210 may be an LCD (Liquid Crystal Display) display panel, an OLED (Organic Light-Emitting Diode) display panel, a Micro-LED (Micro Light-Emitting Diode) display panel, or a Mini LED (Mini Light-Emitting Diode) display panel, etc., and this disclosure does not limit it.
[0045] The display panel 210 outputs two-dimensional planar image information. The display panel 210 includes multiple sub-pixels arranged in an array. For example, the sub-pixels may include sub-pixels of various colors such as red, green, and blue. The planar image displayed by the display panel 210 includes left-eye image information to be projected onto the left eye and right-eye image information to be projected onto the right eye, with a parallax corresponding to spatial depth between the left-eye and right-eye image information.
[0046] Multiple prisms 220 are disposed on the light-emitting side of the display panel 210, and the arrangement direction of the prisms 220 is the same as the arrangement direction of the pixel columns in the display panel 210. Each prism 220 corresponds to at least one sub-pixel of the display panel 210. The function of the prisms 220 is to change the propagation direction of the light emitted by their corresponding sub-pixels, so that the planar image light rays that originally emitted in the same direction are directed to different spatial directions, for example, deflected towards the driver's left or right eye, thereby forming two images with parallax at the observer's left and right eyes.
[0047] Alternatively, the prism 220 may have a wedge-shaped cross section, a triangular cross section, or other geometry capable of deflecting light rays; this disclosure does not impose any specific limitations.
[0048] When light emitted from a sub-pixel on the display panel 210 is incident on its corresponding prism 220, the light is refracted at the incident and exit surfaces of the prism 220, and the exit direction is deflected relative to the incident direction. The deflected light is then projected onto the area where the user's eyes are located by a subsequent reflective optical system (such as a mirror group, windshield, etc.).
[0049] By configuring prisms 220 for different sub-pixels, the light emitted by the sub-pixels carrying left-eye image information on the display panel 210 can be directed primarily to the driver's left eye; and the light emitted by the sub-pixels carrying right-eye image information can be directed primarily to the driver's right eye. Thus, the planar image displayed on the display panel 210 is split into two images corresponding to the left and right eyes respectively. There is a parallax offset between the left-eye and right-eye images corresponding to spatial depth. After the driver's eyes receive these two images respectively, they are fused by the visual center of the brain and perceived as a 3D virtual image surface 10 with stereoscopic depth.
[0050] Figure 5 The diagram shown is a schematic representation of step S100 provided in an embodiment of this disclosure. Please refer to it. Figure 3 and Figure 5 In a specific embodiment of step S100 of this disclosure, the method for implementing step S100 includes, but is not limited to, steps S110 and S120. Specifically, step S100, the method for obtaining rendering information based on eye position, may include the following sub-steps: Step S110: Provide a first data lookup table. The first data lookup table is a pre-established data structure that records the correspondence between eye positions and first virtual image points. The method for generating the first data lookup table will be described in detail below and will not be repeated here.
[0051] Step S120: Obtain the first virtual image point corresponding to the eye position according to the first data lookup table. It should be noted that the eye position can be detected separately for the left and right eyes, or it can be calculated by detecting the head position and combining it with the pupillary distance model; this disclosure does not impose any specific limitations. After obtaining the eye coordinates, use these coordinates to find the corresponding first virtual image point in the first data lookup table.
[0052] This embodiment transforms complex geometric optics calculations into simpler lookup operations by pre-establishing a first data lookup table, which helps reduce the computational burden on the display device and improve rendering latency. As a result, the display device can perform parallax compensation in a timely manner in dynamic scenes where the user's head is moving, thus improving image jitter or ghosting caused by computational latency.
[0053] Furthermore, the first virtual image point in the first data lookup table is calibrated data, recording the relative true correspondence between the eye position and the virtual image point. For sub-pixels that may cause crosstalk, their virtual image points are also calibrated. Therefore, the first virtual image point in the first data lookup table is relatively accurate data. Based on this, the parallax compensation disclosed herein can more accurately correct the virtual image shift caused by changes in eye position, thereby improving the clarity of the display product and enhancing the display effect.
[0054] Figure 6 The diagram shown is a schematic representation of a method for generating a first data lookup table according to an embodiment of this disclosure. Please refer to it. Figure 4 and Figure 6 In one optional embodiment of this disclosure, the image generation method of the display device further includes generating a first data lookup table; the method for generating the first data lookup table includes: Step S310: Provide a center ray data table; wherein, the center ray data table includes center ray information for each sub-pixel. The center ray refers to the ray along the central axis of the beam emitted from a sub-pixel, that is, the ray propagating along the main emission direction at the geometric center of the sub-pixel beam.
[0055] In one optional embodiment of this disclosure, the central ray information includes at least the position coordinates of the second virtual image point corresponding to the central ray of the beam emitted by each sub-pixel on the virtual image plane 10, and the landing position of the central ray of the beam emitted by each sub-pixel in the eyebox. Wherein: The second virtual image point refers to the intersection of the backward extension of the central ray emitted by a sub-pixel after propagation through the optical system and the virtual image surface 10. This intersection point represents the position on the virtual image surface 10 from which the light ray from that sub-pixel is perceived by the human eye under reference conditions. Each sub-pixel on the display panel 210 corresponds to one second virtual image point. In other words, the central ray data table records the set of correspondences between sub-pixels and the coordinates of the second virtual image points.
[0056] The location where the central ray falls within the eyebox refers to the position where the central ray emitted by a sub-pixel intersects the eyebox plane after propagation through the optical system. For each sub-pixel on the display panel 210, its central ray corresponds to a specific eyebox location. In other words, the central ray data table also records a set of correspondences between sub-pixels and eyebox locations.
[0057] It should be noted that Table 1 below is an example of a center ray data table provided in an embodiment of this disclosure. The first column of the table enumerates all sub-pixels, the second column represents the position of the second virtual image point corresponding to each sub-pixel along the X direction of the virtual image plane 10, the third column represents the position of the second virtual image point corresponding to each sub-pixel along the Y direction of the virtual image plane 10, and the fourth column represents the landing point position of the center ray of each sub-pixel in the eyebox. This disclosure uses this table as an example for illustration only and is not intended to be limiting.
[0058] Table 1 Center Ray Data Table
[0059] Please continue to refer to this. Figure 4 , Figure 6 And Table 1, step S320, generating a first data lookup table based on the central ray data table.
[0060] It should be noted that the central ray data table records the virtual image point corresponding to the central ray of each sub-pixel and the landing position of the central ray in the eyebox. The data in the central ray data table is related to the hardware conditions of the display device. When the display device is fixed, the data in the central ray data table is fixed. However, when the eye position changes, the light rays incident on the eye will change, which will cause the intersection point of the backward extension of the light rays and the virtual image surface 10 to change, that is, the virtual image point corresponding to the light rays incident on the eye will change. In this embodiment, a first data lookup table is generated based on the central ray data table. The first virtual image point recorded in the first data lookup table is calibrated data, which records the relative true correspondence between the eye position and the virtual image point. For sub-pixels that may cause crosstalk, their virtual image points are also calibrated. Therefore, the first virtual image point in the first data lookup table is relatively accurate data. The parallax compensation performed by this disclosure can more accurately correct the virtual image shift caused by changes in eye position, thereby improving the clarity of the display product and improving the display effect.
[0061] Figure 7 The diagram shown is a schematic representation of step S320 provided in an embodiment of this disclosure. Please refer to it. Figure 4 , Figure 6 and Figure 7 In one optional embodiment of this disclosure, step S320, the method of generating a first data lookup table based on the center ray data table, includes: Step S321: In the eyebox, set multiple first value points arranged along the first direction F1; wherein, the first direction F1 is parallel to the direction of the line connecting the two eyes of the same user.
[0062] Step S322: Based on the central ray data table, calculate the coordinates of the first virtual image point corresponding to each sub-pixel at different first value points in the virtual image plane 10.
[0063] In this embodiment, multiple first value points are set along the first direction F1 in the eyebox, and the coordinates of the first virtual image point corresponding to each sub-pixel are calculated for each first value point, thereby constructing a first data lookup table covering multiple eye position variation ranges. Table 2 is an example of the first data lookup table provided in this embodiment. The first column of the table enumerates each sub-pixel in the display panel 210, the second column records the landing position of the center ray of each sub-pixel in the eyebox, and the third to the last column record the coordinates of the first virtual image point of each sub-pixel under different first value points.
[0064] Table 2 First Data Lookup Table
[0065] In step S321, multiple first value points are set along a first direction F1. The first direction F1 can be understood as the lateral movement direction of the user's eyes, or the left-right movement direction of the user's eyes. The first value points can represent the possible spatial position of the user's eyes. The number of first value points will determine the number of "columns" in the first data lookup table.
[0066] In practical use, the user's eye position changes continuously, but it is difficult to exhaustively list all positions when setting the first value point, which would also generate a huge amount of data. Therefore, when setting the first value point, the value can be selected according to actual needs.
[0067] This disclosure provides an optional implementation in which the distance between two adjacent first value points is equal. This configuration ensures that the first value points are evenly covered along the first direction F1, which helps to balance the rendering effect at different locations and ensures that the first value points can be set in all areas of the viewport Eyebox. This allows for the acquisition of corresponding first virtual image points based on the first value points, and then rendering based on the image information of the first virtual image points, thus contributing to a balanced rendering effect.
[0068] This disclosure provides an optional implementation in which the distance between two adjacent first value points is not equal. This arrangement allows for a denser arrangement of the first value points in areas where improved rendering is desired. For example, the distance between adjacent first value points can be reduced in the primary areas of user eye activity, while the distance can be increased in secondary areas of user eye activity.
[0069] Optionally, the distance between adjacent first value points is L1, where 3mm ≤ L1 ≤ 10mm. It should be noted that when the distance between adjacent first value points is small, such as L1 < 3mm, the number of first value points is large, leading to excessive data volume. This affects both the generation efficiency of the first data lookup table and the memory consumption, impacting rendering efficiency. When the distance between adjacent first value points is large, such as L1 > 10mm, the actual distance between the user's eye and the first value point may be too far, resulting in minimal improvement to the display effect. Therefore, this disclosure sets the distance L1 between adjacent first value points to 3mm ≤ L1 ≤ 10mm. This setting helps balance data volume and display effect. In other words, when the distance between adjacent first value points is 3mm ≤ L1 ≤ 10mm, it helps reduce data volume, improves the generation and rendering efficiency of the first data lookup table, and enhances the improvement effect on the display product.
[0070] This disclosure provides an optional embodiment in which the distance L1 between adjacent first value points is 5mm; another optional embodiment in which the distance L1 between adjacent first value points is 7mm; yet another optional embodiment in which the distance L1 between adjacent first value points is 9mm; yet another optional embodiment in which the distance 4mm≤L1≤8mm between adjacent first value points; and yet another optional embodiment in which the distance 6mm≤L1≤10mm between adjacent first value points.
[0071] In step S322, for each first value point, the coordinates of the first virtual image point corresponding to each sub-pixel at that eye position need to be calculated. The calculation is based on the central ray information recorded in the central ray data table. Thus, the first data lookup table records the coordinates of the first virtual image point corresponding to each sub-pixel at each first value point.
[0072] During rendering, based on the eye position, a first value point that is the same as or closest to the eye position is found. Based on the data in the column containing this first value point, the corresponding image information of the first virtual image point is assigned to each sub-pixel. For example, along the first direction F1, the eye position is at -65mm. The first data lookup table includes a first value point, e1 = -65mm. Each sub-pixel is then assigned a value based on this column of data. For instance, for sub-pixel (m, n), the coordinates of the corresponding first virtual image point are (X_e1, Y_e1). The sub-pixel is then assigned a value based on the image information corresponding to the first virtual image point. During assignment, the table is re-looked up and a value is assigned for each sub-pixel. This setup allows the virtual image surface 10 to be updated promptly when the eye position changes, resulting in a clearer image for the user. This improves the display effect of the display product and enhances the user experience.
[0073] Figure 8 The diagram shown is a schematic representation of step S322 provided in an embodiment of this disclosure. Please refer to it. Figure 4 , Figure 7 , Figure 8 In conjunction with Tables 1 and 2, this disclosure provides a specific implementation of step S322. In this implementation, step S322, the method for calculating the first virtual image point corresponding to each sub-pixel at different first value points based on the center ray data table, includes, but is not limited to, steps S3221 to S3223, wherein: Step S3221: Obtain the sub-pixel to be tested and the first value point; wherein, the sub-pixel to be tested is any sub-pixel in the first data lookup table.
[0074] Step S3222: Search for the target sub-pixel in the center ray data lookup table; wherein, the landing point of the center ray of the beam emitted by the target sub-pixel in the eyebox coincides with the first value point. Step S3223: When the center ray data lookup table includes the target sub-pixel, the second virtual image point of the target sub-pixel is reused as the first virtual image point of the sub-pixel to be measured.
[0075] This implementation utilizes the center ray information of sub-pixels already in the center ray data table. By matching the landing point positions, the second virtual image point of the target sub-pixel is directly reused as the first virtual image point of the sub-pixel to be tested. This helps to avoid repetitive geometric calculations and improves the efficiency of lookup table generation.
[0076] Specifically, in step S3221, the sub-pixel to be tested is any sub-pixel in the first data lookup table. In other words, the sub-pixel to be tested is the object whose first virtual image point coordinates need to be calculated. During the generation of the first data lookup table, it is necessary to traverse every sub-pixel on the display panel 210 (or at least one sub-pixel that needs to participate in rendering) and calculate the coordinates of the first virtual image point corresponding to different first value points one by one. The sub-pixel currently being processed is the sub-pixel to be tested. The sub-pixel to be tested can be identified by its row and column coordinates or number on the display panel 210, for example, sub-pixel (m, n), where m represents the row number and n represents the column number.
[0077] The first value point is one of several value points arranged along the first direction F1 in the Eyebox. The first direction F1 is parallel to the line connecting the user's two eyes, which is usually horizontal. The first value point represents a possible spatial position of the user's eyes. During the generation of the first data lookup table, each first value point needs to be traversed, and the coordinates of the first virtual image point corresponding to each sub-pixel need to be calculated for each first value point. The first value point currently being processed is the first value point obtained in this step.
[0078] In step S3222, the target sub-pixel is located. The target sub-pixel refers to the sub-pixel recorded in the center ray data table whose center ray falls on the Eyebox of the viewport at a position that coincides with the current first value point. In other words, the center ray of the target sub-pixel falls on the Eyebox of the viewport at a position that is exactly equal to the coordinates of the currently obtained first value point.
[0079] In step S3223, when the center ray data lookup table includes the target sub-pixel, the second virtual image point of the target sub-pixel is reused as the first virtual image point of the sub-pixel to be measured.
[0080] It should be noted that the second virtual image point recorded in the center ray data table is the virtual image point position corresponding to the center ray of the target sub-pixel under the baseline conditions. The landing position of this center ray in the eyebox of the view area coincides with the current first value point, which means that when the user's eye is located at the first value point, the center ray of the target sub-pixel will enter the eye.
[0081] Since the virtual image point position of the sub-pixel under test also needs to be determined at the current first value point, and the center ray point of the target sub-pixel coincides with the first value point, the second virtual image point of the target sub-pixel can be directly used as the first virtual image point of the sub-pixel under test at the first value point.
[0082] In other words, at a certain eye position, if the central ray of a target sub-pixel happens to be directed towards that position, then the position of the virtual image point corresponding to the central ray of that target sub-pixel can represent the equivalent virtual image point position of the sub-pixel under test at that position. That is to say, when the target sub-pixel can be found in the central ray data table, and the position of the central ray of the target sub-pixel coincides with the first value point, the central ray data table can be considered as the first data lookup table.
[0083] The specific operation of reuse is to directly assign the coordinates of the second virtual image point recorded by the target sub-pixel in the central ray data table to the coordinates of the first virtual image point of the sub-pixel under the current first value point.
[0084] In this embodiment, during the generation of the first data lookup table, the system first checks whether the target sub-pixel is included in the central ray data table. If the target sub-pixel is included in the central ray data table, the system directly performs reuse and assignment operations. This setting helps to reduce the amount of computation during the generation of the lookup table and improves the efficiency of the lookup table generation.
[0085] Figure 9 The diagram shown is another schematic representation of step S322 provided in this embodiment of the present disclosure. Please refer to [the diagram]. Figure 4 , Figure 7 , Figure 9 In addition to Tables 1 and 2, when no target sub-pixel is found in the center ray data table, this disclosure provides another optional implementation method, wherein step S322, the method for calculating the first virtual image point corresponding to each sub-pixel at different first value points according to the center ray data table, further includes: Step S3224: When the target sub-pixel is not included in the center ray data lookup table, obtain the lookup group corresponding to the sub-pixel to be measured; Step S3225: In the search group, obtain the first reference sub-pixel and the second reference sub-pixel corresponding to the sub-pixel to be tested; wherein, the landing position of the first reference sub-pixel and the landing position of the second reference sub-pixel are both adjacent to the corresponding first value point. Step S3226: Calculate the first virtual image point corresponding to the sub-pixel to be tested based on the second virtual image point of the first reference sub-pixel and the second virtual image point of the second reference sub-pixel.
[0086] This embodiment provides a method that excludes the target sub-pixel from the center ray data lookup table, i.e., there are no other sub-pixels whose center rays fall at positions in the eyebox that coincide with the current first value point. In this case, by finding two sub-pixels adjacent to the first value point and performing interpolation calculations based on their second virtual image points, the first virtual image point corresponding to the sub-pixel to be measured is obtained.
[0087] Specifically, when the target sub-pixel is not included in the center ray data lookup table, the coordinates of the first virtual image point corresponding to the sub-pixel to be measured need to be calculated using other methods. In step S3224, the lookup group corresponding to the sub-pixel to be measured is obtained, thereby enabling the search for the required first and second reference sub-pixels within the lookup group. This setting helps to narrow the search range and improve the efficiency of generating the first data lookup table.
[0088] In step S3225, the first reference sub-pixel and the second reference sub-pixel corresponding to the sub-pixel to be tested are obtained from the search group. The first reference sub-pixel and the second reference sub-pixel are the two sub-pixels in the search group whose Eyebox landing point position is closest to the current first value point position. It should be noted that the first reference sub-pixel and the second reference sub-pixel are sub-pixels whose center ray information has been recorded in the center ray data table, while the sub-pixel to be tested is the sub-pixel whose first virtual image point needs to be calculated. The first reference sub-pixel, the second reference sub-pixel, and the sub-pixel to be tested can be the same sub-pixel or different sub-pixels. In most cases, the three are different sub-pixels, but this disclosure does not exclude the possibility of overlap.
[0089] In step S3226, the first virtual image point of the sub-pixel to be tested at the current first value point is calculated using the second virtual image point of the first reference sub-pixel and the second virtual image point of the second reference sub-pixel. In an optional embodiment of this disclosure, step S3226, the method of calculating the first virtual image point corresponding to the sub-pixel to be tested based on the second virtual image point of the first reference sub-pixel and the second virtual image point of the second reference sub-pixel, specifically involves: calculating the first virtual image point corresponding to the sub-pixel to be tested using linear interpolation based on the second virtual image point of the first reference sub-pixel and the second virtual image point of the second reference sub-pixel.
[0090] It should be noted that the position of the virtual image point changes continuously with the position of the eyebox landing point in the view area. When the eyebox landing points of two sub-pixels are located on both sides of and adjacent to the first value point, the first virtual image point of the sub-pixel under the first value point can be obtained by interpolation of the second virtual image points of these two sub-pixels. For example, the landing point of the first reference sub-pixel in the eyebox is S_L, and the coordinates of the second virtual image point corresponding to the first reference sub-pixel are (X_L, Y_L). The landing point of the first reference sub-pixel in the eyebox is S_R, and the coordinates of the second virtual image point corresponding to the first reference sub-pixel are (X_R, Y_R). The current first value point is ei, and S_L < ei < S_R. The coordinates (X_e, Y_e) of the first virtual image point of the sub-pixel to be tested are calculated as follows: X_e = X_L + (X_R - X_L) × (ei - S_L) / (S_R - S_L), Y_e = Y_L + (Y_R - Y_L) × (ei - S_L) / (S_R - S_L).
[0091] The landing points of both the first and second reference sub-pixels are adjacent to the current first value point, meaning their light rays are spatially close within the eyebox. In optical systems, light rays with similar spatial positions have a strong correlation between their propagation paths and virtual image point positions. Therefore, interpolation based on the second virtual image points of these two sub-pixels yields the first virtual image point of the sub-pixel to be measured. This implementation complements the target sub-pixel reuse scheme. When a precisely matching target sub-pixel exists, the target sub-pixel reuse scheme improves compensation accuracy; when a precisely matching target sub-pixel does not exist, an interpolation scheme is used. Combining these two schemes improves both the generation efficiency and the completeness of the first data lookup table.
[0092] Please refer to Figure 1 and Figure 2 In one optional embodiment of this disclosure, the display device includes a display panel 210 and a plurality of prisms 220, the prisms 220 being located on the light-emitting side of the display panel 210; the prisms 220 are used to convert the planar image of the display panel 210 into a stereoscopic image, and to project part of the stereoscopic image to the left eye and part to the right eye. The number of sub-pixels corresponding to a single prism 220 is p. The lookup group includes the corresponding sub-pixel and k sub-pixels arranged in the same row as the sub-pixel, respectively located on both sides of the sub-pixel; wherein, the two sides of the sub-pixel are the two sides of the sub-pixel along the arrangement direction of the prisms 220. The lookup group includes 2k+1 sub-pixels; where k is a positive integer; 2k+1>p.
[0093] Specifically, the display panel 210 includes a plurality of sub-pixels arranged in an array, and each prism 220 corresponds to at least one sub-pixel setting of the display panel 210. The number of sub-pixels corresponding to a single prism 220 is p, where p is a positive integer. The value of p can be determined based on the size of the prism 220, the size of the sub-pixels, and the resolution of the display panel 210. For example, p can be 5, 8, 11, or more. The arrangement direction of the prisms 220 is generally consistent with the arrangement direction of the sub-pixel columns of the display panel 210. In a specific example, the prisms 220 are arranged along the horizontal direction (row direction) of the display panel 210, and each prism 220 corresponds to one or more sub-pixel columns.
[0094] In one optional embodiment of this disclosure, when the target sub-pixel is not included in the center ray data lookup table, a lookup group corresponding to the sub-pixel to be tested needs to be obtained. The lookup group includes the corresponding sub-pixel and k sub-pixels arranged in the same row as the sub-pixel, located on either side of the sub-pixel. The corresponding sub-pixel refers to the sub-pixel to be tested itself. The lookup group is constructed centered on the sub-pixel to be tested. Taking the position of the sub-pixel to be tested on the display panel 210 as the i-th row and j-th column, with the prisms 220 arranged horizontally as an example, the lookup group includes sub-pixels from the jk-th column to the j+k-th column of the i-th row, totaling 2k+1 sub-pixels. Among them, the sub-pixels from the jk-th column to the j-1-th column are located on one side of the sub-pixel to be tested, and the sub-pixels from the j+1-th column to the j+k-th column are located on the other side of the sub-pixel to be tested. The lookup group includes 2k+1 sub-pixels, and 2k+1>p. That is, the size of the lookup group is greater than the number of sub-pixels corresponding to a single prism 220. For example, a single prism 220 corresponds to 10 sub-pixels, k is 5, and the lookup group includes 2×5+1 sub-pixels in total.
[0095] In a 3D display device comprising multiple prisms 220, each prism 220 deflects the light direction of its corresponding sub-pixel. The central light rays of sub-pixels corresponding to different prisms 220 may have significant gaps at their landing points in the eyebox. If the search group is limited to sub-pixels corresponding to a single prism 220 (i.e., 2k+1≤p) or is small in number, when the first value point falls between the landing points of sub-pixels corresponding to two prisms 220, the search group may not contain sub-pixels located on either side of the first value point, leading to interpolation failure. This embodiment sets 2k+1>p, enabling the search group to span at least two sub-pixels corresponding to prisms 220, thereby covering the landing point gaps at the boundaries of the prisms 220. Therefore, even if the first value point falls between the landing points of two prisms 220, the search group can still provide adjacent sub-pixels located on either side, ensuring the continuity and accuracy of the interpolation calculation. Meanwhile, the deflection effect of the prism 220 causes the distribution of light rays from different sub-pixels in the eyebox to be non-uniform and continuous, exhibiting discrete characteristics related to the structure of the prism 220. With a sufficiently large number of sub-pixels in the lookup group, the light emitted by the sub-pixels can cover various viewing angles. This setup facilitates finding the first and second reference sub-pixels that are relatively closer to the first value point in the lookup group, thereby improving the accuracy of the first virtual image point corresponding to the sub-pixel in the first data lookup table, and thus further improving the display effect.
[0096] Figure 10 The diagram shown is another schematic representation of step S100 provided in this embodiment of the present disclosure. Please refer to [the diagram]. Figure 3 , Figure 4 , Figure 5 , Figure 10 And Table 2, in an optional embodiment of this disclosure, the first data lookup table includes multiple sub-lookup tables. It should be noted that the table shown in Table 2 can be considered as one of the sub-lookup tables in the first data lookup table. Step S110: After providing the first data lookup table, the image generation method further includes: Step S130: Monitor eye movement and obtain eye position.
[0097] Step S140: Determine the corresponding sub-lookup table based on the eye position.
[0098] In this embodiment, the first data lookup table is divided into multiple sub-lookup tables according to the range of eye positions. In actual use, the eye position is obtained by monitoring the movement of the eyes, and the sub-lookup table to be called is determined accordingly.
[0099] Specifically, in step S130, the movement state of the eyes is monitored. For example, taking a 3D HUD as the display device, an eye position detection structure can be installed on the steering wheel to monitor the driver's eye movement state in real time. The eye position detection structure can be an infrared camera, a stereo vision camera, or other eye position detection devices commonly used in the art. The monitored content includes at least the spatial position of the eyes within the eyebox; for example, the coordinates of the eyes along a first direction F1 and along a second direction F2 within the eyebox are monitored.
[0100] In step S140, the corresponding sub-lookup table is determined based on the eye position.
[0101] Figure 11 The diagram shown is a schematic representation of a sub-lookup table provided in an embodiment of this disclosure. Please refer to it. Figure 4 and Figure 11 In one optional embodiment of this disclosure, the method for setting a sub-lookup table includes: step S141, setting a plurality of second value points arranged along a second direction F2 in the eyebox; wherein, the second direction F2 is the direction in which the eye moves up and down in the eyebox. Step S142, generating a corresponding sub-lookup table based on the second value points. In this embodiment, a plurality of second value points are set along the second direction F2 in the eyebox, wherein, the second direction F2 is the direction in which the eye moves up and down in the eyebox, and then a corresponding sub-lookup table is generated based on each second value point. Thus, the first data lookup table is divided into a plurality of sub-lookup tables according to the position of the eye in the vertical direction, and each sub-lookup table corresponds to a vertical eye position interval.
[0102] The second direction, F2, represents the vertical direction in which the eyes move up and down within the eyebox, perpendicular to the line connecting the user's two eyes. For example, when the display device is a 3D HUD used in a vehicle, the driver's eyes not only move laterally along the line connecting the eyes (horizontal direction) but also change position vertically. For instance, adjusting seat height, changing posture, or experiencing vehicle bumps or inclines will all cause changes in eye position vertically. The second direction, F2, is perpendicular to the first direction, F1. The first direction, F1, is horizontal (parallel to the line connecting the eyes), and the second direction, F2, is vertical. Together, they constitute a two-dimensional coordinate system within the eyebox plane.
[0103] Within the viewport (Eyebox), multiple second value points are set along the second direction F2. These second value points represent multiple discrete eye position sampling points selected within the vertical variation range of the eye position. Specifically, multiple second value points can be set at equal intervals along the second direction F2 within the viewport (Eyebox), or they can be set at unequal intervals along the second direction F2 within the viewport (Eyebox). Similar to the setting method of the first value points, when the distance between two adjacent second value points is equal, the second value points are evenly covered along the second direction F2, which helps to balance the rendering effect at different positions and ensures that second value points can be set in all areas of the viewport (Eyebox) as much as possible. This allows for the acquisition of corresponding sub-lookup tables based on the second value points, and then rendering based on the image information of the first virtual image point corresponding to each sub-pixel in the sub-lookup table, thus contributing to a balanced rendering effect. When the distance between two adjacent second value points is not equal, the second value points can be arranged more densely in areas where improved rendering effect is desired.
[0104] Optionally, the distance between adjacent second value points is L2, where 1mm≤L2≤10mm.
[0105] Specifically, when the distance between adjacent second value points is small, for example, when the distance L2 between adjacent second value points is < 1mm, the number of second value points is large, resulting in a large number of sub-lookup tables, excessive data volume, and excessive memory consumption, thus affecting rendering efficiency. When the distance between adjacent second value points is large, for example, when the distance L2 between adjacent second value points is > 10mm, there may be situations where the actual position of the user's eye is far from the second value point, resulting in the inability to obtain a more accurate sub-lookup table, thus affecting the display effect. Therefore, this disclosure sets the distance L2 between adjacent second value points to 1mm ≤ L2 ≤ 10mm. This setting is beneficial for balancing data volume and display effect. In other words, when the distance between adjacent second value points is 3mm ≤ L2 ≤ 10mm, it is beneficial for reducing data volume, improving rendering efficiency, and also for improving the display product's performance.
[0106] This disclosure provides an optional embodiment in which the distance L2 between adjacent second value points is 2mm; another optional embodiment in which the distance L2 between adjacent second value points is 4mm; yet another optional embodiment in which the distance L2 between adjacent second value points is 9mm; yet another optional embodiment in which the distance 3mm≤L2≤8mm between adjacent second value points; and yet another optional embodiment in which the distance 1mm≤L2≤7mm between adjacent second value points.
[0107] Please refer to Figure 3and Figure 4 In one optional embodiment of this disclosure, the rendering information further includes viewpoint sampling points and image information. Step S100, which obtains rendering information based on eye position, further includes: determining viewpoint sampling points based on eye position; and acquiring image information at the viewpoint sampling points.
[0108] Step S200 renders and generates an image based on the rendering information, including: assigning values to the corresponding sub-pixels based on the image information and the first virtual image point.
[0109] Specifically, when acquiring rendering information, in addition to obtaining the first virtual image point, it is also necessary to determine the viewpoint sampling point and acquire image information. The viewpoint sampling point refers to the virtual observation point in the 3D scene used to acquire image information. For example, in the rendering process of a 3D HUD, a virtual camera model needs to be constructed based on the driver's eye position to acquire image information from the 3D scene. The viewpoint sampling point is the position of this virtual camera in the 3D scene space. Optionally, since the 3D HUD needs to project images with parallax to the left and right eyes respectively, the viewpoint sampling points typically include a left-eye viewpoint sampling point and a right-eye viewpoint sampling point. There is a horizontal offset between the left-eye and right-eye viewpoint sampling points corresponding to the driver's interpupillary distance. In actual operation, the actual position of the driver's eyes in the Eyebox can be directly used as the position of the viewpoint sampling point.
[0110] Image information refers to two-dimensional image information acquired from a 3D scene at a viewpoint sampling point. This image information describes the projection of each element in the scene onto the imaging plane when viewed from the viewpoint sampling point. Image information may include: the geometry, lighting, color, brightness, and transparency of the elements to be displayed in the scene.
[0111] After determining the corresponding rendering information (image information and the first virtual image point), values are assigned to the corresponding sub-pixels based on the rendering information. Specifically, it is determined which position of the image in the virtual image surface 10 a given sub-pixel should display, so that after the sub-pixel is projected by the optical system, the expected image content is presented at its corresponding first virtual image point. Further, for each sub-pixel on the display panel 210, the coordinates of its corresponding first virtual image point at the current eye position are known. At the same time, the expected positional distribution of the image information to be displayed on the virtual image surface 10 is known. By matching the coordinates of the first virtual image point with the image information, the pixel value that the sub-pixel should display can be determined.
[0112] In one specific embodiment, the image generation process of the display device is as follows: (1) Obtain the eye position. The information corresponding to the eye position includes the coordinates of the left eye, the coordinates of the right eye, and the center coordinates. The center coordinates are calculated based on the coordinates of the left eye and the right eye. For example, the coordinates of the left eye are (sL, tL), the coordinates of the right eye are (sR, tR), and the center coordinates are (sm, tm), where sm = (sL + sR) / 2 and tm = (tL + tR) / 2.
[0113] (2) Determine the sub-lookup table based on the eye position. Specifically, find the second value point that is closest to the tm value of the center coordinate. The tm value represents the coordinate of the eye position along the second direction F2 (vertical direction). Each second value point corresponds to a sub-lookup table. Find the second value point that coincides with or is closest to the tm value to determine the corresponding sub-lookup table for subsequent rendering.
[0114] (3) Determine the viewpoint sampling point based on the eye position. Specifically, the left and right eye positions are used as viewpoint sampling points, and image information is collected at the viewpoint sampling points. That is, when observing the 3D scene from the viewpoint sampling point, the projection of each element in the scene onto the imaging plane is obtained. In other words, the image that the user's eyes should see at the current position is determined.
[0115] (4) Determine whether each sub-pixel displays the left-eye image or the right-eye image. Specifically, refer to the data in the second column (s column) of the sub-lookup table to determine whether the sub-pixel displays the left-eye image or the right-eye image. For example, when the s value of the sub-pixel is less than the sm value of the center coordinate, the sub-pixel displays the left-eye image; when the s value of the sub-pixel is greater than the sm value of the center coordinate, the sub-pixel displays the right-eye image; when the s value of the sub-pixel is equal to the sm value of the center coordinate, the sub-pixel displays either the left-eye image or the right-eye image.
[0116] (5) Find the closest first value point in the sub-lookup table based on the eye coordinates and determine the corresponding first virtual image point. Specifically, when the sub-pixel displays the left eye image, find the first value point closest to the sL value of the left eye coordinates and obtain the corresponding first virtual image point. When the sub-pixel displays the right eye image, find the first value point closest to the sR value of the right eye coordinates and obtain the corresponding first virtual image point.
[0117] (6) Assign values to sub-pixels based on image information and the first virtual image point.
[0118] In the image generation method provided in this disclosure, rendering information is obtained based on the eye position, which facilitates timely adjustment of the displayed image according to the user's eye position, allowing the user to observe a clearer image. Specifically, the first virtual image point in the rendering information is pre-calibrated, and adjustments are made to the first virtual image points of some interfering sub-pixels. This adjusts the content displayed by the interfering sub-pixels, thereby improving the blurriness of the generated image, enhancing clarity, and ultimately improving the display effect of the display product.
[0119] Please refer to Figure 1 Based on the same inventive concept, this disclosure provides a display device that generates an image using any of the image generation methods provided in the embodiments of this disclosure.
[0120] The display device provided in this disclosure can be a 3D HUD or any other electronic device with 3D display functionality similar to a 3D HUD. The display device provided in this disclosure generates images using any of the image generation methods provided in this disclosure, and has the beneficial effects of the image generation methods provided in this disclosure. For details, please refer to the specific descriptions of the image generation methods for the display device in the above embodiments; these will not be repeated here.
[0121] As can be seen from the above embodiments, the display device and image generation method provided in this disclosure achieve at least the following beneficial effects: This disclosure provides a display device and an image generation method thereof. The image generation method includes acquiring rendering information based on eye position and rendering based on the rendering information to generate an image. The rendering information includes a first virtual image point, a central ray of a first sub-pixel corresponding to the first virtual image point, and a non-central ray of a second sub-pixel corresponding to a second virtual image point. The central ray of the first sub-pixel corresponding to the first virtual image point means that the intersection of the backward extension of the central ray of the first sub-pixel with the virtual image plane is the first virtual image point; the non-central ray of the second sub-pixel corresponding to the second virtual image point means that the intersection of the backward extension of at least one non-central ray of the second sub-pixel with the virtual image plane is the first virtual image point. In this disclosure, both the first and second sub-pixels are rendered according to the image information of the first virtual image point. With this configuration, the content presented by the central ray of the beam emitted by the first sub-pixel and the non-central ray of the beam emitted by the second sub-pixel received by the user's eye is the same, thus helping to avoid crosstalk between them, improving the problem of blurry generated images, enhancing clarity, and thus improving the display effect of the display product.
[0122] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0123] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An image generation method for a display device, characterized in that, include: Rendering information is obtained based on the eye position; wherein, the rendering information includes a first virtual image point; the multiple sub-pixels of the display device include a first sub-pixel and a second sub-pixel, the center ray of the first sub-pixel corresponds to the first virtual image point, and at least one non-center ray of the second sub-pixel corresponds to the first virtual image point; Rendering is performed based on the rendering information to generate an image.
2. The image generation method for the display device according to claim 1, characterized in that, The non-center ray of the second sub-pixel overlaps with the center ray of the first sub-pixel.
3. The image generation method for the display device according to claim 1, characterized in that, When the eye position changes, at least one of the first virtual image points corresponding to the sub-pixel changes.
4. The image generation method for the display device according to claim 1, characterized in that, The display device is a 3D head-up display device; The display device includes a display panel and a plurality of prisms, the prisms being located on the light-emitting side of the display panel; the display panel is used to display planar images, and the prisms are used to transform the planar images of the display panel into stereoscopic images, and to project part of the stereoscopic image to the left eye and part to the right eye.
5. The image generation method for the display device according to claim 1, characterized in that, The method for obtaining rendering information based on eye position includes: Provide the first data lookup table; The first virtual image point corresponding to the eye position is obtained according to the first data lookup table.
6. The image generation method for the display device according to claim 5, characterized in that, It also includes generating the first data lookup table; The method for generating the first data lookup table includes: A center ray data table is provided; wherein, the center ray data table includes center ray information for each of the sub-pixels; The first data lookup table is generated based on the central ray data table.
7. The image generation method for the display device according to claim 6, characterized in that, The central ray information includes at least the position coordinates of the second virtual image point corresponding to the central ray of the beam emitted by each sub-pixel on the virtual image plane, and the landing point position of the central ray of the beam emitted by each sub-pixel in the view area.
8. The image generation method for the display device according to claim 7, characterized in that, The method for generating the first data lookup table based on the central ray data table includes: In the visual area, a plurality of first value points are set along a first direction; wherein, the first direction is parallel to the direction of the line connecting the two eyes of the same user; Based on the central ray data table, calculate the coordinates of the first virtual image point corresponding to each sub-pixel at different first value points in the virtual image plane.
9. The image generation method for a display device according to claim 8, characterized in that, The distance between any two adjacent first value points is equal.
10. The image generation method of the display device according to claim 8, characterized in that, The method for calculating the first virtual image point corresponding to each sub-pixel at different first value points based on the central ray data table includes: Obtain the sub-pixel to be tested and the first value point; wherein, the sub-pixel to be tested is any sub-pixel in the first data lookup table; The target sub-pixel is searched in the central ray data lookup table; wherein the central ray of the beam emitted by the target sub-pixel coincides with the first value point at the landing point in the view area. When the center ray data lookup table includes a target sub-pixel, the second virtual image point of the target sub-pixel is reused as the first virtual image point of the sub-pixel to be tested.
11. The image generation method for the display device according to claim 10, characterized in that, The method for calculating the first virtual image point corresponding to each sub-pixel at different first value points based on the central ray data table includes: When the target sub-pixel is not included in the center ray data lookup table, obtain the lookup group corresponding to the sub-pixel to be tested; In the search group, the first reference sub-pixel and the second reference sub-pixel corresponding to the sub-pixel to be tested are obtained; wherein, the landing position of the first reference sub-pixel and the landing position of the second reference sub-pixel are both adjacent to the corresponding first value point. The first virtual image point corresponding to the sub-pixel under test is calculated based on the second virtual image point of the first reference sub-pixel and the second virtual image point of the second reference sub-pixel.
12. The image generation method for the display device according to claim 11, characterized in that, The method for calculating the first virtual image point corresponding to the sub-pixel to be tested based on the second virtual image point of the first reference sub-pixel and the second virtual image point of the second reference sub-pixel is as follows: Based on the second virtual image point of the first reference sub-pixel and the second virtual image point of the second reference sub-pixel, the first virtual image point corresponding to the sub-pixel to be tested is calculated using linear interpolation.
13. The image generation method for the display device according to claim 11, characterized in that, The display device includes a display panel and a plurality of prisms, the prisms being located on the light-emitting side of the display panel; the prisms are used to transform the planar image of the display panel into a stereoscopic image, and to project part of the stereoscopic image to the left eye and part to the right eye; The number of sub-pixels corresponding to a single prism is p; The search group includes a sub-pixel corresponding to the sub-pixel and k sub-pixels arranged in the same row as the sub-pixel, located on both sides of the sub-pixel; wherein, the two sides of the sub-pixel are the two sides of the sub-pixel along the prism arrangement direction; The search group includes 2k+1 sub-pixels; where k is a positive integer; 2k+1>p.
14. The image generation method for a display device according to claim 8, characterized in that, The distance between adjacent first value points is L1, where 3mm ≤ L1 ≤ 10mm.
15. The image generation method for a display device according to claim 5, characterized in that, The first data lookup table includes multiple sub-lookup tables; After providing the first data lookup table, the image generation method further includes: Monitor eye movement to obtain the eye position; The corresponding sub-lookup table is determined based on the eye position.
16. The image generation method for a display device according to claim 15, characterized in that, The method for setting the sub-lookup table includes: Within the visual field, multiple second value points are set along a second direction; wherein, the second direction is the direction in which the eye moves up and down within the visual field. The corresponding sub-lookup table is generated based on the second value point.
17. The image generation method for a display device according to claim 16, characterized in that, The distance between adjacent second value points is L2, where 1mm ≤ L2 ≤ 10mm.
18. The image generation method of the display device according to claim 1, characterized in that, The rendering information also includes viewpoint sampling points and image information; The image generation method further includes: The viewpoint sampling point is determined based on the eye position; The image information is acquired at the viewpoint sampling point; The image information and the first virtual image point are used to assign values to the corresponding sub-pixels.
19. A display device, characterized in that, An image is generated using the image generation method of the display device according to any one of claims 1 to 18.
