High-efficiency near-eye light field display system and method for projecting virtual pixels using near-eye light field display system

Abstract

To improve effective resolution and collimation of a near-eye light field light-emitting display device.SOLUTION: The present disclosure relates to a near-eye light field display system comprising a light-emitting display device (1) comprising a plurality of display pixels (10, 12) and a lens array (14). Each display pixel (10, 12) has color information and is individually addressable to be set to inactive or active to generate a light beam (111). The lens array (14) comprises a plurality of lenses (140) and is configured to project the light beams (111) of the activated display pixels (10, 12) to form a projected virtual pixel image (26). The projected pixel image (26) has color information and a position in space determined by the number of activated display pixels (10, 12) and the position of the activated display pixels (10, 12) on the light emitting display (1). The present disclosure further relates to a method of projecting a projected virtual pixel with a near-eye light field display system.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to display devices, and more particularly to display devices that provide three-dimensional (3D) images with correct monocular depth cues, and more particularly to virtual and augmented reality goggles and glasses, and near-eye lightfield display devices for virtual and augmented reality applications. [Background technology]

[0002] Today's light-field display devices use, among other concepts, a conventional light-emitting two-dimensional (2D) display device in combination with an attached lens array or other optical element to accomplish the same thing. In such light-field display devices, each projected virtual pixel is represented by multiple actual display pixels (on the display device). This typically results in a reduction in the effective resolution of the overall system and causes redundancy of color information per virtual pixel, up to 30 to 60 times the redundancy. Another drawback is that the collimation (obtaining parallel light rays) performed by each lens on the light emitted by the actual pixel is imperfect, partly due to the necessarily imperfect optical properties of the lenses, but also due to diffraction at their small apertures. Summary of the Invention

[0003] The present disclosure relates to a near-eye light field display system using a two-dimensional light emitting display device with a lens array that overcomes the drawbacks and limitations of the state of the art at the time of filing.

[0004] The present disclosure relates to a near-eye light field display system comprising a light-emitting display device located between a near focal plane and a pupil plane, the light-emitting display device comprising a plurality of display pixels, each display pixel individually addressable with color information and configured to be inactivated or activated to generate a light beam toward the pupil plane. The light field display system further comprises a lens array positioned between the light emitting display device and the pupil plane, the lens array comprising an array of lenses configured to project the light beams generated by the actuated display pixels to form a projected virtual pixel image, the color information of the projected pixel image and its location in space being determined by the number of actuated display pixels and their locations on the light emitting display device.

[0005] The present disclosure further relates to a method for projecting a projected virtual pixel using a near-eye light field display system.

[0006] The present disclosure further relates to a non-transitory machine-readable medium storing machine-executable instructions that, when executed, cause a computing system to perform a method for projecting projected virtual pixels using a near-eye light field display system.

[0007] The near-eye light field display system disclosed herein requires minimal data and computational load to provide at least the same perceived quality as using known systems, using high-density emissive displays with low color resolution per pixel.

[0008] Exemplary embodiments of the invention are disclosed in the description and illustrated by the drawings. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 shows a near-eye light field display system including a light-emitting display device according to an embodiment. [Figure 2] FIG. 2 illustrates the near-eye light field display system of FIG. 1, showing the projection of light beams and projected virtual pixels of a virtual image, according to an embodiment. [Figure 3] Figure 3 shows the projection of two virtual pixels from actuated display pixels of an emissive display device (Figure 3a), the distribution of actuated display pixels that construct a first virtual pixel (Figure 3b) and a second virtual pixel (Figure 3c), and the combination of the distributions that construct the two virtual pixels according to the embodiment. [Figure 4] FIG. 4 illustrates a wearable device including a near-eye light field display system according to one embodiment. [Figure 5] FIG. 5 illustrates a method for projecting a projected virtual pixel using a near-eye light field display system showing the configuration of a subset of display pixels for first and second virtual pixels (FIG. 5a) and their distribution to individual physical sub-panels (FIG. 5b). DETAILED DESCRIPTION OF THE INVENTION

[0010] FIG. 1 illustrates a near-eye light field display system according to one embodiment. The light field display system includes an emissive display device 1 including a plurality of display pixels 10, 12, each of which carries color information and is addressable to either an "off" state (non-activated) in which it does not generate light, or an "on" state (activated) in which it generates a light beam 111 toward a pupil plane 130. In other words, the emissive display device 1 is a binary display because the light-emitting periods of the digital grayscale display are a binary display of "emission" and "non-emission." The light field display system further includes a lens array 14 disposed between the emissive display device 1 and the pupil plane 130. The lens array 14 includes an array of a plurality of lenses 140, each of which encompasses a subset of the plurality of display pixels 10, 12. The emissive display device 1 is two-dimensional.

[0011] The lenses 140 in the lens array 14 may comprise grating optical elements, refractive optical elements, Fresnel optical elements, holographic optical elements, or a combination of these optical elements.

[0012] 2, the lens array 14 is configured to project the light beams 111 generated by the activated display pixels 10, 12 to form projected virtual pixels 26 of the virtual image. The color information of the projected virtual pixels 26 and their positions relative to the pupil plane 130 are determined by the number of activated display pixels 10, 12 and their positions on the light emitting display device 1. For example, the activated display pixels are integrated on a flat light emitting display device 1.

[0013] The light field display system may include imaging optics 16 configured to project light beams 111 to form a projector image plane 22 located between lens array 14 and pupil plane 130. Imaging optics 16 deflects light beams 111 to intersect and form projected virtual pixels 26 in projected image plane 22.

[0014] The light field display system may further include projection optics 18 configured to project the light beam 111 so as to define an eyebox 20 at a pupil plane 130. The eyebox 20 defines the characteristics of the observer's eye, such as the expected diameter, orientation (i.e., rotation), and position of the pupil. The projection optics 18 may include an eyepiece or a combiner.

[0015] 2 further illustrates the configuration of the projection optics 18 of the virtual image as seen by the eyebox 20 and a single projected virtual pixel 26 in front of the eyebox virtual pixel 24. The projected virtual pixel 24 and the eyebox virtual pixel 26 are comprised of cross sections of the light beams 111 generated by the activated display pixels 10, 12. In particular, the projected virtual pixel 24 and the eyebox virtual pixel 26 each comprise the sum of the light beams 111 projected by each lens 140 whose corresponding display pixel 10, 12 is activated.

[0016] In one embodiment, lens array 14 comprises a row and column array comprising between 25 and 64 lenses 140. In such a configuration, projected virtual pixels 24 and eyebox virtual pixels 26 can be constructed from light beams 111 generated by actuated display pixels 10, 12, with each display pixel 10, 12 illuminating one lens 140 to form the cross-section of light beams 111 generated by 25 display pixels 10, 12.

[0017] The color of the projected virtual pixel 24 and the eyebox virtual pixel 26 is determined by the ratio of activated display pixels 10, 12 to inactivated display pixels 10, 12. The projected and eyebox color information of the virtual pixels 24, 26 is the sum of the color information of the activated display pixels 10, 12 on the light emitting display device 1.

[0018] Figure 3a shows the projection of a first projected virtual pixel 26a from a light beam 111 generated by a first set of actuated display pixels 10, 12 (grey dots), with each actuated display pixel 10, 12 of the first set being projected by a separate lens 140 of the lens array 14. Figure 3a also shows the projection of a second projected virtual pixel 26b from a light beam 111 generated by a second set of actuated display pixels 10, 12 (black dots), with each actuated display pixel 10, 12 of the second set being projected by a separate lens 140 of the lens array 14. In the example of Figure 3a, a lens array 14 comprising 5x5 lenses 140 is used for illustration. The first projected virtual pixel 26a and the second projected virtual pixel 26b are created in front of the projection optics 18 (not shown in Figure 3).

[0019] Each of the first and second projected virtual pixels 26a and 26b can correspond to a predefined ratio of actuated display pixels 10, 12 to inactive display pixels 10, 12. Each of the first and second projected virtual pixels 26a and 26b can correspond to a random distribution of actuated display pixels 10, 12 on the light-emitting display device 1. FIG. 3b shows an exemplary ratio and distribution of actuated display pixels 10, 12 on the light-emitting display device 1 that make up the first projected virtual pixel 26a. FIG. 3c shows an exemplary ratio and distribution of actuated display pixels 10, 12 on the light-emitting display device 1 that make up the second projected virtual pixel 26b. FIG. 3d shows a combination of the ratio and distribution of actuated display pixels 10, 12 on the light-emitting display device 1 that make up the first and second projected virtual pixels 26a and 26b. The combination of the first and second projected virtual pixels 26a, 26b can correspond to a full-color virtual pixel from a plurality of low-color resolution display pixels 10, 12 on the binary light-emitting display device 1.

[0020] In various embodiments, the display pixels 10, 12 may have 1 bit of color resolution (two colors, often black and white), where the projected virtual pixels 26a, 26b can have 26 gray levels per base color from the display pixels 10, 12 on the binary emissive display device 1. Alternatively, the display pixels 10, 12 may have 2 bits of color resolution, such that the projected virtual pixels 26a, 26b can have 104 grayscale levels. Because the display pixels 10, 12 have more than 1 bit of color resolution, the emissive display device 1 is no longer a binary display. The display pixels 10, 12 may have 3 bits of color resolution, such that the projected virtual pixels 26a, 26b can have 208 grayscale levels. The display pixels 10, 12 may even have more than 3 bits of color resolution. Projected virtual pixels 26a, 26b with 256 grayscale levels have, for example, 1-bit color resolution if lens array 14 comprises an array of 16x16 lenses 140 and display pixels 10, 12 have 1-bit color resolution, or 2-bit color resolution if lens array 14 comprises an array of 8x8 lenses 140 and display pixels 10, 12.

[0021] In some respects, the grayscale levels may be increased by pixel density. When using sub-optimal levels of the optical systems 14, 16, 18, the pixel resolution can be higher than the optical resolution of the optical systems 14, 16, 18. For example, a 2x2 display pixel on the binary emissive display device 1 can be perceived as one pixel with five color levels.

[0022] Color resolution increases proportionally to the number of lenses 140 in the lens array 14. However, color resolution is less of a requirement for a wide range of augmented reality applications.

[0023] In various embodiments, lens array 14 may comprise a 2x2, 4x4, 5x5, 6x6, 7x7, 8x8, 9x9, 10x10, 11x11, 12x12, 13x13, 14x14, 15x15, or 16x16 array of lenses 140. However, other configurations of lens array 14 are possible; for example, lens array 14a may comprise a rectangular matrix or any other matrix geometry.

[0024] Because the lens array 14 is positioned outside the viewer's (eyebox) coverage area (the image of the lens array is typically located within the eyebox 20, which coincides with the viewer's pupil), and is technically positioned in Fourier space, the lens array 14 can have a sparse and large number of lenses 140.

[0025] In some aspects, the lenses 140 have a width (or lens pitch) between 1 and 5 mm.

[0026] In practice, lens array 14 may comprise an array of 2x2 lenses 140, each lens having a diameter of 1 to 5 mm. In such a configuration, each display pixel 10, 12 naturally has high color information (1 bit or more color resolution). Furthermore, additional aperture filters may be required to approximate pinhole projections (images of lenses 140) for each viewpoint.

[0027] On the other hand, a lens array 14 with lenses 140 larger than 16x16 may require a larger light-emitting display panel, even with small lenses 140 (e.g., 0.3mm to 1mm diameter), to keep the size of each pixel beam (light beam 111) above the allowable aperture or self-diffraction limit.

[0028] A lens array 14 with lenses 140 between 5x5 and 8x8 is considered optimal for most applications.

[0029] The light-emitting display device 1 may have a high density of display pixels 10, 12. For example, the light-emitting display device 1 may include a plurality of micro-LEDs, micro-OLEDs, or other light-emitting devices capable of micrometer-scale or sub-micron pixel pitch. As noted above, the display pixels 10, 12 may have a low color resolution (such as 1-bit color resolution).

[0030] Either or both of the imaging optics 16 and the projection optics 18 are optional for the light field display system to function. That is, the lens array 14 alone can be the exit pupil of the light field projection system. On the other hand, the imaging optics 16 and / or the projection optics 18 enable the creation of a physical distance between the light field source (light emitting display device 1) and the eyebox 20. For example, the imaging optics 16 and / or the projection optics 18 enable a larger eye relief.

[0031] 2, the axially symmetric arrangement of the imaging optics 16 and the projection optics 18 is for illustrative purposes only. The imaging optics 16 and the projection optics 18 may be asymmetric and may include light guides and diffractive optical elements, so long as the optics do not replicate the entrance pupil or individual pixel beams (light beam 111) in the light field portion of the image formed by the projected virtual pixel 24 and the eyebox virtual pixels 26, 24.

[0032] In one aspect, the light field display system may include a pupil replicator (not shown), such as an imaging waveguide with an imaging outcoupling element, for example, for pupil dilation in peripheral portions of the image. The pupil replicator can create a peripheral eyebox region that expands the size of the eyebox 20.

[0033] In one embodiment, a wearable device comprises the near-eye lightfield display system disclosed herein. The wearable device can be adapted for virtual reality or augmented reality applications. The wearable device may comprise virtual reality or augmented reality glasses. As illustrated in FIG. 4 , the wearable device can be embodied in an eyeglass form factor comprising two light-emitting displays 1 and two lens arrays 14 to form a binocular near-eye lightfield display, where a first light-emitting display and lens array set is associated with the user's left eye and a second light-emitting display and lens array set is associated with the user's right eye. In the configuration of FIG. 4 , the projection optics 18 comprise a combiner integrated into the eyeglass lenses (the combiner is transparent to light from the real world, i.e., it allows external light to pass through to the viewer's eye pupil while projecting the light beam 111 toward the eyebox 20). The light-emitting displays 1 and imaging optics 16 may be included in separate portions of the eyeglass hinges or temples.

[0034] 5a and 5b illustrate a method of using a near-eye light field display system according to one embodiment. providing vector data corresponding to color information of a projected virtual pixel 26 and the position of the projected pixel 26 relative to a pupil plane 130; addressing the light emitting display device 1 with vector data that selects a subset of the display pixels 110, 120 of the light emitting display device 1; setting (activating) the selected subset of display pixels 110, 120 to an ON state such that each display pixel 10, 12 within the selected subset of display pixels 110, 120 generates a light beam 111; Equipped with.

[0035] In one aspect, the vector data may comprise a color model and a coordinate set. For example, the color model may comprise a 3-byte hexadecimal number representing the red, green, and blue components of a color. The coordinate set may comprise Cartesian x, y, and z coordinates.

[0036] Figure 5a shows a first subset of display pixels 110 comprising actuated display pixels a1 to a25, calculated from vector data corresponding to first projected virtual pixels 26a. Figure 5a also shows a second subset of display pixels 120 comprising actuated display pixels b1 to b25, calculated from vector data corresponding to second projected virtual pixels 26b. The first and second subsets of display pixels 110, 120 may correspond to different numbers of actuated display pixels 10, 12 (a1 to aN and b1 to bN) and / or distributions of the actuated display pixels 10, 12 on the light emitting display device 1.

[0037] The vector data may be used to calculate two-dimensional coordinates on the light emitting display device 1 corresponding to the selected subset of display pixels 110, 120. The calculations may be based on trigonometry. The set of coordinates of the virtual pixels 26a, 26b determine the two-dimensional coordinates of each actuated display pixel 10, 12 in the subset of display pixels 110, 120. Each actuated display pixel 10, 12 in the subset of display pixels 110, 120 carries partial color information for the virtual pixels 26a and 26b.

[0038] Although the method is illustrated using two subsets of display pixels 110, 120, it applies to more than two subsets of display pixels 110, 120 and a plurality of virtual pixels 26a, 26b.

[0039] As shown in Figure 5b, the actuated display pixels 10, 12 within a subset of display pixels 110, 120 are then distributed among individual physical sub-panels, where a sub-panel may correspond to a lens 140 containing a plurality of subsets of display pixels 10, 12.

[0040] The light emitting display device 1 may comprise a driver circuit (not shown) configured to control the plurality of display pixels 10, 12 to either deactivate or activate each display pixel 10, 12. Vector data may then be provided to the driver circuit.

[0041] In one embodiment, it is a computer program comprising instructions adapted to perform the methods disclosed herein when executed by a device having processing capability. [Explanation of symbols]

[0042] 1 Light-emitting display device 10,12 display pixels 110 first subset of display pixels 111 Light Beam 120 second subset of display pixels 14 Lens Array 16 Imaging optics 18 Projection optics, eyepieces 20 Eye Box 22 Image surface of light-emitting display device 24 eyebox virtual pixels 26, 26a, 26b Projected virtual pixels 130 Pupil plane 140 Lens

Claims

1. a light-emitting display device (1) positioned between a near focal plane (22) and a pupil plane (130), the light-emitting display device (1) being individually addressable to be set to inactive or active, the light-emitting display device (1) comprising a plurality of display pixels (10, 12), each display pixel (10, 12) carrying color information and generating a light beam (111) towards the pupil plane (130); a lens array (14) positioned between the light emitting display device (1) and the pupil plane (130), the lens array (14) comprising an array of lenses (140) configured to project light beams (111) generated by actuated display pixels (10, 12) to form a projected virtual pixel image (26); 1. A near-eye light field display system comprising: a near-eye light field display system, wherein the color information of the projected pixel image (26) and the position in space of the projected pixel image (26) are determined by the number of activated display pixels (10, 12) and the positions of the activated display pixels (10, 12) on the light-emitting display device (1).

2. 10. The near-eye light field display system of claim 1, wherein the lens array (14) comprises a row and column array comprising between 25 and 64 lenses (140).

3. 3. The near-eye light field display system of claim 2, wherein the lens (140) has a size of 1 to 5 mm.

4. 4. A near-eye light field display system according to any one of claims 1 to 3, wherein the color depth of the display pixels (10, 12) is 1-bit color or 2-bit color.

5. 5. A near-eye light field display system according to any one of claims 1 to 4, comprising imaging optics (16) configured to project the light beam (111) onto an image plane (22).

6. 6. The near-eye light field display system of claim 5, comprising projection optics (18) configured to project the light beam (111) so as to define an eyebox (20) at the pupil plane (130).

7. 7. A near-eye light field display system as claimed in any one of claims 1 to 6, wherein the light-emitting display device (1) comprises a drive circuit configured to control a plurality of the display pixels (10, 12) to set each display pixel (10, 12) to be activated or deactivated.

8. A wearable device for virtual reality or augmented reality applications, comprising a near-eye light field display device according to claims 1 to 7.

9. The wearable device of claim 8 , comprising virtual reality or augmented reality glasses.

10. 8. A method of projecting projected virtual pixels using a near-eye light field display system according to any one of claims 1 to 7, comprising the steps of: providing vector data corresponding to color information of the projected virtual pixels (26) and the position of the projected virtual pixel images (26) relative to the pupil plane (130); addressing the light emitting display device (1) with said vector data to select a subset of said display pixels (110, 120) of said light emitting display device (1); setting the selected subset of display pixels (110, 120) to be active so that each display pixel (10, 12) in the selected subset of display pixels (110, 120) generates a light beam (111); a near-eye light field display system for projecting a projected virtual pixel, the near-eye light field display system comprising:

11. The method of claim 10 , wherein the vector data comprises a color model and a set of coordinates.

12. The method of claim 11 , wherein the color model comprises a 3-byte hexadecimal number representing the red, green, and blue components of a color.

13. 13. The method of claim 11 or 12, wherein the set of coordinates comprises Cartesian x, y, z coordinates.

14. 14. The method according to any one of claims 10 to 13, wherein the vector data is used to calculate two-dimensional coordinates corresponding to the selected subset of display pixels (110, 120) on a light-emitting display device (1).

15. The light emitting display device (1) comprises a drive circuit configured to control a plurality of display pixels (10, 12) to activate or deactivate each display pixel (10, 12); 15. The method of claim 10, wherein the method comprises providing the vector data to the driver circuitry.

16. A computer program comprising instructions adapted to carry out the method of any one of claims 10 to 15 when executed by a device having processing capabilities.

Images