High resolution light field projector

Abstract

A light field projector for projecting an image, comprising: a light field image source (1) comprising an image generating device (20a, 20b) comprising a plurality of pixel components (15a-15d) generating a plurality of modulated light beams (111); an imaging optical element (70) projecting an image of the plurality of modulated light beams (111) to a source image plane (34); a projection optical arrangement (2) configured to project the modulated light beams (111) so as to define an eyebox (121) and form projector pixel images (16a-16d); the projection optical arrangement (2) being further configured to form the projector pixel images (16a-16d) at a projector image plane (115) and to form a projector virtual pixel image (26a) at a shifted plane (114). The imaging optical element (70) is configured to coincide the position of the projector image plane (115) with the position of the shifted plane (114).

Description

TECHNICAL FIELD

[0001] The present disclosure relates to displays, and more particularly to light field projectors for projected images, such as 3D displays providing corrected monocular depth cues. In particular, the present disclosure relates to near-eye light field projectors for virtual and augmented reality goggles and glasses and virtual and augmented reality applications.

[0002] Related Art

[0003] Among other concepts, today's light field displays use a set of "always in focus" images projected in succession through a set of spatially distinct virtual pinholes. This approach does not allow the wavefront of a virtual pixel to be reconstructed because the image components are projected in time sequence, whereas wavefront reconstruction requires interference, and thus the coincidence of the wavefront components. Therefore, the perceived resolution is limited by the resolution or individual light field components delivered to the imaging sensor, e.g. into the eye. The resolution is determined by the effective numerical aperture of the pinhole and the difference between the focal plane of the image of the image source, i.e. the plane of the image pixels, and the plane of focus of the imaging optics, typically the human eye. Therefore, the expression "always in focus" is approximate, because the optical path of the light field components always has a finite aperture, which carries an image of limited depth of field when the aperture is large, or which reduces the resolution of the image source due to diffraction on the aperture when the aperture is small.

[0004] The same problem exists in light field systems based on a light emitting display and a lens array. The individual pixels are decoupled and no mutually coherent light beams with mutually coupled phase are provided, which makes them unable to interfere, while the finite aperture of the lenses in the lens array controls the resolution and depth of field of the individual light field components.

[0005] A light field image is typically composed of a plurality of images projected through spatially distinct pupils in the optical path. It can be implemented in various ways, including sequential light field projection or splitting and collimating the light beams from a light emitting display through a lens array.

[0006] These optical systems obviously display virtual pixels in a focal plane different from the plane of the image source itself. This can make the element that improves the apparent focus of the virtual pixels on the image source with respect to the real pixels just the content itself and the large depth of field of each light field component ensured by the small pupil through which each light field component can pass compared to the combined aperture of all the spatially distinct sub-pupils. On the other hand, the finite aperture of each sub-pupil limits the effective resolution of the light field image, especially at positions far from the focal plane of the image of the image source itself. SUMMARY

[0007] The present disclosure relates to a light field projector for projecting an image, the light field projector comprising: an image source comprising an image generating device comprising a plurality of pixel components generating a plurality of modulated light beams; an imaging optical element collimating the plurality of modulated light beams; a projection optical arrangement comprising a first projection element configured to project the modulated light beams so as to define an eyebox and form projector pixel images of the pixel components at a projector image plane between the first projection element and the eyebox, the modulated light beams intersecting at projector virtual pixel images. The imaging optical element is configured to displace the projector image plane to a displaced plane between the first projection element and the eyebox at which the projector pixel images coincide with the projector virtual pixel images.

[0008] Furthermore, the image source can be controllable so as to adjust the spatial distribution of the pixel components forming the image components to be projected by the light field projector. The controllable image source can enable the modulated light beams to build the intended light field output.

[0009] The light field projector disclosed herein overcomes the effective resolution limit of known light field projection systems.

[0010] The light field projector can enable the modulated light beams to maintain the quality of the light field created by known light field projection systems and, in addition, it can also control which focal plane coincides with the image plane of the image source, wherein the image resolution is limited only by the image source resolution and the aperture in the individual light path of each light field component.

[0011] The disclosed light field projector is more robust than traditional projection systems that mimic the depth of focus of a planar image by dynamic adaptation of the focal length of a zoom element based on eye tracking information. These traditional systems do not produce a light field independent of the viewer but mimic what the eye should see according to its actual focal point.

[0012] In the invention disclosed herein, the light field characteristics independent of the viewer are always present and a corrective adjustment cue is provided without any additional action. The variable focal length mechanism in the disclosed light field projector can boost the resolution at the desired focal plane in the light field scene on demand.

[0013] The light field projector can be used for near-eye light field displays for virtual and augmented reality goggles and glasses and for virtual and augmented reality applications.

[0014] BRIEF DESCRIPTION OF DRAWINGS

[0015] Exemplary embodiments of the invention are disclosed in the specification and shown in the drawings, wherein:

[0016] Figure 1 A light field projector comprising a light field image source, an imaging optical element and a projection optical arrangement is shown.

[0017] Figure 2 A light field projector is shown comprising an imaging optical element according to one embodiment, and wherein the light field image source comprises a temporally sequentially activatable array of point lights;

[0018] Figure 3a A light field projector is shown, wherein the light field image source comprises a light emitting display;

[0019] Figure 3b A light field projector is shown comprising an imaging optical element according to one embodiment, Figure 3a of the light field projector;

[0020] Figure 3c A light field projector is shown comprising an imaging optical element according to another embodiment, Figure 3a of the light field projector; and

[0021] Figure 4 A light field projector is shown according to one embodiment, wherein the light field image source comprises a coherent light source illuminating a reflective or transmissive and diffractive display.

[0022] Exemplary Embodiments

[0023] Figure 1 A light field projector is shown comprising an image source 1. The image source comprises a light source 10 comprising a temporally sequentially activatable array of point lights (also called pin lights) each producing a light beam 110. In Figure 1 two point lights 10a and 10b are shown. The light field projector further comprises a spatial light modulator (SLM) 20a comprising a plurality of pixel components 15a, 15b each modulating a light beam component 111 reflected and diffracted on the pixel (e.g. 15a and 15b).

[0024] In the example of Figure 1 pixel component 15a is active for point light 10a and pixel component 15b is active for point light 10b. In practice, however, the light source 10 can comprise any number, e.g. tens or hundreds, of spatially distinct, temporally sequentially illuminated point lights 10a, 10b. The spatial difference can only be apparent, produced by a single light source or a small array of light sources multiplied by another active element such as a beam steering mirror or a phase spatial light modulator. The function of the light source array can also be replaced by an active phase array. The SLM 20a can comprise a complex image pattern with thousands or millions of active pixel components 15a, 15b.

[0025] The image source 1 further comprises an imaging optical element 70 projecting the image of the light source 10 to a light source image plane 34 of the point light source. The imaging optical element 70 can further collimate the modulated light beams 111.

[0026] The light field image source 1 can further comprise a collimating element 50 collimating the light beam 110 on the collimating SLM 20a. In this context, the combination of the first and second optical elements 50, 70 projects the image of the light source 10 onto the light source image plane 34 of the point light source.

[0027] In one aspect, the light source image plane 34 can comprise a passive or active Fourier filter. The Fourier filter can be formed by an array of pinholes or an array of optical shutters. The array of optical shutters can comprise for example a DMD, a fast modulator of FLCOS, a phase change material shutter or any other device used as an optical shutter. The light source image plane 34 can be considered as an entrance pupil at which the fully constructed light field enters the projection optics.

[0028] The light field projector further comprises a projection optics 2 configured to project the modulated light beam 111 so as to define an eyebox 121. The eyebox 121 can be defined along a projection axis 170 of the light field projector, but more generally the eyebox 121 is defined in the direction of the projected modulated light beam 111. More particularly, the projection optics 2 can comprise a first projection element 32 configured to project the modulated light beam 111 so as to form a projector image plane 115 and so as to form in the projector image plane 115 a projector pixel image 16a, 16b of the pixel components 15a, 15b respectively. Figure 1 The two projector pixel images 16a, 16b of the pixel components 15a, 15b respectively are shown.

[0029] The projection optics 2 can further comprise a second projection element 40 configured to project the image of the projector image plane 115 to any selected position in the direction of the projected modulated light beam 111. The second projection element 40 can comprise an eyepiece or a combiner. In Figure 1 In the example, the image of the projector image plane 115 is located in a "neutral" position so that the eyebox projector image, i.e. the image of the pixel components 15a, 15b as seen from the eyebox 121, is at infinity. In Figure 1 In the example, the eyebox projector image is not shown but its location at infinity is indicated by the collimated modulated light beam 111.

[0030] In Figure 1In this case, the two exemplary modulated light beams 111 from the pixel 15a and 15b intersect at a projector virtual pixel image 26a that is different from the projector image plane 115. The projector virtual pixel image 26a corresponds to an image of a virtual pixel before the second projection element 40 in the light field projector. Thus, as seen from the eyebox 121, the eyebox virtual pixel image 24 is formed by the apparent intersection of the pixel light beams. The eyebox virtual pixel image 24 is located at a different distance than the eyebox projector image plane (not shown), which in this example is at infinity.

[0031] The light field projector can be configured to project an image onto an imaging optical device, such as a camera or the eye 90 of a viewer as shown. Figure 1 The exit pupil of the light field projector can coincide with the entrance pupil of the imaging optical device, for example, with the eye pupil 130 of the eye 90 of the viewer.

[0032] When the light field projector projects an image towards the eye 90 of the viewer, the lens 130 of the eye 90 focuses on the location of the eyebox virtual pixel image 24. Thus, the lens 130 projects the modulated light beams 111 on the same location, which in this case is the sensor virtual pixel image 27a near the sensor of the eye retina 92. Note that the light field projector can also project an image towards a camera. Then, the lens of the camera projects the modulated light beams 111 onto the sensor virtual pixel image 27a near the sensor of the camera.

[0033] The sensor pixel images 17a, 17b of the pixel components 15 form before the retina 92 (or sensor) at a different distance than the sensor virtual pixel image 27a. The sensor pixel images 17a, 17b correspond to the second image of the pixel components 15a and 15b, respectively, in the eye 90 of the viewer or in the imaging optical device.

[0034] In some aspects, the light field projector can also be configured to project content or react to content-related instructions in order to enhance the resolution of a virtual object of interest, for example, the resolution of virtual text at a particular distance.

[0035] Figure 2 A light field projector according to one embodiment is shown, wherein the imaging optical element 70 is configured to move the position of the projector image plane 115 such that the position of the projector image plane 115 can coincide with the position of the projector pixel images 16a, 16b.

[0036] In the example of Fig. 6, the imaging optical element 70 comprises a movable lens. In this embodiment, the lens 70 moves towards the SLM 20a. As in the previous embodiment, the lens 70 is configured to move the position of the projector image plane 115 such that the position of the projector image plane 115 can coincide with the position of the projector pixel images 16a, 16b. Figure 2 Figure 1 ​In contrast, the projector image plane 115 is shifted to a shifted plane 114 where the projector pixel images 16a, 16b coincide with the projector virtual pixel images 26a. In other words, the projector pixel images 16a, 16b are located at the intersection of the modulated light beams 111 from the pixels 15a and 15b.

[0037] Other configurations of the imaging optical element 70 that move the position of the projector image plane 115 to the shifted plane 114 are possible. For example, the imaging optical element 70 can comprise an optical element with variable focal length, such as an optical element comprising an element with variable geometry, a medium with variable refractive index, an electro-optical material, a birefringent element with a polarization-rotating element such as a liquid crystal, or other elements that do the same operation.

[0038] Furthermore, the SLM 20a can be controlled to adjust the spatial distribution of the pixel components 15a, 5b forming the image components to be projected by the light field projector. As Figure 2 shown, the pixel components 15a and 15b are distributed differently (closer) in the SLM 20a compared to the configuration of Figure 1 The adjustment of the active pixel components 15a, 15b forming the image components can build the intended light field output and make the projector pixel images 16a, 16b coincide with the projector virtual pixel images 26a.

[0039] As Figure 2 shown, the eyebox virtual pixel image 24 is formed by the apparent intersection of the modulated light beams 111 as seen from the eyebox 121. The position of the eyebox virtual pixel image 24 coincides with the position of the eyebox projector images 18a, 18b.

[0040] When the light field projector projects the image to the viewer eye 90, the lens 130 of the eye 90 focuses on the position of the eyebox virtual pixel image 24. The lens 130 projects the modulated light beams 111 at the same position, in this case a sensor virtual pixel image 27a near the eye retina 92. Sensor pixel images 17a, 17b of the pixel components 15 are also formed near the eye retina 92 and coincide with the sensor virtual pixel image 27a.

[0041] In the focal distance of interest, i.e. in the distance of the eyebox virtual pixel image 24, the perceived resolution of the virtual image is substantially limited only by the resolution of the image source on the SLM 20a.

[0042] In one aspect, the light source image plane 34 can comprise a Fourier filter, especially when a highly diffractive SLM 20a is used. In this case, the light source image plane 34 can be moved along the projection axis 170 towards the SLM 20a in order to make the position of the projector image plane 115 coincide with the position of the projector pixel images 16a, 16b.

[0043] Figure 3a A light field projector according to another embodiment is shown, wherein the image source 1 comprises a light emitting display 20b. Herein, the light field projector is based on the decomposition of the light emitting display into modulated light beams 111 with collimated pixel light beams by a lens array. The light emitting display 20b can comprise a liquid crystal display (LCD), an organic light emitting diode (OLED), a micro-LED or other suitable display. The pixel components 15a, 15b of the virtual pixel 24 are directly emitted by the light emitting display 20b and collimated by the imaging optical elements 70 located in the image plane 34 or the entrance pupil of the light source. Herein, the imaging optical elements 70 comprise an array of collimating optical elements, e.g. a lens array. The imaging optical elements 70 can also act as a Fourier filter and determine for each pixel component 15a, 15b a pinhole aperture corresponding to an individual modulated light beam 111.

[0044] Similarly to the light field projector configuration of Figure 1 and 2 the projection optics 2 can comprise a first projection element 32 and a second projection element 40.

[0045] In Figure 3a the lenses 130 of the viewer's eye 90 are focused on the projector image plane 115 in the intermediate position, so that the position of the eyebox projector image, i.e. the image of the pixel components 15a, 15b as seen from the eyebox 121, is at infinity. Figure 3a The eyebox projector image (image of the projector SLM plane) is not shown in but its position at infinity is indicated by the collimated modulated light beams 111 entering the eyebox 121.

[0046] The projector virtual pixel images 26a and 27a correspond to the intersection of the modulated light beams 111 from pixels 15a and 15b, and the projector virtual pixel images 26b and 27b correspond to the intersection of the modulated light beams 111 from pixels 15c and 15d. The projector pixel images 16a, 16b, 16c and 16d correspond to the image (focal point) of the modulated light beams 111 from pixels 15a, 15b, 15c and 15d in the projector image plane 115, respectively. The sensor pixel images 17a, 17b, 17c and 17d correspond to the second image of the pixel components 15a, 15b, 15c and 15d in the viewer's eye 90 or in the imaging optics, respectively.

[0047] In the configuration of Figure 3a none of the projector virtual pixel images 26a, 26b coincide with the projector pixel images 16a, 16b, 16c and 16c, and none of the sensor virtual pixel images 27a, 27b coincide with the sensor pixel images 17a, 17b, 17c and 17d.

[0048] Figure 3b A light field projector is shown Figure 3a in which the imaging optical element 70 has been moved from its "neutral" position shown in Figure 3a to a position that moves the projector image plane 115 to a position closer to the light emitting display 20b's movement plane 114. In addition, the light emitting display 20b is controlled so as to adjust the active pixel components 15 that form the image components to be projected by the light field projector. As shown Figure 3b , the pixel components 15a and 15b are differently distributed (closer to each other) in the light emitting display 20b compared to the pixel components 15c and 15d.

[0049] As a result, the projector pixel images 16a, 16b coincide with the projector virtual pixel image 26a. When the light field projector works in conjunction with the viewer's eye 90, the lens 130 projects the modulated light beam 111 to the sensor virtual pixel image 27a near the eye's retina 92, and the sensor pixel images 17a, 17b are formed near the eye's retina 92 and coincide with the sensor virtual pixel image 27a.

[0050] As a result, the projector pixel images 16a, 16b coincide with the projector virtual pixel image 26a. When the light field projector works in conjunction with the viewer's eye 90, the lens 130 projects the modulated light beam 111 to the sensor virtual pixel image 27a near the eye's retina 92, and the sensor pixel images 17a, 17b are formed near the eye's retina 92 and coincide with the sensor virtual pixel image 27a.

[0051] In addition, the imaging optical element 70 can include a zoom optical element configured to move the position of the projector image plane 115 so that the position of the projector image plane 115 can coincide with the position of the projector pixel images 16a, 16b.

[0052] Figure 3c A light field projector is shown Figure 3a in which the imaging optical element 70 has been moved from its "neutral" position shown in Figure 3a to a position that moves the projector image plane 115 to a position closer to the light emitting display 20b's movement plane 114. In addition, the light emitting display 20b is controlled so as to adjust the active pixel components 15 that form the image components to be projected by the light field projector. As shown Figure 3b , the pixel components 15a and 15b are differently distributed (closer to each other) in the light emitting display 20b compared to the pixel components 15c and 15d.

[0053] As a result, the projector pixel images 16a, 16b coincide with the projector virtual pixel image 26a. When the light field projector projects an image to the eye 90 of the viewer, the lens 130 projects the modulated light beams 111 at the sensor virtual pixel image 27a near the eye retina 92 and the sensor pixel images 17c, 17d are formed near the eye retina 92 and coincide with the sensor virtual pixel image 27b. Due to the unmodulated pixel components 15a and 15b in the emissive display 20B, the projector pixel images 16a, 16b do not coincide with the projector virtual pixel image 26a and the sensor pixel images 17a, 17b do not coincide with the sensor virtual pixel image 27a.

[0054] Figure 4 A light field projector according to another embodiment is shown, wherein the image source 1 comprises a coherent monochromatic light source 11 projecting mutually coherent light beams, which illuminate a transmissive or reflective and diffractive display 20b, such as Figure 3a and 3b configuration. In Figure 4 embodiments, the light field projector can be used as a wave front shaping projection system.

[0055] For example, the coherent monochromatic light source 11 can comprise one laser. The display 20b can be transmissive or reflective for modulating the amplitude and possibly the phase of the incident light. The display 20b acts as an SLM, diffracting the light beams of the coherent light source 11 at each pixel and widening the light beams before each lens of the lens array of the source image plane 34. The light beams of the coherent light source 11 can optionally be collimated by a collimating element 50 and modulated on the display 20b acting as an SLM.

[0056] The imaging optical element 70 comprises an array of collimating optical elements, for example a lens array. The imaging optical element 70 can also act as a Fourier filter and determines the pinhole aperture corresponding to each light field component image of the pixel components 15a, 15b. However, no Fourier filtering is needed in this embodiment.

[0057] The modulated light beams 111 exiting the image source 1 through the different lenses of the imaging optical element 70 and possibly the source image plane 34 are mutually coherent. Therefore, the modulated light beams 111 are able to interfere at their point of incidence where they build a wave front composed of a plurality of wave front components. Then, the wave front of an arbitrary virtual pixel can be reconstructed when the light field components coincide near or exactly at the sensor of the imaging optical device, for example the retina 92 of the eye 90.

[0058] The wave front construction is Figure 4are shown in the magnified part of Fig. 1, where arrows represent wave vectors and the wavefronts of the individual light field components are lined out. Plane waves are considered only for illustration. In reality, each component can be approximated by a spherical wave with mutually different origins. The coherent light field components build up a spherical wave with apparent origins in the virtual point sources representing the virtual pixels.

[0059] Reference numerals and symbols

[0060] 1 image source

[0061] 2 light field projection optics

[0062] 10 light source

[0063] 10a, 10b point light

[0064] 11 coherent monochromatic light source

[0065] 15a-15d image component

[0066] 16a-16d projector image

[0067] 17a-17d sensor image

[0068] 18a, 18b eyebox projector image

[0069] 20a spatial light modulator

[0070] 20b display

[0071] 24 eyebox virtual pixel image

[0072] 26a projector virtual pixel image

[0073] 26b projector virtual pixel image

[0074] 27a sensor virtual pixel image

[0075] 27b sensor virtual pixel image

[0076] 32 first projection element, lens

[0077] 34 light source image plane, Fourier filter

[0078] 40 second projection element

[0079] 50 collimation element

[0080] 70 imaging optical element

[0081] 90 imaging optical device, eye

[0082] 92 sensor, retina

[0083] 110 light beam

[0084] 111 modulated light beam

[0085] 114 shifted plane

[0086] 115 projector image plane

[0087] 121 eye movement range

[0088] 130 eye pupil, lens

Claims

1. A light field projector for projecting an image, the light field projector comprising: - an image source (1) comprising an image generating device comprising a plurality of pixel components (15a-15d) generating a plurality of modulated light beams (111); - an imaging optical element (70) collimating the plurality of modulated light beams (111); - a projection optical arrangement (2) comprising a first projection element (32) configured to project the modulated light beams (111) so as to define an eyebox (121) and to form projector pixel images (16a-16d) of the pixel components (15a and 15b) at a projector image plane (115) between the first projection element (32) and the eyebox (121), the modulated light beams (111) intersecting at a projector virtual pixel image (26a) between the first projection element (32) and the eyebox (121); characterized in that the imaging optical element (70) is configured to displace the projector image plane (115) to a displaced plane (114) between the first projection element (32) and the eyebox (121), wherein the projector pixel images (16a, 16b) coincide with the projector virtual pixel image (26a).

2. The light field projector according to claim 1, wherein - the imaging optical element (70) comprises a movable lens.

3. The light field projector according to claim 1, wherein - the imaging optical element (70) comprises an optical element with variable focal length.

4. The light field projector according to claim 1, wherein, - the image generating device is controlled to adjust a spatial distribution of the pixel components (15a-15d) forming image components to be projected by the light field projector.

5. The light field projector according to claim 1, wherein - the image source (1) comprises a light source (10) comprising an array of point lights activatable in time sequence, each point light generating a light beam illuminating the image generating device; and - wherein the image generating device comprises a spatial light modulator (SLM) (20a).

6. The light field projector according to claim 1, wherein - the image source (1) comprises a light emitting display (20b).

7. The light field projector according to claim 6, wherein - the light emitting display (20b) comprises a liquid crystal display (LCD), an organic light emitting diode (OLED) or a micro-LED.

8. The light field projector according to claim 3, wherein - the image source (1) comprises a light source (10) comprising a coherent light source (11) projecting mutually coherent light beams on a display (20b).

9. The light field projector according to claim 8, wherein, - the display (20b) is configured to diffract and widen the modulated light beams (111) at each pixel component (15a, 15b) before each of the optical elements of the imaging optical element (70).

10. The light field projector according to claim 6, wherein - the imaging optical element (70) comprises an array of collimating optical elements located in a light source image plane (34).

11. The light field projector of claim 1, wherein, the projection optics (2) comprise an optical combiner (40) that projects the modulated light beams (111) from the projector image plane (115) to form sensor pixel images (17a-17d); and wherein the imaging optical element (70) shifts the projector image plane (115) at a location where the sensor pixel images (17a-17d) coincide with a sensor virtual pixel image (27a).

12. The light field projector of claim 11, wherein the image source (1) comprises a light source (10) that comprises a coherent light source (11) that projects mutually coherent light beams on a display (20b); and wherein the modulated light beams (111) generated by the display (20b) are mutually coherent and are configured to construct a wavefront composed of a plurality of wavefront components at the sensor virtual pixel image (27a).

13. The light field projector of claim 1, comprising a light source image plane (34) that comprises a Fourier filter.

14. The light field projector of claim 11, configured to project an image onto an imaging optical device (90) that comprises a sensor (92); and wherein the sensor pixel images (17a-17d) of the pixel components (15a-15d) are formed substantially at the sensor (92) and coincide with the sensor virtual pixel image (27a).

15. The light field projector of claim 1, wherein the light field projector is configured to project content or react to content-related instructions.

Images