Split-lohmann multifocal displays
Patent Information
- Application Number
- PCT/US2024/013715
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-02-01
- Filing Date
- 2024-01-31
- Publication Date
- 2024-11-28
AI Technical Summary
Existing 3D displays lack spatial selectivity in placing pixels at different depths, relying on mechanical motion and time multiplexing, which increases bandwidth requirements and frame rate.
A Split-Lohmann multifocal display system that uses a modified Lohmann lens with physically separated cubic phase plates and a phase spatial light modulator in the Fourier plane to achieve spatial selectivity in focal lengths, eliminating mechanical motion and time multiplexing by optically controlling the displacement between the plates.
Enables simultaneous placement of different pixels at various depths with spatial selectivity, reducing bandwidth requirements and maintaining high frame rates, allowing for efficient and interactive 3D content display without the need for high-bandwidth electronics.
Smart Images

Figure US2024013715_28112024_PF_FP_ABST
Abstract
Description
Split-Lohmarm Multifocal DisplaysRelated Applications
[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 442,611, filed February 1, 2023, the contents of which are incorporated herein by reference.Government interest
[0002] This invention was made with U.S. government support under contracts 1730147 and 2008464, granted by the National Science Foundation. The U.S. government has certain rights in this invention.Background of the Invention
[0003] As we view the world around us, parts of it come into focus and defocus. The eye has a lens and, by changing its focal length, it can focus objects at a specific depth, resolving them at sharp detail, while points at other depths go into defocus. This feature of the eye, referred to as the accommodation cue, is an important characteristic needed for true 3D displays.
[0004] Satisfying the accommodation cue requires a display that can optically place content at its desired depth. The difference between points light sources placed at different depths is the curvature of the wavefronts incident on an observer. Specifically, a point light source produces a spherical wavefront, one that gets flatter as it propagates. Hence, to satisfy the accommodation of the eye, a 3D display needs to induce spherical wavefronts with depth-dependent curvatures onto each of the virtual scene points.
[0005] There are numerous approaches to enable accommodation in a display. One simple means uses a focus-tunable lens, as commonly done in multifocal and varifocal displays, where a focus-tunable lens provides control over the curvature of the resulting wave. A display placed behind the lens can be made to appear from a different depth by appropriate selection of the focal length of the tunable lens. However, a focus-tunable lens provides no spatial selectivity, that is, all pixels of the display are mapped to the same field curvature associated with some desired depth. A 3D scene is produced by sequencing through multiple focal planes by rapidly changing the focal length of the lens and using the persistence of human vision, which allows these time-multiplexed focal planes to be perceived as one 3D scene. Time multiplexing presents its own set of challenges due to the need for high bandwidth electronics. For example, the frame rate of the display increases linearly with the number of focal planes that that need to be shown.
[0006] Therefore, it would be desirable to provide a "lens" that can simultaneously place different pixels of the display onto different depths.Summary of the Invention
[0007] Disclosed herein is a novel system that enables focus tunability with spatial selectivity. In this system, the lens that appears to have different focal lengths over different parts of the display. This is achieved with an improved and modified version of a Lohmann lens, which is a focus-tunable lens constructed out of two co-located cubic phase plates. In the Lohmann lens, when the two cubic plates are translated with respect to each other, they produce a quadratic phase plate (i.e., a lens) the focal length of which depends on the amount of displacement between them. However, the prior art Lohmann lens does not offer any spatial selectivity. Further, its reliance on mechanical motion often makes it an inferior choice to liquid lenses that are electronically controlled.
[0008] The invention disclosed herein provides a modification to the Lohmann lens that enables spatial selectivity, while replacing mechanical motion with optical translation. This novel configuration results in a system which is referred to herein as the "Split-Lohmann" multifocal display. Split-Lohmann displays rely on the following modifications to Lohmann lenses: (1) the two cubic plates are physically separated, while optically collocating them using a 4f relay; (2) a phase spatial light modulator (SLM) is introduced in the Fourier / pupil plane of the 4f system. A phase ramp on the SLM, mimicking the effect of a prism or adiffraction grating, produces the same effect as mechanical motion because it is placed in the Fourier plane of the cubic plates. Changing the slope of the phase ramp allows electronic control the amount of translation between the cubic plates and, consequently, the focal length of the resulting lens; and (3) to enable a spatially varying focal length, a display is optically collocated onto this SLM and the slope of the phase ramp on the SLM is changed locally. By varying the local slope of the phase ramp, different regions of the display can be associated with different translations between the cubic plates. In turn, each region observes a lens with a different focal length (and is placed at a different depth) thereby achieving the desired ability of spatial-selectivity of focal length.Brief Description of the Drawings
[0009] FIG. 1A is a schematic depiction of a tradition model of a multifocal display; FIG. IB is a schematic representation of a Lohmann lens wit focus tunability.
[0010] FIG. 2A is a schematic representation of a Split-Lehmann display with avoidance oof physical translation; FIG. 2B is a schematic representation of a Split-Lehmann display with avoidance of time multiplexing.
[0011] FIG. 3A is a schematic representation of showing the basics of a 4f field relay;FIG. 3B is a schematic representation of a phase ramp disposed at P2; FIG. 3C is a schematic representation of a lens disposed at P2.
[0012] FIG. 4 is a schematic representation of an analysis of the Split-Lohmann display as a wave propagates through the display.
[0013] FIG. 5 is an associated ray diagram showing the positioning of the eye with respect to the system.
[0014] FIG. 6 is a diagram showing the preferred placement of 3D content with respect to the system.
[0015] FIG. 7 shows simulation results with and without quantization of the depth map.
[0016] FIG. 8 is a listing of an algorithm for estimating the SLM phase function given scene depth map.
[0017] FIG. 9 shows a conceptual design of a Split-Lohmann multifocal display.
[0018] FIG. 10 shows photographs captured from a prototype of the disclosed system at near and mid focus.
[0019] FIG. 11 is a representation of a prototype system in accordance with the present disclosure.
[0020] FIG. 12 is a schematic representation of a more compact version of the system.Detailed Description
[0021] The details of the invention are explained in the context of a comparison with a traditional multifocal display using focus tunable lenses, and progressively proceeds to the novel Split-Lohmann display. FIG. 1A shows a schematic of a multifocal display, consisting of a translating display behind an eyepiece. If thefocal length of the eyepiece is fe, then placing the display at a distance fefrom the eyepiece will result in display pixels being collimated. The net result is that the display is optically placed at infinity. Moving the display closer to the eyepiece results in it being optically placed at a distance closer than infinity. The eye is placed at the focus plane of the eyepiece (i.e., a distance fefrom the lens). This choice has certain benefits over placing the eye right against the eyepiece.
[0022] FIG. IB provides a way to implement the multifocal display shown in FIG. 1A, using focus tunable lenses that are realized as Lohmann lenses. Here, instead of physically translating the display, a 4f relay with a focus-tunable lens is used to optically translate the display in the pupil / Fourier plane. When the focus-tunable lens has zero power (or infinite focal length), the 4f lens system produces a copy of the display at some nominal focus plane. Changing the power of the tunable lens results in an axial translation of the focus plane. It is worth noting that the entire wavefront is axially translated. That is, what was originally at the nominal plane now appears at a different plane. When the tunable lens has positive power (corresponding to a convex lens) the display shifts away from the eyepiece, and vice versa for negative powers. The net result is that the setup shown in FIG. 1A can be achieved with appropriate placement of the eyepiece with respect to the range of virtual displays created by the 4f relay.
[0023] The Split-Lohmann display disclosed herein eliminates the physical motion inherent in a Lohmann lens, as well as its reliance on time-multiplexing, to createcontent on multiple focal planes. Physical motion is avoided using the setup shown in FIG. 2A. Here, the two cubic phase plates comprising the Lohmann lens are split, while retaining their collocation by placing them at either end of a 4f relay. As a 4f relay flips the fields along both axes, the same phase profile is used at both ends of the relay. In their nominal state, the two cubic phase plates cancel themselves out (as is typical to Lohmann lenses). To introduce translation, without mechanical motion, Fourier optics is replied upon. Basic Fourier analysis suggests that translation of a signal can be achieved using a linear phase shift to its frequency domain counterpart, where the slope of the linear phase shift controls the amount of translation. In the Split-Lohmann display, this reduces to introducing a phase ramp in the Fourier plane of the cubic phase plates. Such a phase ramp can be implemented using a phase SLM, with which the first cubic plate can be translated with respect to the second. The effective phase modulation is proportional to:
[0024] The quadratic phase term Ax2enables the focus tunable lens, where A is controlled by the slope of the phase ramp on the SLM. There is also an extra term that is linear in x. This is a consequence of translating just one of the cubic phase plates.
[0025] Time multiplexing is also eliminated in the disclosed implementation of the Split- Lohmann display. The display is optically aligned with the SLM via the 4f relay, asillustrated in FIG. 2B. If there were no cubic phase plate in between, the use of the 4f relay ensures that a display pixel is resolved onto the SLM. The global phase ramp on the SLM can be replaced with a local one, which implies that, depending on the slope of phase ramp shown on the phase SLM locally, different choices of focal lengths of the tunable lens can be induced. In essence, spatial selectivity has been enabled in choice of power of the focus tunable lens, which allows avoiding time multiplexing all together.
[0026] Split-Lohmann displays have many desirable properties. For example, many recent works in multifocal displays have dramatically increased the number of displayed focal planes. However, this comes at the cost of requiring a high frame-rate display and high-bandwidth electronics. However, the disclosed system provides a similar number of focal planes in a single-shot manner with a standard display. The disclosed system has a large etendue; for the most part, the numerical aperture of the optical elements used is limiting. Content generation for the disclosed display is computationally lightweight because the mapping from the input RGBD content to the phase SLM pattern is near- analytical. In turn, this allows the effortless showcasing of interactive 3D games in real time.
[0027] A limiting feature of this design is that point-to-point mapping between the display and the SLM is not obtained because the 4f relay connecting the two has a cubic phase plate in its Fourier plane. As a consequence, light from a pixel on the display is spread over multiple pixels on the SLM, denoted as the pointspread function (PSF) of the cubic plate. The size and features of this PSF are determined by the parameters of the cubic plate and the focal length of relay lenses. Typically, the spread of this PSF can be as large as 50 to 100 pm or tens of SLM pixels. This also provides the limits of the spatial selectivity in depth that the design provides.
[0028] This section provides a wave optics-based formation model of Split- Lohmann multifocal display will now be presented. This model is subsequently used for the design and analysis of the system.
[0029] An image formation model on how light propagates through the system. Wil now be presented. Much of this derivation relies on Fourier transforming properties of lenses as well as spatial filtering of 4f systems.
[0030] A 4f system consists of two lenses, which are assumed to have the same focal length fQ, that are placed at a distance of f0and 3 / 0from an input plane, which is denoted as PrThe plane that is 2f0from Pr(i.e., the midpoint between the two lenses) is called the Fourier or pupil plane and is denoted as P2. Finally, the plane 4f0from Ptis the output plane and is denoted as. It is assumed that lenses are ideal thin lenses. The analysis requires four basic properties.
[0031] (1) the Fourier transforming property of lenses. When is the incidentwave at P1;the wave after propagation to P2is given as:where:U1is the Fourier transform
[0032] (2) Field relay. As shown in FIG. 3A, if there is no optical element at P2, then the phasor field at P3, denoted as u3(x3, 3), is given as:This makes the 4f system an ideal field relay.
[0033] (3) Linear phase ramp at P2. As shown in FIG. 3B, when we have a prism or a linear phase ramp at P2of the form:then the field at P3is a shifted (and flipped) copy of urFor analytical tractability, the effects of any limiting apertures are ignored here and in the rest of the analysis.
[0034] (4) A lens at P2. As shown in FIG. 3C, placing a lens of focal length resultsin an axial translation of the field at P3. Specifically, the wavewhich previously appeared at P3or a distance of 4 / o from Pltaxially translates to:
[0035] A detailed breakdown of wavefront propagation and image formation in Split- Lohmann displays will now be provided. This analysis will also touch upon the different sources of non-idealities in the design and provide a mechanism to reason about their effects. Specifically, wavefronts formed at different locations in the system (labeled as the planes P1(..., P5) are derived as light propagates from the display and to the eye. Each plane is 2f0from its preceding one. A visual overview of this analysis in provided in FIG. 4.
[0036] P1; Display. A display with spatially incoherent pixels, emitting narrow band illumination centered around a wavelength A is assumed. Spatial incoherence allows the analysis of the pixels in isolation, as their wavefronts do not interfere. In terms of implementation, an OLED display as well as Liquid Crystal Display with a large area source as a backlight provides a good approximation to this model. Consider a point at a location on the display and track itswavefront through the system. To help visualize the wavefronts, a few such points are highlighted in FIG. 4.
[0037] P2: The first cubic phase plate. The first lens collimates the light from a point on P1, to form a plane wave at P2. The wavefront at P2, prior to the cubic phase plate is given as:The cubic phase plate changes this wavefront to:Here, c0determines the curvature of the phase plate.
[0038] P3: Phase SLM. The wavefront at P3is estimated by noting that P3is 4f away from P1, with a cubic phase function at its pupil / Fourier plane at isdenoted as the Fourier transform of the cubic phase plate, then the wavefront prior to the SLM placed at P3is given as:That is, a copy of / i(x,y) is observed at the location (— x0, — y0). This wavefront is phase modulated by the SLM. Split-Lohmann displays deploy a phase function on the SLM at P3that is piecewise linear. However, for tractability in analysis, the analysis starts with a single phase ramp at P3. This assumption is accurate for points which lie entirely within a linear phase ramp. With this, the wave after the SLM is given as:where: vQis the slope parameter that controls the amount of shift induced on the wavefront at P4.Lohmann lenses require the same amount of shift along both axes and hence, the same slope parameter v0is used for both x and y. The specific choice of v0is determined by the desired depth map.
[0039] P4: The second cubic phase plate. Under the assumptions made on the wavefront at P3being concentrated into a single linear piece of the SLM, the optical path from> is one of a 4f system with a linear phase ramp in its Fourier plane. Hence, the wavefront prior to the second cubic phase plate is a translated copy of the one at P2. Denoting A= vofofrom Eq. (3), this can be written as:With some simplifications, and ignoring constants, the wavefront after the second cubic phase plate can be written as:Because u2> corresponds to a planar wavefront, its translation only changes it by a constant / scalar. In simpler words, the wavefront after the second cubic phase plate is the same as the one before the first cubic plate, multiplied by two phase terms: a quadratic term corresponding to a lens with focal length ft= and a linear term. As will be seen next, the quadratic termresults in a desired axial shift of the point and the linear term results in an unwanted lateral shift.
[0040] P5: Virtual display plane. If the quadratic and linear phase terms in Eq. (10) are ignored, then the wavefront at P5is the same as the one in P1(i.e., a delta function centered at The quadratic phase term induces an axial shiftsothis delta function appears at a location of:with respect to the P5plane, with positive values indicating displacements towards the eyepiece. Because v0is controlled via the phase patterns shown on the SLM, a display pixel can be placed at a specific axial distance from the eyepiece, which in turn allows the eye to perceive it as arising from a desired depth.
[0041] The linear phase term provides a lateral shift so that the delta function appears at:This lateral shift is undesired but is often very small due to its dependence on
[0042] The Split-Lohmann multifocal display described thus far and illustrated in FIGS. 2 and 3 have a number of free parameters, which include: fe, the focal length ofthe eyepiece, fQ, the focal length of the relay lenses, and c0the parameter determining the curvature of the cubic phase plate. Each of these parameters controls salient attributes of the display, such as its field of view, eyebox, and its depth range, and depth resolution.
[0043] Eye relief. The system assumes that the eye is at a distance fefrom the eyepiece. This placement ensures that the axial shift of a display pixel does not change its viewing direction. The associated ray diagram is provided in FIG. 5. It is a requirement that the eye be placed at the focal plane of the eyepiece (i.e., a distance of fefrom the eyepiece). This placement is important as it ensures that an axially translating point on the display remains at the same viewing angle at the eye. This allows the decoupling of depth placement and viewing direction, which is extremely helpful when porting content onto the device. This positioning of the eye has another advantage. If the eye were in a different position, the coupling between viewing direction and depth placement could result in requiring the same display pixel, under different axial shifts, for multiple scene points. This would significantly complicate the ability to faithfully reproduce a 3D scene on the system.
[0044] Eyepiece focal length fo. A primary criterion in the selection of the eyepiece is the field of view of the display. Suppose that Edispis the smaller of the display and SLM width / height, the field of view of the multifocal display is given as:(13)
[0045] All things considered a smaller focal length is desirable. However, the increase in field of view comes with a commensurate reduction in the eyebox of the display because refractive optics preserve the etendue of the system.
[0046] Another factor that needs to be kept in mind is the working range of the display in terms of the depth it can produce. Suppose it is desired to have a working range of W diopters (D), say from [0, W]D. The focal length of the eyepiece allows the characterization of the amount of axial displacement of the virtual displays. Using basic ray tracing, the total axial displacement Z that is needed in the plane p5is given as:The total axial displacement is usually in the order of a few millimeters. For example, a prototype of the disclosed system is built with fe= 40mm and for a working range of W = 4D, which results in Z = 402 x 4 / 1000 = 6.4mm. A smaller eyepiece focal length results in requiring a smaller range of axial displacement of the virtual displays, which is beneficial. However, this invariably comes at the cost of having fewer focal planes over the working range, due to inherent limitations in axial resolution of virtual displays. It is worth noting that a number of prior works have tried to characterize the number of focal planes needed in multifocal displays and that this number can be as low as five focal planes, when the evaluation metric is a satisfactory level of accommodation, to as large as 40,when the evaluation metric is visual acuity. An implementation of the disclosed system aims for 50 focal planes for reasons which will be enumerates shortly.
[0047] Curvature of the cubic phase plate cnand tilt range of the SLM. The curvature of the cubic plates plays a direct role in the axial displacement as seen in Eq. (11). A higher curvature, corresponding to a smaller c0, allows for a smaller value of SLM tilt v0to produce the necessary axial translation; however, this results in a larger PSF on Pethat reduces the spatial resolution of depth selection.
[0048] To understand these dueling concerns better, is first necessary to characterize the limits on v0, the slope of the phase modulation performed by the SLM. Given an SLM with a pixel pitch of 8stm, the phase gradient cannot exceed toavoid aliasing effects. This constrains
[0049] Combining this with Eq. (11), the working range Z can be bound as:and using Eq. (14), this yields:
[0050] Generally, large values of c0are preferred as the resulting surface with its smaller curvature is more easily fabricated. The value of c0is also crucial in determining the spread of the wavefront on the SLM, placed at P3, for light arising from a single pixel on the display at Pt. As derived in Eq. (7), this wavefront is shaped by the Fourier transform of the cubic phase function. The scaling property of Fourier pairs can be used to show that doubling c0reduces the size of the PSF h by a factor of two in each dimension. Thus, using the largest value of c0will produce a wavefront at the SLM whose energy is most concentrated, spatially. Reducing the spatial spread of this wavefront in turn reduces the number of artifacts generated near depth discontinuities and characterizes the spatial resolution at which we can obtain depth selectivity.
[0051] Selecting relay lens focal length fn. Given the central role of the relay lenses in the overall design, the choice of their focal length f0affects many aspects of the display: (1) Size. By its construction, the length of the light path scales linearly with / o. This suggests that smaller values of / 0are preferred; (2) Diameter. All of the analysis of wavefront propagation presented so far are with ideal lenses, which assumes little of the wavefront is clipped by the relay lenses. In turn, this requires that the relay lenses have sufficiently large diameters. Practically, a rule of thumb that arises from ray tracing is that the diameter of relay lenses should be greater than the sum of the image and Fourier plane aperture widths. Because the focal length of a spherical lens cannot be smaller than its diameter, this forces us to avoid lenses with small f0; (3) The cubic phase plate. Becausethe upper bound on the cQderived in Eq. (17) is cubic in f0, decreasing f0forces a significant reduction in c0, which has the negative consequences discussed above, suggesting the use of larger f0.
[0052] Placement of the Eyepiece. The placement of the eyepiece with respect to P5determines the working range of the display. From Eq. (11), pixels on the display can be virtually translated, with respect to P5, to axial locations in the range:To use the entire working range, the eyepiece should be placed such that this entire range lies closer the lens than its focal plane. Further, aligning the furthest axial shift with the focal plane of the eyepiece will place that plane at infinity to the eye. As seen in FIG. 6, this requires the eyepiece to be placed at a distance zefrom P5such that:With reference to FIG. 6, to place a point at a depth d0from the eye, an axial shift to the display pixel corresponding to that point by an amount 8Zfrom P5is needed.
[0053] To display a 3D scene, the system takes in as input an intensity image t(j%, / 3) and a depth imaged d(a, (3), where a and / 3 are in the tangent of the anglesubtended at the eye (i.e., the space corresponding to the locations on the image plane of a perspective camera). Therefore, the intensity at apixel corresponds to a 3D scene point
[0054] FIG. 6 provides a detailed view on the placement of the eyepiece and eye with respect to the plane P5, and ray diagrams on how to compute the virtual display location to obtain a specific depth. The eyepiece is placed at a distance zefrom P5, as given in Eq. (18). As shown in FIG. 5, the eye is placed at a distance fefrom the eyepiece which ensures that the axial shift of a display pixel does not change its viewing direction. Hence, the display of intensity at a location( / eao» / e^o)onthe display at Prmust be ensured. Note that this calculation ignores the effect of the unwanted shift which can be made extremely small.
[0055] To place the same point at a depth from the eye, we need toinduce a displacement 6Zfrom the nominal plane P5needs to be induced, such that:
[0056] This expression can be derived from the ray diagram shown in FIG. 6, and simplifies to:
[0057] To induce this axial shift, Eq. (11) can be used to calculate that the SLM slope parameter v0corresponding to this pixel:
[0058] Consequently, accounting for the xy flip on P3, the pattern on the SLM must have a slope of v0as in Eq. (21) at the location
[0059] Need for Quantizing the depth map. Given the input depth map , it istempting to calculate an axial shift mapby replacing dQin Eq. (20) withand subsequently a phase SLM slope map using Eq. (20). Aphase imagefor the SLM can now be created using Eq. (8):
[0060] This phase pattern, however, will not work as its gradient is not the desired one; specifically:
[0061] The second term can produce grossly incorrect phase gradients, which leads to incorrect 2D translation of the cubic phase plates.
[0062] To fix this, a simple heuristic is used: the depth map is quantized in thediopter space. Given that natural scenes have depth maps that are piecewise smooth, quantizing it makes it piecewise constant. A piecewise constant image has zero-valued gradients, except at discontinuities. This simple fix ensures thatthe second term of the phase gradient is zero at most locations. In one embodiment, the scene is quantized with 50 uniformly spaced values in the working range of the display, which is typically 25 cm to00or [0, 4] diopters. This results in a quantization error of 0.04 diopters. The choice of 50 quantization levels is based on prior work that characterizes the number of focal planes required to maintain visual acuity when a point is in focus at the eye.
[0063] FIG. 7 shows simulations results with and without the quantization of the depth map. In the absence of quantization, the eye is only able to focus on regions that have a locally-constant depth map, for example, the front of the car and truck. For parts of the scene with smoothly-varying depth, like the road and the building facade, the shift induced by the SLM is no longer equal along x and y axis. The resulting modulation is no longer quadratic and hence, the eye is unable to focus on it.
[0064] For the scene shown in FIG. 7 at (a, b), simulation results are shown with (d) and without (c) quantizing the depth map prior to deriving the SLM phase function. The columns in (c, d) correspond to the eye in focus at 4D (left) and 3D (right). As seen in (d), using a quantized results in realistic focus and defocus cues. In contrast, in the absence of quantization, correct focus cues are only observed for objects with locally-constant depth maps like the front of the vehicles. Even defocus cues, for example the text on the building profiles are incorrect in (c). This is a consequence of the SLM inducing unequal shifts of the cubic plates along the two axes. This results in elliptical phase delays being induced on pointsthat makes it impossible for the eye to focus on them. The arrows overlaid on the images show interest points, color coded in green for correct and red for incorrect (de)focus cues.
[0065] A pseudocode for the process of estimating the SLM phase function, given a scene depth map is provided in as Algorithm 1, shown in FIG. 8. This process is near-analytical, having a light computational footprint. As a consequence, interactive applications can be enabled without requiring sophisticated computing platforms or even optimized implementations.
[0066] FIG. 9 shows a conceptual design of a Split-Lohmann multifocal display. Given a scene in the form of an RGBD image, a RGB display is used to project the color image while a phase spatial light modulator (SLM) shows a simple analytical function of the depth map. This produces a 3D multifocal scene without requiring time multiplexing. FIG. 10 shows photographs captured from a prototype of the disclosed system at near (left) and mid (right) focus.
[0067] In one embodiment, a prototype of a Split-Lohmann Display is shown in FIG. 11. While specific specifications are given for the components of the prototype, as would be realized by one of skill in the art, other components may be substituted without deviating from the intended scope of the invention. The OLED used is a 1.03"-diagonal Micro-OLED having a 2560 x 2560 RGB resolution with a pixel pitch of S0LED= 7 pm. The SLM is a Holoeye GAEA-2, which is a 0.7"- diagonal phase-only reflective SLM having a 2464 x 4000 resolution with a pixel pitch of 8SLM= 3.74 pm. The smaller area of the SLM makes it the limiting factorin terms of displaying content and hence, the field of view. A 40mm eyepiece is used, which provides a diagonal field of view of approximately 25°. In terms of optics, the relay lenses were achromatic doublets with focal length f0= 100mm. The cubic phase plate is a custom component, fabricated using subtractive manufacturing (i.e., by using laser to sculpt the desired shape). It was fabricated over a diameter of 12mm with a thickness parameter of c0= 0.0193m2 using a material with a refractive index of 1.46. Under these settings, the maximum sag in the phase plate was 50 pm, which was also the limit for the manufacturing process.
[0068] The current implementation of the Split-Lohmann display lacks portability, which is likely sufficient for its use in ophthalmology and robot surgery. However, there are numerous applications where portability of the display is a primary concern.
[0069] One step towards a more compact design is to use lenses with smaller focal lengths in the construction of the 4f system, as shown schematically in FIG. 12. For example, the use of a 30mm lens reduces the physical size of the system as well as provides a 3x increase in the size of the eyebox. Alternatively, polarization-sensitive optics may be used to reduce the physical footprint of a device without changing its optical footprint. However, 4f systems also require lenses with sufficiently large diameters to avoid vignetting. Specifically, the diameter needs to be greater than sum of the image and Fourier plane fields diameters. So, a second step is required where the design goes beyond 4f systems to implement the system. In an alternate embodiment, the use of 4frelays can be avoided. Here, the 4f system is replaced with a single lens of focal length f0placed at 2f0from the display as well as the phase SLM. The cubic phase plate is collocated with this lens. This system provides a 1:1 image relay between the display and SLM. Suppose that the SLM has a constant phase pattern on it. The lack of telecentricity would result in the reflected light to be significantly shifted when it appears again on the lens plane. This would mean that we would couple the cubic wavefront with a shifted copy, where the shift depends on the lateral location of the point on the display. To resolve this, we introduce a field lens on top of the SLM. This lens is designed such that the reflected light from each point at the SLM overlaps exactly with the aperture of the cubic phase plate. Due to mirror reflection, the same wavefront is produced but flipped on the SLM and so the contributions from the cubic phase plate cancels out as when the SLM does nothing. To create spatially-varying focus effects, as before, linear phase ramps are added on the SLM which produces the appropriate shifts between the cubic plates.
[0070] The biggest advantage of this system is that it can be implemented with lenses with small f0, and smaller diameters than what we would be able to with a 4f system.
[0071] A system for providing multifocal displays by providing a single-shot display that has a vastly enhanced spatial selectivity of depth for display pixels has been disclosed herein. The central technical novelty of the system is an optical arrangement wherein a spatially varying focus tunable lens is achieved using a"split" version of Lohmann lens, enabling independent local and non-mechanical control on the amount of shift between the two cubic phase plates that constitute the Lohmann lens. The multifocal display preserves the resolution of the display, allowing the depth selection across 25-50 focal planes without any temporal multiplexing.
[0072] The concept of Split-Lohmann potentially applies to systems well beyond 3D displays. At its core, it provides the ability to change focus planes associated with different spatial locations on an image plane. Basic duality between displays and cameras should in principle permit adaptive and spatially-varying focus capabilities, and perhaps even all-in-focus captures with a reworked system.
[0073] As would be realized by one of skill in the art, many variations on implementations discussed herein which fall within the scope of the invention are possible. Specifically, many variations in the components could be used to obtain similar or modified results. The invention is not meant to be limited to the particular exemplary model disclosed herein. Moreover, it is to be understood that the features of the various embodiments described herein were not mutually exclusive and can exist in various combinations and permutations, even if such combinations or permutations were not made express herein, without departing from the spirit and scope of the invention. Accordingly, the system disclosed herein is not to be taken as a limitation on the invention but as an illustration thereof. The scope of the invention is defined by the claims which follow.Claims1. A system comprising: a display; a first 4f relay comprising a first and second lens having the same focal length f, wherein the first lens is disposed at a distance f from the display and the second lens is disposed at a distance 3f from the display; a second 4f relay comprising a third and fourth lens having the focal length f, wherein the third lens is disposed at a distance 5f from the display and the fourth lens is disposed at a distance 7f from the display; a first cubic phase plate located at a distance of 2f from the display; a second cubic phase plate located at a distance of 6f from the display; and a phase spatial light modulator disposed at a distance 4f from the display.2. The system of claim 1 where a virtual display is formed at a distance 8f from the display.3. The system of claim 1 wherein a phase ramp of the phase spatial light modulator is locally varied to independently set a focus setting for individual
Claims
pixels on the display.
4. The system of claim 3 wherein the phase ramp of the phase spatial light modulator is locally varied based on a phase function derived from a depth map of an image displayed on the display.
5. The system of claim wherein the depth map is quantized in diopter space prior to deriving the phase function for locally varying the phase ramp of the phase spatial light modulator.
6. The system of claim 2 further comprising: an eyepiece for focusing the virtual display for viewing by a human eye.
7. A system comprising: a display; a first lens having a focal length f; a cubic phase plate, wherein the first lens and the cubic phase plate are collocated at a distance 2f from the display; a second lens having a focal length 2f;a phase spatial light modulator, wherein the second lens and the phase spatial light modulator are collocated at a distance 4f from the display; and a beamsplitter located at a distance f from the display for redirecting light reflected from the phase spatial light modulator to a virtual display.
8. The system of claim 7 further comprising: a third lens having a focal length f located at a distance f from the beamsplitter.
9. The system of claim 8 wherein a phase ramp of the phase spatial light modulator is locally varied to independently set a focus setting for individual pixels on the display.
10. The system of claim 9 wherein the phase ramp of the phase spatial light modulator is locally varied based on a phase function derived from a depth map of an image displayed on the display.
11. The system of claim 10 wherein the depth map is quantized in diopter space prior to deriving the phase function for locally varying the phase ramp of the phase spatial light modulator.
12. The system of claim 6 wherein a virtual display is formed at a distance f from the beamsplitter.
13. The system of claim 10 further comprising: an eyepiece for focusing the virtual display for viewing by a human eye.
14. A method comprising: receiving a depth map of an image displayed on a display; locally varying a phase ramp of the phase spatial light modulator based on the depth map to independently set a focus setting for individual pixels on the display.
15. The method of claim wherein locally varying the phase ramp of the phase spatial light modulator mimics the physical movement of two cubic phase plates with respect to each other when the cubic phase plates are separated and each cubic phase plate is placed between the lenses of a 4f system.
16. The method of claim 14 further comprising: quantizing the depth map in diopter space prior; and deriving a phase function for locally varying the phase ramp of the phase spatial light modulator based on the quantized depth map.
Citation Information
Patent Citations
Display system for rendering a scene with multiple focal planes
US20210364817A1
Optical measurement method and system and optical device manufacturing system
US20220034751A1
Optical method and system for light field displays based on beams with extended depth of focus
US20220155611A1
Scanning device with point-to-point focusing
US20220187592A1
