Display with foveal optical correction

Abstract

A display system (500) for displaying an image to a user's eye includes a light guide optical element (LOE) (506) and an image projector (512) that projects image illumination of a collimated image onto the LOE. The image projector includes electrically controllable variable lenses (10, 13, 71, 77, 58A, 58B, 59, 58C, 58D, 58E, 58F1, 58F2, 58G1, 58G2, 58H, 1223). A controller (18) determines a current region of interest of the image either from tracking the user's eyes or from analyzing the image content, and controls the variable lens to reduce aberrations in the current region of interest at the expense of increasing aberrations in at least one area of ​​the image outside the current region of interest.

Description

[Technical field]

[0001] Optical systems for near-eye displays project an image from a small image generator into an eyebox area (eye motion box or "EMB") that corresponds to the estimated location of the observer's pupil. The optical system used for this projection typically includes a combination of mirrors, lenses, and prisms. The image is typically transmitted to the observer's eye by a coupler based on a transparent waveguide or a transparent reflector.

[0002] The optics are optimized to reduce optical aberrations in the projected image, including defocus, spherical, coma, and astigmatism.

[0003] The larger the projected field, the more complicated it becomes to produce an aberration-free image, or at least to keep the aberrations at a level acceptable to the observer. Optical systems can be optimized to minimize aberrations in the center of the field. However, maintaining a good image throughout the entire field requires large, complex, and expensive optics. Furthermore, different wavelengths have different optical powers, which results in chromatic aberrations. Thermal changes in the optical system also produce variations in optical power, thereby moving the optical system from its nominal (minimum aberration) position.

[0004] Waveguide-based systems include an image projector that couples a collimated image (an image at infinity) into the waveguide. Another cause of image quality degradation is when an uncollimated (or poorly collimated) image is injected into the waveguide.

[0005] An eye-tracking subsystem is typically implemented in near-eye display systems, so that the direction of gaze (the current "gaze") is typically known to the system with low latency. Summary of the Invention

[0006] The present invention is a display system and corresponding method for displaying an image to a user's eye.

[0007] In accordance with the teachings of an embodiment of the present invention, a display system for displaying an image to an eye of a user, the eye being positioned within an eye motion box, the display system comprising: (a) a light guide optical element (LOE) having a pair of parallel major outer surfaces; (b) an image projector for projecting image illumination of a collimated image, the image projector being optically coupled to the LOE such that the image illumination is introduced into the LOE to propagate within the LOE by internal reflection at the pair of major outer surfaces, the image projector including an electrically controllable variable lens; and (c) a coupling-out arrangement associated with at least a coupling-out region of the LOE, the coupling-out arrangement for projecting image illumination within the LOE. A display system is provided comprising: (a) a coupling-out arrangement configured to redirect at least a portion of the propagating image illumination to an eye motion box for viewing by a user's eye; and (d) a controller comprising at least one processor, the controller being associated with the electrically controllable variable lens and configured to: (i) determine a current region of interest of the image; and (ii) generate an actuation signal that changes characteristics of the electrically controllable variable lens to reduce at least one type of aberration in the current region of interest of the image viewed by the user's eye, at the expense of increasing aberration in at least one area of ​​the image outside the current region of interest.

[0008] According to a further feature of the invention there is also provided an eye tracking device deployed to track a current gaze of a user's eyes, the controller determining a current region of interest based on input received from the eye tracking device.

[0009] According to a further feature of the invention, the image is a video image and the controller processes content of the video image to derive the current region of interest.

[0010] According to a further feature of the invention, the image is a video image and the controller receives a data stream indicative of a current region of interest associated with the video image.

[0011] According to a further feature of the invention, at least one optical aberration varies across a field of a collimated image from the image projector, the electrically controllable variable lens has variable optical power along at least one axis, and the controller varies the optical power to reduce the at least one optical aberration in a region of the collimated image corresponding to a current region of interest.

[0012] In accordance with a further feature of the invention, there is also provided a temperature sensor generating a signal indicative of temperature, and the controller, in response to the signal indicative of temperature, varies the optical power to at least partially compensate for temperature related variations in optical properties of the image projector.

[0013] According to a further feature of the invention, the image projector sequentially projects different colored image illumination at different time periods, and the controller varies the optical power synchronously with the time periods to provide color-specific aberration compensation.

[0014] According to a further feature of the invention, the controller is further configured to store a value indicative of a correction required for manufacturing tolerances of the image projector, and the controller is further configured to operate the electrically controllable variable lens to both provide a correction for the manufacturing tolerances of the image projector and to reduce at least one optical aberration in a region of the collimated image corresponding to the current region of interest.

[0015] In accordance with a further feature of the invention, there is also provided a user input device associated with the controller, the controller being responsive to user input provided via the user input device during the calibration process to store new values ​​indicative of corrections required for manufacturing tolerances of the image projector.

[0016] In accordance with a further feature of the invention, the collimated focal field from the image projector is non-uniform and the electrically controllable variable lens has a variable focal length and the controller varies the focal length to enhance collimation of the collimated image within the region of interest.

[0017] According to a further feature of the invention, the uniformity of the image observed at the eye motion box varies across the field of the image as a function of the polarization of the image illumination introduced to the LOE, and the electrically controllable variable lens is a variable polarization correcting lens, and the controller varies the variable polarization correcting lens to reduce the non-uniformity in the region of the image corresponding to the current region of interest.

[0018] According to a further feature of the invention, an image projector comprises a narrow beam illumination source and a scanning device for generating a scan pattern of the narrow beam across the angular field of the image, and an electrically controllable variable lens is disposed in an optical path between the narrow beam illumination source and the scanning device or adjacent to the scanning device.

[0019] According to a further feature of the present invention, the image projector comprises a spatial light modulator and a collimating optics, and the electrically controllable variable lens is disposed in an optical path between the collimating optics and the LOE or is integrated with the collimating optics.

[0020] According to a further feature of the present invention, the image projector comprises (a) a narrow beam illumination source; (b) a scanning device for generating a scan pattern of the narrow beam across an angular field of the image to generate a real image at the image plane; and (c) a collimating optical system for collimating the image illumination from the image plane so as to be introduced to the LOE as a collimated image, wherein the electrically controllable variable lens is disposed in an optical path between the image plane and the LOE, and the display system further comprises a second electrically controllable variable lens disposed in the optical path between the narrow beam illumination source and the image plane.

[0021] According to a further feature of the invention, the coupling-out arrangement comprises a plurality of mutually parallel partially reflective surfaces in the LOE, the partially reflective surfaces being oblique to the major outer surface.

[0022] According to a further feature of the invention, the coupling-out arrangement comprises a diffractive optical element associated within the LOE and configured to couple out a portion of the image illumination.

[0023] In accordance with the teachings of an embodiment of the present invention, a method for displaying an image to an eye of a user, the eye being positioned within an eye motion box, the method comprising the steps of (a) providing a display system, the display system including: (i) a light guide optical element (LOE) having a pair of parallel major outer surfaces; (ii) an image projector for projecting image illumination of a collimated image, the image projector being optically coupled to the LOE such that the image illumination is introduced into the LOE to propagate within the LOE by internal reflection at the pair of major outer surfaces, the image projector including an electrically controllable variable lens; and (iii) a lens array for projecting at least one of the LOE's illumination components. Also provided is a method comprising: (a) providing an image capturing system including a coupling-out configuration associated with a coupling-out region of the LOE and a coupling-out configuration configured to redirect at least a portion of image illumination propagating within the LOE to an eye motion box for viewing by a user's eye; (b) determining a current region of interest of the image; and (c) varying a characteristic of an electrically controllable variable lens as a function of the current region of interest of the image so as to reduce at least one optical aberration in the region of interest at the expense of a reduction in image quality seen in at least one other region of the image.

[0024] According to a further feature of the invention, the current region of interest is determined by sensing the current line of sight of the user's eyes.

[0025] According to a further feature of the invention, the image is a video image and the current region of interest varies as a function of the content of the video image.

[0026] In accordance with the teachings of one embodiment of the present invention, a display system for displaying an image to an eye of a user, the eye being positioned within an eye motion box, the display system comprising: (a) a light guide optical element (LOE) having a pair of parallel major outer surfaces; (b) an image projector for projecting image illumination of a collimated image, the image projector optically coupled to the LOE such that the image illumination is introduced into the LOE to propagate within the LOE by internal reflection at the pair of major outer surfaces, the image projector including an electrically controllable variable lens having variable optical power along at least one axis; and (c) a coupling-out region associated with at least the coupling-out region of the LOE. A display system is also provided comprising: (d) a coupling-out arrangement, wherein the coupling-out arrangement is configured to redirect at least a portion of image illumination propagating within the LOE to an eye motion box for viewing by a user's eye; (e) a controller comprising at least one processor, the controller associated with the temperature sensor and the electrically controllable variable lens, and the controller generating an actuation signal to vary optical power to at least partially compensate for temperature-related variations in optical properties of the image projector in response to the signal indicative of the temperature.

[0027] According to a further feature of the invention, the controller is further configured to store a value indicative of a correction required for manufacturing tolerances of the image projector, and the controller is further configured to operate the electrically controllable variable lens to provide both correction for the manufacturing tolerances of the image projector and at least partial compensation for temperature related variations in optical properties of the image projector.

[0028] In accordance with a further feature of the invention, there is also provided a user input device associated with the controller, the controller being responsive to user input provided via the user input device during the calibration process to store new values ​​indicative of corrections required for manufacturing tolerances of the image projector. [Brief description of the drawings]

[0029] The invention is herein described, by way of example only, with reference to the accompanying drawings. [Figure 1A] FIG. 1 is a schematic side view of a display system for displaying an image to a user's eye, constructed and operative in accordance with an embodiment of the present invention, in which coupling of the image from a waveguide to the eye is achieved by a reflective and diffractive device, respectively. [Figure 1B] FIG. 1 is a schematic side view of a display system for displaying an image to a user's eye, constructed and operative in accordance with an embodiment of the present invention, in which coupling of the image from a waveguide to the eye is achieved by a reflective and diffractive device, respectively. [Figure 1C] FIG. 2 is a block diagram of an implementation of the display system of FIGS. 1A and 1B. [Figure 2A] 1A-1D are schematic representations of the operation of a Pancharatnam Berry phase lens in the context of the present invention, shown in transmissive and reflective configurations, respectively; [Figure 2B] 1A-1D are schematic representations of the operation of a Pancharatnam Berry phase lens in the context of the present invention, shown in transmissive and reflective configurations, respectively; [Diagram 3] FIG. 2 is an enlarged partial view of an image projector according to an implementation of the system of FIGS. 1A and 1B, illustrating the positioning of an adjustable optical device according to one embodiment of the present invention. [Figure 4A(i)] FIG. 2 shows three exemplary points in an angular view of the display of FIGS. 1A-1C. [Figure 4B(i)] FIG. 2 shows three exemplary points in an angular view of the display of FIGS. 1A-1C. [Figure 4C(i)]FIG. 2 shows three exemplary points in an angular view of the display of FIGS. 1A-1C. [Figure 4A(ii)] 4A(i), 4B(i), and 4C(i) are graphs showing the depth of focus as a function of the off-axis angle at the entrance aperture of a waveguide corrected with a variable lens for successive exemplary points in FIGS. 4A(i), 4B(i), and 4C(i). [Figure 4B(ii)] 4A(i), 4B(i), and 4C(i) are graphs showing the depth of focus as a function of the off-axis angle at the entrance aperture of a waveguide corrected with a variable lens for successive exemplary points in FIGS. 4A(i), 4B(i), and 4C(i). [Figure 4C(ii)] 4A(i), 4B(i), and 4C(i) are graphs showing the depth of focus as a function of the off-axis angle at the entrance aperture of a waveguide corrected with a variable lens for successive exemplary points in FIGS. 4A(i), 4B(i), and 4C(i). [Figure 4A(iii)] 1 is a close-up plot of the angular spread of three exemplary points for three correction states of a variable lens. [Figure 4B(iii)] 1 is a close-up plot of the angular spread of three exemplary points for three correction states of a variable lens. [Figure 4C(iii)] 1 is a close-up plot of the angular spread of three exemplary points for three correction states of a variable lens. [Figure 4C(iv)] FIG. 4C(ii) is a plot showing chromic separation of a white dot with the implementation of variable lens correction of FIG. 4C(ii). [Figure 5A(i)] 4C(i) is a graphical representation of a plot of focal plane dot position as a function of position across the aperture for the peripheral exemplary points of FIG. 4C(i) for tangential and sagittal aberrations. [Figure 5B(i)] Similar to FIG. 5A(i) after adding cylindrical correction. [Figure 5A(ii)] 5A(i) and 5B(i), respectively, and corresponding spot size diagrams. [Figure 5B(ii)] 5A(i) and 5B(i), respectively, and corresponding spot size diagrams. [Figure 6A]FIG. 1C is an enlarged partial view of an image projector according to an implementation of the system of FIGS. 1A and 1B, illustrating the positioning of an adjustable optical device according to an embodiment of the invention using a scanning laser image generator. [Figure 6B] 6B is a partial view of an alternative deployment position of the variable lens according to a modified implementation of the system of FIG. 6A. [Figure 6C] 6B is a partial view of an alternative deployment position of the variable lens according to a modified implementation of the system of FIG. 6A. [Figure 7] FIG. 1C is a schematic side view of a further alternative implementation of an image projector for the system of FIGS. 1A and 1B using a scanning laser image generator with dual-pupil imaging. [Figure 8] FIG. 1C is a further schematic side view of a further alternative implementation of an image projector for the system of FIGS. 1A and 1B using a scanning laser image generator with dual-pupil imaging. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] The present invention is a display system and corresponding method for displaying an image to a user's eye.

[0031] The principles and operation of a display system according to the present invention may be better understood with reference to the drawings and the accompanying description.

[0032] Referring now to the drawings, FIGS. 1A and 1B show schematic diagrams of the main optical components of the system, while FIG. 1C is a block diagram of a system according to a specific embodiment of the present invention.

[0033] In general, the invention is illustrated herein with reference to a display system 500 for displaying an image to a user's eye 502, the eye being positioned within an eye motion box 504. The display system includes a light guide optical element (LOE) 506 having a pair of parallel major exterior surfaces 508, 510. An image projector 512, which projects image illumination of a collimated image, is optically coupled to the LOE 506 to introduce the image illumination into the LOE for propagation within the LOE by internal reflection at the major exterior surfaces 508, 510. The image projector 512 is interchangeably referred to as a "POD."

[0034] The coupling-out arrangement, associated with at least the coupling-out region of the LOE 506, is configured to redirect at least a portion of the image illumination propagating within the LOE to the eye motion box 504 for viewing by the user's eye 502. A typical implementation of the coupling-out arrangement includes a reflective coupling-out device implemented as a plurality of mutually parallel partially reflective surfaces 514 that are angled relative to the major exterior surfaces 508, 510 within the LOE 506, for example, as shown in FIG. 1A. An alternative coupling-out device uses one or more diffractive optical elements 516 associated with the LOE 506 and configured to couple out a portion of the image illumination, as shown in FIG. 1B. Implementation details of both reflective and diffractive coupling-out devices for guided light displays are well known in the art and will not be described in detail here for the sake of brevity.

[0035] A particularly preferred feature of certain embodiments of the invention is that the image projector 512 includes at least one electrically controllable variable lens, which is represented diagrammatically in FIG. 1C by optical change device 10 and / or polarization change device 13. Various examples of the structure, function and positioning of the at least one electrically controllable variable lens are described below, but in a particularly preferred implementation of the invention, the variable lens is positioned in the optical path before the collimated image enters the LOE.

[0036] In general, the variable lens facilitates certain corrections and / or adjustments of the output image of the image projector 512 to improve the perceived quality of the image seen by the user. In most implementations, the variable lens operates under the control of a controller 18 including at least one processor. According to a first aspect of the invention, the controller 18 is configured to determine a current region of interest of the image and to generate actuation signals that vary characteristics of the electrically controllable variable lens to reduce at least one type of aberration in the current region of interest of the image seen by the user's eye at the expense of an increase in aberration in at least one area of ​​the image outside the current region of interest.

[0037] The present invention exploits the observation that human vision is sensitive to image quality at the fovea, which corresponds to a narrow field located at the center of the viewing line of sight, while the peripheral vision is much less sensitive to image quality.

[0038] According to this aspect of the invention, an adaptive optics component (variable lens) is operated to continuously modify the optical properties of the projector to minimize or reduce aberrations in the field of interest or "region of interest" where image quality is most needed, at the expense of exacerbating aberrations in other regions, e.g., peripheral regions of the user's field of view where image degradation is typically not perceptible or at least less significant. This approach allows the use of optical systems with more significant aberrations than would otherwise be tolerated, thereby allowing the use of smaller, less expensive, and / or more compact optical systems. It should be noted that the phrases "region of interest" and "field of interest" are used interchangeably herein. The term "region" refers to the region of the image, and "field" refers to the angular region of the user's field of view. However, since the image coupled out of the LOE is collimated to infinity, the "region" of the image is entirely defined by the corresponding angular "field."

[0039] The region of interest can be derived in a number of ways. In one set of implementations in which an eye-tracking device is deployed to track the current gaze of the user's eyes, the controller 18 preferably determines the current region of interest based on input received from the eye-tracking device. Because most aberrations vary relatively smoothly across the field of view, optimizing the estimated current gaze can provide aberration reduction over an area large enough to account for gaze measurement errors and the dimensions of the foveal viewing region.

[0040] In other implementations, particularly useful when real-time eye tracking data is not available, the region of interest is defined according to the content of the displayed image. Thus, for example, if an area of ​​the image has text subtitles, the area of ​​the subtitles may be assumed to be of high priority to provide aberration reduction to improve clarity. Similarly, in an augmented reality application where only a portion of the display area is used at a given moment for the display of information or objects overlaid on the real world, the currently displayed area is prioritized over display areas that are not currently active. The region of interest can be defined as the centroid of the currently active area or by using further algorithms to prioritize between these active areas depending on the content.

[0041] Analysis of the image content to determine the current region of interest may be performed in real time by the controller 18, typically by implementing suitable image processing algorithms. These algorithms implement the aforementioned analyses, and other similar analyses, to identify possible areas of interest in the image and define the region of interest accordingly. This typically involves searching for one or more of the following: currently active regions of the display, regions of the image containing text or other high resolution image elements, regions of the image containing faces, regions of the image having motion relative to previous frames of a video image. All of these image processing algorithms are easily implemented using well known image processing techniques and may be performed using functions in standard image processing libraries. The priority between different types of content may be selected by the system designer depending on the intended use of the system, and the system may be switchable between two or more modes depending on the current use of the system, either automatically based on detection of a type of image content, or by user input.

[0042] Alternatively, in some cases, a preselected definition of the region of interest may be provided to the controller 18 as a data stream indicative of the current region of interest associated with the video image. The data stream may optionally be encoded along with the video source or, in the case of dynamically generated graphics, may be provided as an additional output by a processor that generates the displayed graphical elements.

[0043] It should be noted that the term "controller" is used here to refer to a device that generates actuation signals that directly or indirectly adjust the characteristics of the variable lens. A controller typically includes one or more processors, one or more data storage devices, and various interfaces for input and output, all of which are known in the art. Physically, the controller can be a single unit, or its functionality can be subdivided into two or more units, the controller being integrated with a processing system that performs various additional functions, for example related to driving an image projector or any other required function. The processor may be located as part of the display assembly, or it may be divided between different locations linked by suitable communication links, and its various functions may be divided in any desired manner. The processor or processors may be any type of processor, including a general-purpose processor operating under a suitable operating system and configured by suitable software, or as dedicated hardware, or by any combination of hardware and software.

[0044] The term "tunable lens" is used herein generically to refer to any device that modifies the phase components of light passing through or reflected at an aperture. "Tunable lens" includes lenses that have optical power that is adjustable along at least one axis, and devices that variably affect the polarization of light. Examples of such devices are described below.

[0045] The term light guide optical element (LOE) is used interchangeably herein with "waveguide" and "substrate" and refers to a light guide element bounded by at least two major planar parallel outer surfaces through which a collimated image propagates by internal reflection. For augmented reality applications, the LOE is also referred to as a "combiner" in that it combines a projected image with a real-world image. The use of planar parallel surfaces together with a collimated image facilitates an aperture expansion by partial coupling out of the image illumination over an extended area of ​​the substrate. Optionally, the LOE can include a set of multiple features deployed to achieve aperture expansion in multiple dimensions, as described in PCT Patent Application Publication No. 2020 / 049542 (unpublished at the priority date of this application and not prior art).

[0046] An "image" as referred to in this application may be any image covering either the entire area of ​​the display or any region within the display area. The image is usually part of a series of images or frames, providing an apparently time-varying image, referred to as a "video image" or simply "video". It should be noted that the image and / or video need not be a full-frame (e.g., rectangular) image, but instead may define various isolated regions, graphic elements, or objects that are displayed. Furthermore, the image or video need not be stored or streamed in a graphical format, but instead may be generated using vector graphics and / or any other rendering hardware or software. In either case, the signal presented to the user's eye is a graphical output, usually defined as a set of pixel locations and intensity / color values, and is referred to herein as an "image".

[0047] The extent of the "region of interest" where aberrations are reduced is not necessarily clearly delimited, depending on the spatial variation of the aberrations in question. Often the region of interest can be defined in a particular direction, e.g., by the center of the region of interest where aberration reduction is optimized, and the effect of the correction on the remainder of the field essentially follows. However, because the aberrations addressed in this aspect of the invention are those that vary across the field, adjustments will typically optimize aberration reduction for a given type of aberration over a small number of angular fields of view, causing an increase in the same aberration in at least one other region in the field.

[0048] Looking now in more detail at FIG. 1C, this illustrates in schematic form the components of a non-limiting implementation of the system.

[0049] The optical variation device 10 and the polarization variation device 13 modify the optical properties of the image transmitted by the image generator 12. The modified image is transmitted to a combiner (e.g., a waveguide) 14 and to the observer's eye 16. A controller 18 (control devices 10 and 13) preferably receives some or all of the following real-time parameters:

[0050] 1. The orientation of the observer's gaze. This information is provided by the eye tracker system 20. (In particular, the present invention only requires fairly low-resolution eye tracking information that indicates the area of ​​interest in which the observer is looking. For example, an accuracy of approximately 5 degrees is typically sufficient for the implementation of the present invention.)

[0051] 2. Instantaneous Color Illumination: A color sequence illumination system transmits different color images (red, green, and blue) in different time slots. Synchronization information regarding which particular color is illuminated in a particular time slot is preferably received from the image controller 22.

[0052] 3. System Temperature: A parameter detected by one or more temperature sensors 24 positioned at key locations in the system.

[0053] Simulation or testing of the system can be used to generate look-up tables for the necessary corrections of the aberrations produced by the above parameters, i.e. line of sight (field), color, and temperature. In most cases, the corrections required for the various parameters are additive, but more complex correction matrices can be stored in the system memory / data storage 26. The controller 18 can thus actuate the variable lens to optimize the optical image quality in the portion of the field of interest. The look-up tables can take into account various types of aberrations, which may also include aberrations associated with the scanning dynamics of the scanning system.

[0054] In operation, in case of eye-tracking based operation, the controller 18 receives gaze input from the eye tracker 20 and operates one or both of the devices 10 and 13 to optimize the focus and / or optical power and / or polarization along at least one axis for minimum aberration and / or maximum signal strength in the current region of interest where the observer is looking. The correction preferably also takes into account the output of the image controller 22 and the thermal sensor 24 and can retrieve the appropriate correction from an aberration library stored in the data storage 26. The fact that the adjustment is optimized for a specific region of the field of view usually reduces the image quality in other regions of the field of view. However, since these regions are not currently near the observer's line of sight, the degradation in quality in the peripheral field is well tolerated and usually even unnoticeable.

[0055] Introducing optical correction by device 10 also introduces some distortion into the image in some cases. This distortion is usually manifested as an image shift in a localized narrow field of interest (on the line of sight). This shift can generate misalignment in case of binocular systems where the corrections of the two projectors are different. To address this, a distortion map is preferably also stored by memory 26 so that driver 18 can send the predicted distortion (or its correction) to image processor module 28 for pre-compensation.

[0056] Various devices and techniques can be used to implement electrically controllable variable lenses for improving optical properties within the field of interest. Mechanical lens movement by voice coils or piezo actuators is an option, but may be limited in speed and reliability. A more preferred option is a condenser-based device such as VARIOPTIC™ or OPTOTUNE™ adjustable focus lenses available from CORNING™ (USA). An even more preferred option is a liquid crystal-based component, which can be used at high speeds with minimal degradation effects. For example, focusing lenses as described in patent application publication no. 2006 / 022346 can be advantageously used. Alternatively, more complex correction functions such as astigmatism and sphericity can be implemented using one or more Pancharatnam-Berry phase (PBP) lenses (also called "geometric phase lenses"). An even more preferred option is a controllable liquid crystal lens, such as those available from DeepOptics Ltd. (Israel).

[0057] Optionally, a user input (not shown) may be provided to allow user feedback to the controller 18 for focus adjustment. This manual focus adjustment input may provide calibration to compensate for degradation of the optical quality of the device over time.

[0058] Figures 2A and 2B show the polarization aspects of this system implementation based on PBP lenses in a schematic way, but can be adapted to other polarization-based devices. Figure 2A shows a configuration 67 with a pass-through implementation where linearly polarized light enters a λ / 4 waveplate 70 and is converted to circularly polarized light. This light passes through a PBP lens 71 to obtain optical power (not shown) and reverse circular polarization as required. If necessary, another λ / 4 waveplate 72 is used to convert the output image to linearly polarized light.

[0059] The integration of a PBP lens into a polarizing beam splitter (PBS) is shown as scheme 69 in FIG. 2B. The linearly polarized input light, in addition to being acted upon by the optical power of the PBP lens, passes through a λ / 4 waveplate 76 and a PBP lens 77 to become circularly polarized as in FIG. 2A. Here, a λ / 2 waveplate 78 inverts the polarization orientation, and a reflector 79 inverts the polarization orientation once more. As the light passes back through the components, it gains more optical power and emerges with a linear polarization orthogonal to the incident light. In some implementations, this latter configuration can be advantageous, providing a 2x optical power correction for a given tunable lens device.

[0060] An example of an optical system using the present invention is shown in Figure 3. The system is based on a projector 512 that introduces image illumination into a waveguide (LOE 506). The image illumination into the waveguide must be collimated (image at infinity) in a region of the field of view (around the observer's current line of sight) that corresponds to the region of interest.

[0061] In a non-limiting example of an optical distribution device based on a polarizing beam splitter prism, illumination light 50 (from a source not shown) reflects from a polarizing beam splitter (PBS) 52 onto an image generating matrix 54 (e.g., an LCOS spatial light modulator). Alternatively, illumination source 50 may be a scanning laser beam (not shown here, but described further below), in which case 54 is typically a microlens array (MLA) in combination with a reflector, diffuser, or most preferably, a quarter wave plate. The image light is reflected back through the PBS 52 to a reflecting lens 56 (preferably spherical for minimal cost) and back to the PBS 52. The light reflects from the PBS 52 and is coupled into a waveguide 506 via any suitable coupling device, such as, for example, a coupling prism 60.

[0062] If a polarization change device (device 13 in FIG. 1C) is desired, it is preferably implemented as a liquid crystal device positioned as shown at 59. Optical power correction for aberration compensation (device 10 in FIG. 1C) can be implemented using an adjustable lens placed at 58A (equivalent to scheme 67 described above in FIG. 2A) or 58B (equivalent to scheme 69 described above in FIG. 2B). If it is placed at 58B, the polarization shift required for correct routing of the optical path at PBS 52 is preferably provided by the configuration shown in scheme 69. Otherwise, another quarter-wave plate is placed before the reflecting lens 56, as known in the art.

[0063] Images from uncompensated display systems typically have significant aberrations. The most dominant aberration in this configuration is usually "field curvature." Figures 4A(i)-4A(iii) show a conventional approach to minimizing distortion due to field curvature.

[0064] FIG. 4A(i) is a diagram 80 showing the angular field of the system and three points selected for analysis: the center of field 82, a point at the mid-radius of field 84, and the edge of field 86. FIG. 4A(ii), graph 90, shows the field curvature, where the x-axis is the local focal length and the y-axis is the radius of the field. Curves 92 and 96 show the sagittal and tangential focal lengths, respectively, where line 96 represents the position of the system (the position of the image plane actually transmitted). The distance from 96 to 92 and 96 represents the local defocus, which results in a non-collimated image at the corresponding particular field position. Minimizing this distortion across the field is conventionally achieved by optimizing the image plane relative to point 84a (marked with an arrow) so that plot 90 has a minimum distance at 84b. In this way, the defocus represented by the distance from 82b and 86b to line 96, averaged over the entire field, is minimized.

[0065] This solution is clearly imperfect, as shown in plot 100 of FIG. 4A(iii), which shows the central wavelength (green) spot of 82a as 82c, and similarly 84c and 86c for points 84a and 86a, respectively. It is clear that 84 is of the best quality (in focus) while the other image points are much more spread out. The result is poor image quality in the center and at the edges of the field.

[0066] According to one aspect of the invention, dynamic adjustment of focus is performed according to the current line of sight of the observer's eyes, so that focus is optimized in the area the observer is currently looking at (called the current region of interest or "foveal area"), but with reduced quality in areas in the observer's peripheral field of view. This approach is effective in improving perceived image quality across the entire field of view, while taking advantage of the relative insensitivity of peripheral human vision to image quality, such that further image degradation in the peripheral field of view is well tolerated.

[0067] Thus, when the eye tracker detects that the gaze is in the center of the field (indicated by arrow 102 in FIG. 4B(i)), the change device 58B is activated, for example by −½ diopters, to shift the focus to be optimal in this field of interest. (In this example, half the required optical power is required if the active optical device is deployed at 58B in FIG. 3.) Plot 104 in FIG. 4B(ii) illustrates the adjusted focus across the field, showing that the focus in region 82b intersects line 96, indicating that this region of the image is correctly collimated at the input to the light guide optics. FIG. 4B(iii) illustrates the point spread functions obtained for the three reference points, showing a significant improvement in the mid-field spot 82c and a substantial degradation at the edge of the field 86c. Although not apparent in this figure, the spot size also degrades slightly in the mid-field 84c.

[0068] When the region of interest changes to the edge of the field (FIG. 4C(i), arrow 108), then the optical change device 58A is adjusted, for example, to +1 diopter (or half that amount at 58B), resulting in a corresponding improvement in image quality near the edge of the field, as shown in plots 110 and 112 in FIGS. 4C(ii) and 4C(iii). In this case, the improvement comes at the expense of a decrease in image quality in the central field (now in the "periphery" of the observer's field of view).

[0069] In some cases, the effective optical power of the optical variation device 10 varies with wavelength so as to affect different colors that make up the color projection image differently. In FIG. 4C(iv), plot 114 shows, in a schematic and enlarged manner, the spots of red, green, and blue color separation that make up the white pixel in the image during the +1 diopter correction described above. In this example (considered to be typical, but non-limiting), it is clear that all colors are effectively corrected for their focus, but have different magnifications in the field of interest (here appearing as sharp spots, but shifted relative to each other). This can be compensated for in the following way.

[0070] 1. Introducing a different magnification factor into a digital image processor to produce an equal but opposite displacement of the image data information in one or more of the colors; or

[0071] 2. If the colors are illuminated in sequence and the response time of the adjustable lens is fast enough (such as having LCD lens technology), the optical power of the variation device 58 can be changed for the illumination of every color to compensate for this magnification variation.

[0072] More complex aberration compensation (such as a combination of astigmatism and sphere) requires a more complex optical profile to be generated by the variation device 58. This can be achieved, for example, by using an LCD lens manufactured by DeepOptics Ltd. (Israel).

[0073] Other optical configurations and display technologies can also use the above approach of adjusting focus and / or polarization and / or other optical properties that affect image quality across the field of view depending on the current region of interest, which may be derived from input from an eye tracker or determined from the image content, adjusting properties to provide an improved image in the current region of interest. Other related technologies include, but are not limited to, laser scanners to projectors based on waveguides and micro LEDs.

[0074] The variable lens can be placed in other locations than the exemplary location shown in FIG. 3, as exemplified below.

[0075] Turning now to the polarization adjustment device 13, some waveguide-based near-eye displays exhibit image non-uniformity when polarized light is launched into the waveguide. Device 13 in FIG. 1 and 59 in FIG. 3 depict polarization management devices that preferably control the exit polarization to achieve improved image uniformity in the field of interest. In this case, a look-up table must be stored in the driver memory 26 to define the optimal polarization states for various parts of the field. This table can be empirically determined based on laboratory tests performed once per product, or derived based on theoretical calculations, with or without empirical confirmation. The driver then activates device 59 to generate polarization states according to the field of interest (e.g., the current line of sight). It should be noted that the focus correction and the polarization correction are each considered to be of patentable importance in themselves, but can be used together with certain synergistic effects.

[0076] Astigmatism correction Astigmatism can exist in circularly symmetric optical systems and becomes important in the case of misalignment of the optical system or when cylindrical optics are present in the system. Figure 5A(i) shows the aberration of field point 86C of the system of Figure 3 when a misalignment of 1 mm is introduced at the entrance pupil. Plot 150 shows the tangential aberration, where the x-axis is the position of the ray in the aperture and the y-axis is the position of this ray in the focal plane. In this plot, the flat lines represent the smallest aberration. It is clear that plot 150, which represents the defocusing of these rays, is tilted. Plot 152 shows the plot of the curved but relatively flat sagittal rays. These aberration plots are transformed into a focal spot 154 with an elongated shape (Figure 5A(ii)).

[0077] To correct these aberrations, preferably a cylindrical correction is added to the circularly symmetric correction of device 58A, which has a focal length of 1.3 meters. This independent cylindrical correction can be introduced by an additional LCD layer with cylindrical optical power. As a result, the tangential rays 156 are corrected (flatter plot) and the sagittal rays 158 are unchanged (Figure 5B(i)). This results in a reduction of the spot height, as seen at 160 (Figure 5B(ii)).

[0078] Implementing laser scanning using a reduced dimension variable focus lens When a laser scanning system is used to generate the projected image, the small dimensions of the beam in the optical path near or in front of the scanning device allow the use of particularly compact variable-focus lenses, thereby relaxing design requirements and allowing faster response times.

[0079] An exemplary laser scanning system is shown in FIG. 6A. A laser 200 with collimating optics 201 transmits light to scanning mirrors 202V (vertical) and 202H (horizontal) resulting in a scanning beam (schematically shown as diverging arrows from 202H) and collimating optics similar to those shown above with reference to FIG. 3. In this case, element 54 may be a mirror, preferably combined with a diffuser, or most preferably a microlens array (MLA), defining an image plane with controlled divergence of the subsequent laser illumination. It should be noted that the PBS prism-based optics shown here are non-limiting and free-space optics implementations using refractive lenses and microlens arrays through which the light is transmitted are fully equivalent to, and in some cases preferred, to the configuration exemplified here. If free-space transmissive optics are used, a polarization manipulator (polarization change device 13 above) may be used, which can be implemented at any point along the optical path since this configuration typically does not have the optical sensitivity of the image projector components.

[0080] To maintain the coupling of the laser beam into the waveguide, the waveguide entrance pupil 204 is imaged as close as possible to the mirror 202 in an optical arrangement called "pupil imaging". Lens 58C is a variable focus lens. The variable focus lens can be implemented using any of the aforementioned technologies, including but not limited to liquid crystal lenses such as those commercially available from Deep-Optics Ltd. (Israel), or electrostatically actuated lenses such as those commercially available under the name VARICOTIC® by Corning™ or Optotune™. Alternatively, in any of the implementations described herein that use reflective optical elements, the variable optical element may be implemented as, for example, a variable curvature reflector replacing surface 56. This can be implemented by coating an electrostatic lens with a reflective coating. For the purposes of this description, an exemplary device is shown here with a variable lens, but it should be understood that in either case, a variable optical power reflector is also applicable. All of these variable lenses respond faster with a smaller active aperture.

[0081] For very low optical powers, the variable lens 58C can be incorporated into the laser lens 201. When multiple color lasers (red, green, and blue) are projected along the same axis, the variable lens is preferably positioned on the common optical path after multiplexing to combine all of the beams. Details of the laser implementation are not shown here.

[0082] From Figure 6A it is clear that the spatial extent of the laser beam is narrowest at positions near and in front of scanning mirror 202. Thus, by placing variable lens 58C at these positions, a small optical aperture lens (e.g., having a diameter on the order of 1 mm) can be used. This facilitates the implementation of fast response times for variable optics.

[0083] The variation in optical power required for lens 58C is relatively small so that it does not substantially change the divergence and width of the laser beam. A further option is to place this lens close to mirror 202V / H, again with minimal impact on the beam size on the mirror and minimal degradation of the pupil imaging device. As a result, the power loss at the mirror and pupil 204 is minimized. Figure 6B shows a schematic arrangement in which variable lens 58D is between mirrors 202V / H, while Figure 6C shows variable lens 58E after mirrors 202V / H, but close enough to the mirror that the spatial spread of the scanning beam is still relatively small and a small variable lens can be used.

[0084] Display systems in general, and variable lenses in particular, can be implemented using various optical architectures based on reflective and / or transmissive optics. FIG. 7 shows another example of an optical device with dual pupil imaging. The entrance pupil 204 of the waveguide 62 is imaged onto the mirror 202V and once again onto the mirror 202H. The variable optics can be integrated with or implemented adjacent to the lenses 58F1 and 58F2, or the reflectors 58G1 and 58G2. Multiple variable lenses can be used, for example a cylindrical variable LCD can be used at 58F1 and a cylindrical variable LCD with an orthogonal orientation can be used at 58F2. Cylindrical or spherical variable lenses can also be implemented using any suitable technology (e.g. LCD and electrostatic).

[0085] Lens 58H may also be implemented as a variable lens, optionally in combination with laser optics, as described above.

[0086] Turning now to FIG. 8, in some cases of laser scanning image projectors, the projector can be treated as two optical subsections: the laser section 1300U from the scanning laser 1201 to the microlens array (MLA) 1329, and the relay section from the MLA to the exit pupil 1334 (entrance pupil to the waveguide). To minimize the field curvature aberration in the relay section, the MLA 1329 can be made of a curved section as shown. However, this curvature does not correspond to the preferred curvature of the laser section. Therefore, according to one aspect of the invention, the lens 1223 can be set to adapt to generate an optimal laser beam focus on the appropriate section of the MLA (and thereby the field of interest). This lens is small, allowing for quick corrections. It is preferable to place this lens close to the scanning mirror. Alternatively, the lens 1227 can be set to be adaptive, but in this case a larger lens is required.

[0087] Active optical compensation of the laser section is also preferred because thermal gradients induced by the laser source may otherwise inherently distort the optics of the laser section more than the relay section.

[0088] In this example, laser 1201 may be implemented as multiple side-by-side lasers (e.g., red, green, and blue), which are close to each other so that the field curvature is approximately the same. In this case, the compensation implemented by 1223 may be set as the average of the positions of the three lasers, or according to one of the centrally positioned adjacent lasers.

[0089] The relay subsection 1300L from the MLA to the exit pupil can be considered an independent optical system and can optionally include an additional variable lens (typically integrated with one of the lenses illustrated here) to provide additional aberration correction and / or polarization adjustment.

[0090] Omnidirectional correction All of the examples of the invention described thus far include identification of a current region of interest that changes dynamically with eye motion and / or image content. However, it should be noted that the inclusion of an electrically controllable variable lens and corresponding controller may be advantageously used in accordance with various additional aspects of the invention, even when corrections are made globally for the entire field. Examples of such corrections include, but are not limited to, corrections for variations caused by temperature changes, calibration corrections made to compensate for manufacturing tolerances, user-input focus to correct for degradation of image quality due to various factors over time, etc. In each of these cases, the corrections provided herein are performed at the image projector level, i.e., before the image illumination enters the LOE, to enhance collimation of the image incident on the LOE.

[0091] Thus, according to a further aspect of the invention, corresponding, for example, to a subset of features from FIGS. 1A-1C, a display system for displaying an image to a user's eye includes an LOE 506, all as described above, and an image projector 512, including an electrically controllable variable lens 10 having variable optical power along at least one axis. A temperature sensor 24 is integrated with the system, typically as part of the optical housing, and generates a signal indicative of a temperature of at least a portion of the display system. A controller 18 (all as described above) is responsive to the signal indicative of the temperature to generate an actuation signal that varies the optical power to at least partially compensate for temperature-related variations in the optical properties of the image projector. The corrections are typically retrieved from a look-up table stored in a data storage device 26, also as described above.

[0092] Additionally or alternatively, factory corrections (or corresponding adjustments to the look-up table) may be provided based on a post-production calibration process, allowing compensation for a range of accumulated errors or aberrations that may arise due to component tolerances and / or assembly variations in the mass production process.

[0093] Additionally or alternatively, a user input (not shown) may be provided to enable user feedback to the controller 18 for focus adjustment. This manual focus adjustment input can provide a calibration to compensate for degradation of the optical quality of the device over time. Thus, the controller responds to user input provided via the user input device during the calibration process to store new values ​​indicative of the correction required for manufacturing tolerances of the image projector.

[0094] It should be understood that the above description is intended to serve as an example only, and that many other embodiments are possible within the scope of the invention as defined in the appended claims.

Claims

1. 1. A display system for displaying an image to an eye of a user, said eye being positioned within an eye motion box, said display system comprising: (a) a light guide optical element (LOE) having a pair of parallel outer major surfaces; (b) an image projector for projecting image illumination of a collimated image, the image projector being optically coupled to the LOE such that the image illumination is introduced into the LOE for propagation within the LOE by internal reflection at the pair of major exterior surfaces, the image projector including an electrically controllable variable lens; (c) a coupling-out arrangement associated with at least a coupling-out region of the LOE, the coupling-out arrangement configured to redirect at least a portion of the image illumination propagating within the LOE to the eye motion box for viewing by the eye of the user; and (d) a controller comprising at least one processor, the controller being associated with the electrically controllable variable lens; and (i) determining a current region of interest of the image; (ii) generating an actuation signal to change characteristics of the electrically controllable variable lens so as to reduce at least one type of optical aberration in the current region of interest of the image viewed by the eye of the user at the expense of increasing aberration in at least one area of ​​the image outside the current region of interest.

2. 13. The display system of claim 1, further comprising an eye tracking device deployed to track a current gaze of the eye of the user, and wherein the controller determines the current region of interest based on input received from the eye tracking device.

3. The display system of claim 1 , wherein the image is a video image, and the controller processes content of the video image to derive the current region of interest.

4. The display system of claim 1 , wherein the image is a video image, and the controller receives a data stream indicating a current region of interest associated with the video image.

5. 2. The display system of claim 1, wherein the at least one type of optical aberration varies across a field of the collimated image from the image projector, the electrically controllable variable lens has variable optical power along at least one axis, and the controller varies the variable optical power to reduce the at least one type of optical aberration in a region of the collimated image corresponding to the current region of interest.

6. 6. The display system of claim 5, further comprising a temperature sensor generating a signal indicative of temperature, the controller varying the variable optical power in response to the signal indicative of temperature to at least partially compensate for temperature related variations in optical properties of the image projector.

7. 6. The display system of claim 5, wherein the image projector sequentially projects different colored image illumination at different time periods, and the controller varies the variable optical power in synchronization with the time periods to provide color-specific aberration compensation.

8. 6. The display system of claim 5, wherein the controller is further configured to store a value indicative of a correction required for manufacturing tolerances of the image projector, and wherein the controller is further configured to operate the electrically controllable variable lens to both provide a correction for the manufacturing tolerances of the image projector and to reduce the at least one type of optical aberration in a region of the collimated image corresponding to the current region of interest.

9. 9. The display system of claim 8, further comprising a user input device associated with the controller, the controller storing new values ​​indicative of corrections required for manufacturing tolerances of the image projector in response to user input provided via the user input device during a calibration process.

10. 2. The display system of claim 1, wherein a focal field of the collimated image from the image projector is non-uniform, the electrically controllable variable lens has a variable focal length, and the controller varies the focal length to enhance collimation of the collimated image within the region of interest.

11. 2. The display system of claim 1, wherein the uniformity of the image observed at the eye motion box varies across the field of the image as a function of the polarization of the image illumination introduced to the LOE, the electrically controllable variable lens is a variable polarization correcting lens, and the controller varies the variable polarization correcting lens to reduce non-uniformity in an area of ​​the image corresponding to the current region of interest.

12. 12. A display system according to any one of claims 1 to 11, wherein the image projector comprises a narrow beam illumination source and a scanning device for generating a scan pattern of the narrow beam across an angular field of the image, and the electrically controllable variable lens is disposed in an optical path between the narrow beam illumination source and the scanning device or adjacent to the scanning device.

13. 12. The display system of claim 1, wherein the image projector comprises a spatial light modulator and a collimating optical system, and the electrically controllable variable lens is disposed in an optical path between the collimating optical system and the LOE or is integrated with the collimating optical system.

14. The image projector includes: (a) a narrow beam illumination source; (b) a scanning device for generating a scan pattern of the narrow beam across an angular field of the image to generate a real image at an image plane; (c) a collimating optic that collimates image illumination from the image plane so as to be introduced into the LOE as a collimated image; the electrically controllable variable lens is disposed in an optical path between the image plane and the LOE; 12. The display system of claim 1, further comprising a second electrically controllable variable lens disposed in an optical path between the narrow beam illumination source and the image plane.

15. 12. A display system according to any preceding claim, wherein the coupling-out arrangement comprises a plurality of mutually parallel partially reflective surfaces in the LOE, the partially reflective surfaces being oblique to the outer major surfaces.

16. 12. The display system of claim 1, wherein the coupling-out arrangement comprises a diffractive optical element associated within the LOE and configured to couple out a portion of the image illumination.

17. 1. A method for displaying an image to a user's eye, the eye being positioned within an eye motion box, the method comprising: (a) providing a display system, said display system comprising: (i) a light guide optical element (LOE) having a pair of parallel outer major surfaces; (ii) an image projector for projecting image illumination of a collimated image, the image projector being optically coupled to the LOE such that the image illumination is introduced into the LOE and propagated within the LOE by internal reflection at the pair of major exterior surfaces, the image projector including an electrically controllable variable lens; (iii) providing a coupling-out arrangement associated with at least a coupling-out region of the LOE, the coupling-out arrangement configured to redirect at least a portion of the image illumination propagating in the LOE to the eye motion box for viewing by the eye of the user; (b) determining a current region of interest of the image; (c) varying a characteristic of the electrically controllable variable lens as a function of the current region of interest of the image so as to reduce at least one optical aberration in the region of interest at the expense of a reduction in image quality seen in at least one other region of the image.

18. The method of claim 17 , wherein the current region of interest is determined by detecting a current line of sight of the eye of the user.

19. The method of claim 17 , wherein the image is a video image and the current region of interest varies as a function of the content of the video image.

Images