Eye imaging with off-axis imager

Abstract

To provide favorable eye imaging with an off-axis imager.SOLUTION: Examples of an imaging system for use with a head mounted display (HMD) are disclosed. The imaging system can include a forward-facing imaging camera, and a surface of a display of the HMD can include an off-axis diffractive optical element (DOE) or hot mirror configured to reflect light to the imaging camera. The DOE or hot mirror can be segmented. The imaging system can be used for eye tracking, biometric identification, multiscopic reconstruction of the three-dimensional shape of an eye, etc.SELECTED DRAWING: Figure 7A

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of priority under 35 U.S.C.§119(e) to U.S. Provisional Application No. 62 / 222,671, filed on September 23, 2015, entitled "EYE IMAGING WITH AN OFF - AXIS IMAGER", which is hereby incorporated by reference in its entirety.

[0002] (Background) The present disclosure relates to virtual reality and augmented reality imaging and visualization systems, and more particularly to an imaging system for acquiring an image of an eye.

Background Art

[0003] (Description of Related Fields) Modern computing and display technologies have facilitated the development of systems for so - called "virtual reality" or "augmented reality" experiences, in which a digitally reproduced image or a portion thereof is presented to a user in a manner that appears or is perceived to be real. A virtual reality, i.e., a "VR" scenario typically involves the presentation of digital or virtual image information without transparency to other actual real - world visual inputs. An augmented reality, i.e., an "AR" scenario typically involves the presentation of digital or virtual image information as an augmentation of the visualization of the actual world around the user. Or mixed reality "MR" is related to the fusion of the real world and the virtual world to create a new environment in which physical and virtual objects co - exist and interact in real - time. In conclusion, the human visual perception system is very complex, and it is difficult to generate VR, AR, or MR technologies that facilitate a comfortable and natural - feeling rich presentation of virtual image elements among other virtual or real - world image elements. The systems and methods disclosed herein address various challenges related to VR, AR, and MR technologies.

Summary of the Invention

Means for Solving the Problem

[0004] Embodiments of a head-mounted display (HMD) configured to be worn on a user's head are disclosed. The HMD includes a frame having a pair of ear supports, a pair of optical elements supported by the frame such that each of the pair of optical elements can be disposed in front of the user's eyes, a forward imager mounted on one of the pair of ear supports, and a reflective element disposed within or on one of the pair of optical elements and configured to reflect infrared light toward the forward imager so as to receive the infrared light reflected by the reflective element. Each of the pair of optical elements can be transmissive to visible light. The reflective element can include a plurality of segments having the same or different optical properties. The imager can be configured to acquire an image of the eyes of the wearer of the HMD. The HMD can include a processor that analyzes an image acquired by the imager, estimates an accommodation state of the eyes, or images the retina of the eyes for eye tracking, biometric identification, or multi-viewpoint reconstruction of the shape of the eyes. This specification also provides, for example, the following items. (Item 1) A head-mounted display (HMD) configured to be worn on a user's head, the HMD comprising: a frame having a pair of ear supports; and a pair of optical elements, each of the pair of optical elements being supported by the frame such that each can be disposed in front of the user's eyes; and a forward imager mounted on one of the pair of ear supports; and a reflective element disposed within or on one of the pair of optical elements and configured to reflect infrared light toward the forward imager so as to receive the infrared light reflected by the reflective element. An HMD comprising the above. (Item 2) The HMD according to item 1, wherein each of the pair of optical elements is transparent to visible light. (Item 3) The HMD according to item 1, wherein each of the pair of optical elements is configured to display an image to the user. (Item 4) The HMD according to item 3, wherein each of the pair of optical elements includes a light field display. (Item 5) The HMD according to item 4, wherein the light field display includes a waveguide stack configured to output the image to the user. (Item 6) The HMD according to any one of items 1 to 5, wherein the reflective element includes a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis volumetric diffractive optical element (OAVDOE). (Item 7) The HMD according to item 1, wherein the reflective element is transparent to visible light. (Item 8) The HMD according to item 1, wherein the reflective element includes a plurality of segments, and each segment within the plurality of segments has an optical property that is different from the optical property of at least one other segment within the plurality of segments. (Item 9) The HMD according to item 8, wherein the optical property includes a reflection angle or a refractive power. (Item 10) The HMD according to item 8, wherein the plurality of segments includes 2, 3, 4, 5, 6, 7, 8, or 9 segments. (Item 11) The HMD according to item 1, wherein the forward imager is mounted on a temple portion of one of the pair of ear supports. (Item 12) The HMD according to item 1, wherein the imager includes a focus control lens assembly. (Item 13) The near - far control lens assembly includes an offset lens, a tilt lens, or an offset - tilt lens, the HMD according to item 12. (Item 14) A display system, An optical element configured to display an image to a user, the optical element being configured to be positioned in front of the user's eyes, the optical element, and A forward imager, A reflective element disposed within or on the optical element, the reflective element being configured to reflect infrared light received from the user's eyes toward the forward imager, the reflective element comprising a display system. (Item 15) The optical element includes a light field display, the display system according to item 14. (Item 16) The reflective element includes a hot mirror, an off - axis diffractive optical element (DOE), an off - axis holographic mirror (OAHM), or an off - axis volumetric diffractive optical element (OAVDOE), the display system according to item 14. (Item 17) The reflective element includes a plurality of segments having different refractive powers or different reflection angles, the display system according to item 14. (Item 18) A non - transitory memory configured to store an image of the user's eyes obtained by the forward imager, and A hardware processor communicating with the non - transitory memory, the hardware processor accessing the image of the eyes, tracking the user's eyes, extracting biometric information associated with the user's eyes, reconstructing a shape of a part of the user's eyes, estimating a near - far adjustment state of the user's eyes, or imaging the retina, iris, or other elements of the user's eyes, performing one or more of the following a hardware processor programmed to perform the following The display system according to any one of items 14 to 17, further comprising the following (Item 19) A head-mounted display system comprising a frame configured to support the display system according to item 14 such that the optical element is positioned in front of the first eye of the user (Item 20) The head-mounted display system according to item 19, wherein the frame supports the second display system according to item 14 such that the optical element of the second display system is positioned in front of the second eye of the user

[0005] Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages will be apparent from the description, drawings, and claims. Neither this summary nor any form of the following detailed description purports to define or limit the scope of the subject matter of the invention

Brief Description of the Drawings

[0006]

Figure 1

[0007]

Figure 2

[0008]

Figure 3

[0009]

Figure 4

[0010]

Figure 5

[0011]

Figure 6

[0012]

Figure 7A

Figure 7B

Figure 7C

Figure 7D

Figure 7E

Figure 7F

[0013]

Figure 7G

Figure 7H

[0014] Throughout the drawings, reference numbers may be reused to indicate correspondences between referenced elements. The drawings are provided to illustrate exemplary embodiments described herein and are not intended to limit the scope of the present disclosure. DETAILED DESCRIPTION

[0015] Overview The eyes of a wearer of a head-mounted display (HMD) can be imaged using an off-axis diffractive optical element (DOE). In some implementations, the DOE may be a holographic optical element (HOE), an off-axis holographic mirror (OAHM), or an off-axis volumetric diffractive optical element (OAVDOE). The resulting images can be used for, among other things, tracking one or both eyes, imaging the retina, reconstructing the eye shape in three dimensions, extracting biometric information (e.g., iris identification) from the eye, and the like.

[0016] A head-mounted display (HMD) can use information about the wearer's eye condition for various purposes. For example, this information can be used to estimate the wearer's line of sight or for biometric identification. However, imaging the eyes of an HMD wearer can be difficult. The distance between the HMD and the wearer's eyes is short. Furthermore, while gaze tracking requires a larger field of view, biometric identification requires a relatively high number of pixels on the iris target. For an imaging system that will attempt to accomplish both of these purposes, the requirements of the two tasks are significantly opposed. Additionally, both problems can be further complicated by occlusion by eyelids and eyelashes.

[0017] Embodiments of the imaging systems described herein address some or all of these problems. For example, the imaging system can include an imager configured to view the wearer's eyes. The imaging system can be mounted in proximity to the wearer's temples (e.g., on a frame of a wearable display system, such as an ear support). In some embodiments, a second imager can be used for the wearer's other eye such that each eye is imaged separately. The imager can include an infrared digital camera that is sensitive to infrared radiation. The imager can be mounted to be forward-facing (in the direction of the wearer's vision) rather than rear-facing and directed towards the eyes. By positioning the imager closer to the wearer's ear, the weight of the imager can also be closer to the ear, and the HMD can be easier to wear compared to an HMD where the imager is rear-facing and positioned closer to the front of the HMD. Additionally, by placing the forward-facing imager in the vicinity of the wearer's temple, the distance from the wearer's eyes to the imager is approximately doubled compared to a rear-facing imager positioned in the vicinity of the front of the HMD. Since the depth of field of the image is approximately proportional to this distance, the depth of field for the forward-facing imager is approximately doubled compared to the rear-facing imager. The larger depth of field for the imager can be advantageous for imaging the wearer's eye region having a large or protruding nose, supraorbital arch, etc.

[0018] The imager can be positioned to view the inner surface of an optical element that is otherwise transparent. The optical element can be part of a display of an HMD (or a lens within a pair of glasses). The optical element can comprise a surface that reflects wavelengths in a first range but is substantially transmissive to wavelengths in a second range (different from the wavelengths in the first range). The wavelengths in the first range can be within the infrared, and the wavelengths in the second range can be within the visible. For example, the optical element can comprise a hot mirror that reflects infrared light but transmits visible light. Visible light from the outside world can be transmitted through the optical element and can be perceived by the wearer. In fact, the imaging system acts like a virtual imager that is retro-directed towards the wearer's eye. The virtual imager can image virtual infrared light that propagates from the wearer's eye through the optical element. The hot mirror (or other DOE described herein) can be disposed on the inner surface of the optical element, on the outer surface of the optical element, or within the optical element (e.g., a volumetric HOE).

[0019] Infrared radiation can include radiation with wavelengths in the range of 700 nm to 10 μm. Infrared radiation can include near-infrared radiation with wavelengths in the range of 700 nm to 1.5 μm. In many implementations, eye imaging is performed within the near-infrared at wavelengths of 700 nm to 900 nm. (3D display)

[0020] FIG. 1 depicts an illustration of an augmented reality scenario with a virtual reality object and an actual reality object that are visually recognized by a person. FIG. 1 depicts an augmented reality scene 100, and to a user of AR technology, a real-world park-like setting 110 is visible, featuring people, trees, buildings in the background, and a concrete platform 120. In addition to these items, the user of AR technology also "sees" a robotic statue 130 standing on the real-world platform 120 and a flying comic-like avatar character 140 that appears to be a anthropomorphic bumblebee, although these elements do not exist in the real world.

[0021] It is desirable for a three-dimensional (3D) display to generate a perspective adjustment response corresponding to the virtual depth for each point within the display's field of view in order to generate a true sense of depth, more specifically, a simulated sense of surface depth. If the perspective adjustment response for a display point does not correspond to the virtual depth of that point as determined by the binocular depth cues of convergence and stereopsis, the human eye experiences a vergence conflict, resulting in unstable imaging, harmful eye strain, headaches, and, in the absence of vergence information, a nearly complete lack of surface depth.

[0022] VR, AR, and MR experiences can be provided by a display system having a display that provides an image viewer corresponding to a plurality of depth planes. The images may vary for each depth plane (e.g., providing a slightly different presentation of the scene or object), are separately focused by the viewer's eyes, and thereby, based on the eye vergence required to focus on different image features of the scene located on different depth planes and / or based on observing different image features on different depth planes that are out of focus, can help provide depth cues to the user. As discussed anywhere in this specification, such depth cues provide a believable perception of depth.

[0023] Figure 2 illustrates an example of a wearable display system 200 that can be used to present a VR, AR, or MR experience to a display system wearer or viewer 204. The display system 200 includes a display 208 and various mechanical and electronic modules and systems to support the functions of the display 208. The display 208 may be coupled to a frame 212 that is wearable by a display system user, wearer, or viewer 204 and configured to position the display 208 in front of the wearer 204's eyes. The display 208 may be a light field display. In some embodiments, a speaker 216 is coupled to the frame 212 and positioned adjacent to the user's ear canal (in some embodiments, another speaker, not shown, is positioned adjacent to the user's other ear canal to provide stereo / formable acoustic control). The display 208 is operably coupled to a local data processing module 224 and can be mounted in various configurations, such as being fixedly attached to the frame 212 by a wired conductor or wireless connection, fixedly attached to a helmet or hat worn by the user, built into headphones, or removably attached to the user 204 in another manner (e.g., in a backpack configuration, in a belt attachment configuration).

[0024] The local processing and data module 224 may include a hardware processor and a non-transitory digital memory such as a non-volatile memory (e.g., flash memory), both of which can be used to assist in data processing, caching, and storage. The data may include (a) data captured from sensors such as an image capture device (e.g., a camera), a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a wireless device, and / or a gyroscope (e.g., operably coupled to frame 212 or otherwise attachable to user 204), and / or (b) data obtained and / or processed using remote processing module 228 and / or remote data repository 232, possibly for passage through display 208 after such processing or reading. The local processing and data module 224 may be operably coupled to the remote processing module 228 and / or the remote data repository 232 via communication links 236 and / or 240, such as via a wired or wireless communication link, such that these remote modules 228, 232 are available as resources to the local processing and data module 224. Additionally, the remote processing module 228 and the remote data repository 232 may be operably coupled to each other.

[0025] In some embodiments, the remote processing module 228 may include one or more processors configured to analyze and process data such as video information and / or image information captured by an image capture device. The video data may be stored locally within the local processing and data module 224 and / or the remote data repository 232. In some embodiments, the remote data repository 232 may include a digital data storage facility, which may be available through the Internet or other networking configurations in a "cloud" resource configuration. In some embodiments, all data is stored and all calculations are performed in the local processing and data module 224, enabling complete autonomy from the remote modules.

[0026] The human visual system is complex, and it is difficult to provide a realistic perception of depth. Although not limited by theory, it is thought that an object viewer can perceive an object in three dimensions due to a combination of vergence and accommodation. The vergence of the two eyes relative to each other (i.e., the rotational movement of the pupils towards or away from each other to converge the lines of sight of the eyes and fix them on an object) is closely associated with the focusing (or "accommodation") of the eye's lens. Under normal conditions, changing the focus of the eye's lens, or accommodating the eye, to change the focus from one object to another at a different distance will automatically cause a coordinated change in vergence at the same distance under the relationship known as the "accommodation-vergence reflex". Similarly, a change in vergence will, under normal conditions, induce a matching change in accommodation. A display system that provides better coordination between accommodation and vergence can form a more realistic or comfortable simulation of a three-dimensional image.

[0027] Figure 3 illustrates a side view of an approach for simulating a 3D image using multiple depth planes. Referring to Figure 3, objects at various distances from eyes 302 and 304 on the z-axis are focused by eyes 302 and 304 such that those objects are in focus. Eyes 302 and 304 take on a particular focused state and focus the objects at different distances along the z-axis. As a result, a particular focused state can be said to be associated with a particular one of depth planes 306 having an associated focal length such that an object or a part of an object in that particular depth plane is in focus when the eyes are in the focused state with respect to that depth plane. In some embodiments, the 3D image may be simulated by providing different presentations of the image for each of eyes 302 and 304 and also by providing different presentations of the image corresponding to each of the depth planes. Although shown as being distinct for purposes of clarity, it should be understood that the fields of view of eyes 302 and 304 may overlap, for example, as the distance along the z-axis increases. Additionally, although shown as being flat for purposes of facilitating illustration, it should be understood that the contours of the depth planes may be curved in physical space such that all features within the depth plane are in focus with the eyes in a particular focused state. Without being limited by theory, it is believed that the human eye can typically interpret a finite number of depth planes to provide depth perception. As a result, a highly realistic simulation of the perceived depth can be achieved by providing the eyes with different representations of the image corresponding to each of these limited number of depth planes. (Waveguide stack assembly)

[0028] FIG. 4 illustrates an example of a waveguide stack for outputting image information to a user. Display system 400 includes a stack of waveguides or a stacked waveguide assembly 405 that can be utilized to provide three-dimensional perception to eye 410 or the brain using a plurality of waveguides 420, 422, 424, 426, 428. In some embodiments, display system 400 may correspond to system 200 of FIG. 2, and FIG. 4 schematically shows some parts of that system 200 in more detail. For example, in some embodiments, waveguide assembly 405 may be integrated within display 208 of FIG. 2.

[0029] Continuing to refer to FIG. 4, waveguide assembly 405 may also include a plurality of features 430, 432, 434, 436 between the waveguides. In some embodiments, features 430, 432, 434, 436 may be lenses. In some embodiments, features 430, 432, 434, 436 may not be lenses. Rather, they may be spacers (e.g., cladding layers and / or structures for forming an air gap).

[0030] Waveguides 420, 422, 424, 426, 428 and / or a plurality of lenses 430, 432, 434, 436 may be configured to transmit image information to the eye with various levels of wavefront curvature or ray divergence. Each waveguide level may be associated with a specific depth plane and may be configured to output image information corresponding to that depth plane. Image input devices 440, 442, 444, 446, 448 may be utilized to input image information into waveguides 420, 422, 424, 426, 428, each of which may be configured to disperse incident light across an individual waveguide for output toward the eye 410. Light exits from the output surface of image input devices 440, 442, 444, 446, 448 and is input into the corresponding input edges of waveguides 420, 422, 424, 426, 428. In some embodiments, a single beam of light (e.g., a collimated beam) may be input into each waveguide and output an entire field of cloned collimated beams directed toward the eye 410 at a specific angle (and amount of divergence) corresponding to the depth plane associated with a particular waveguide.

[0031] In some embodiments, image input devices 440, 442, 444, 446, 442 are discrete displays, each of which generates image information for input into the corresponding waveguides 420, 422, 424, 426, 428, respectively. In some other embodiments, image input devices 440, 442, 444, 446, 448 are the output ends of a single multiplexed display that may pipe image information to each of image input devices 440, 442, 444, 446, 448, for example, via one or more optical conduits such as optical fiber cables.

[0032] The controller 450 controls the operation of the stacked waveguide assemblies 405 and the image input devices 440, 442, 444, 446, 448. In some embodiments, the controller 450 includes programming (e.g., instructions in a non-transitory computer-readable medium) that adjusts the timing and provision of image information to the waveguides 420, 422, 424, 426, 428. In some embodiments, the controller 450 may be a single integrated device or a distributed system connected by wired or wireless communication channels. The controller 450 may, in some embodiments, be part of the processing module 224 or 228 (illustrated in FIG. 2). In some embodiments, the controller may communicate with an inward-facing imaging system 452 (e.g., a digital camera), an outward-facing imaging system 454 (e.g., a digital camera), and / or a user input device 466. The inward-facing imaging system 452 (e.g., a digital camera) captures an image of the eye 410 and can be used, for example, to determine the size and / or orientation of the pupil of the eye 410. The outward-facing imaging system 454 can be used to image a portion of the world 456. The user can input commands to the controller 450 via the user input device 466 and interact with the display system 400.

[0033] Waveguides 420, 422, 424, 426, 428 may be configured to propagate light within each individual waveguide by total internal reflection (TIR). Each of waveguides 420, 422, 424, 426, 428 may be planar, or have another shape (e.g., curved), with a major top and bottom surface and an edge extending between those major top and bottom surfaces. In the illustrated configuration, each of waveguides 420, 422, 424, 426, 428 includes light extraction optical elements 460, 462, 464, 466, 468 configured to redirect light, propagate it within each individual waveguide, and extract the light from the waveguide by outputting image information to eye 410 from the waveguide. The extracted light may also be referred to as external coupled light, and the light extraction optical elements may also be referred to as external coupling optical elements. The beam of extracted light is output by the waveguide at the location where the light propagating within the waveguide impinges on the light redirecting element. The light extraction optical elements (460, 462, 464, 466, 468) may be, for example, reflective and / or diffractive optical features. For ease of explanation and clarity of the drawings, they are shown disposed on the bottom major surface of waveguides 420, 422, 424, 426, 428, but in some embodiments, the light extraction optical elements 460, 462, 464, 466, 468 may be disposed on the top and / or bottom major surfaces and / or may be disposed directly within the volume of waveguides 420, 422, 424, 426, 428. In some embodiments, the light extraction optical elements 460, 462, 464, 466, 468 may be attached to a transparent substrate and formed within a layer of the material forming waveguides 420, 422, 424, 426, 428. In some other embodiments, waveguides 420, 422, 424, 426, 428 may be a monolithic piece of material, and the light extraction optical elements 460, 462, 464, 466, 468 may be formed on and / or within the surface of that piece of material.

[0034] Continuing to refer to FIG. 4, as discussed herein, each of the waveguides 420, 422, 424, 426, 428 is configured to output light and form an image corresponding to a particular depth plane. For example, the waveguide 420 closest to the eye may be configured to deliver collimated light to the eye 410 as it is input into such waveguide 420. The collimated light may represent an optically infinite focal plane. The next waveguide 422 may be configured to output collimated light that passes through a first lens 430 (e.g., a negative lens) before reaching the eye 410. The first lens 430 may be configured to generate a somewhat convex wavefront curvature such that the eye / brain interprets the light resulting from the next waveguide 422 as originating from a first focal plane that is closer inwardly from the optically infinite towards the eye 410. Similarly, the third waveguide 424 passes its output light through both the first lens 430 and a second lens 432 before reaching the eye 410. The combined refractive power of the first and second lenses 430 and 432 may be configured to generate another increment of wavefront curvature such that the eye / brain interprets the light resulting from the third waveguide 424 as originating from a second focal plane that is even closer inwardly from the optically infinite towards the person than the light from the next waveguide 422.

[0035] Other waveguide layers (e.g., waveguides 426, 428) and lenses (e.g., lenses 434, 436) are similarly configured to send their output through all of the lenses between them and the eye for the aggregated focus power representing the focal plane closest to the person using the highest waveguide 428 in the stack. When viewing / interpreting light originating from the world 456 on the other side of the stacked waveguide assembly 405, a compensation lens layer 438 may be disposed on top of the stack to compensate for the stack of lenses 430, 432, 434, 436. Such a configuration provides the same number of perceived focal planes as there are available waveguide / lens pairs. Both the light extraction optical elements 460, 462, 464, 466, 468 of waveguides 420, 422, 424, 426, 428 and the focusing sides of lenses 430, 432, 434, 436 may be static (e.g., not dynamic or electrically active). In some alternative embodiments, either or both may be dynamic using electrically active features.

[0036] Continuing to refer to FIG. 4, the light extraction optical elements 460, 462, 464, 466, 468 may be configured to both redirect light from their respective waveguides and output this light using an appropriate amount of divergence or collimation for a particular depth plane associated with the waveguide. As a result, waveguides having different associated depth planes may have light extraction optical elements of different configurations that output light with different amounts of divergence depending on the associated depth plane. In some embodiments, as discussed herein, the light extraction optical elements 460, 462, 464, 466, 468 may be three-dimensional or surface features configured to output light at a specific angle. For example, the light extraction optical elements 460, 462, 464, 466, 468 may be three-dimensional holograms, surface holograms, and / or diffraction gratings. Light extraction optical elements such as diffraction gratings are described in U.S. Patent Publication No. 2015 / 0178939, published Jun. 25, 2015, which is incorporated herein by reference in its entirety. In some embodiments, the features 430, 432, 434, 436 may not be lenses. Rather, they may simply be spacers (e.g., cladding layers and / or structures for forming an air gap).

[0037] In some embodiments, the light extraction optical elements 460, 462, 464, 466, 468 are diffraction features that form a diffraction pattern, i.e., a “diffractive optical element” (also referred to herein as a “DOE”). Preferably, the DOE has a relatively low diffraction efficiency such that only a portion of the light of the beam is deflected towards the eye 410 using each intersection of the DOE while the remainder continues to travel through the waveguide via total internal reflection. The light carrying the image information is thus split into several associated output beams that exit the waveguide at multiple locations, resulting in a very uniform output emission pattern towards the eye 410 with respect to this particular collimated beam that bounces within the waveguide.

[0038] In some embodiments, one or more DOEs may be switchable between an “on” state that actively diffracts and an “off” state that does not significantly diffract. For example, a switchable DOE may comprise a layer of polymer dispersed liquid crystal in which microdroplets form a diffraction pattern in a host medium, and the refractive index of the microdroplets can be switched to substantially match that of the host material (in which case the pattern does not significantly diffract incident light), or the microdroplets can be switched to a refractive index that does not match that of the host medium (in which case the pattern actively diffracts incident light).

[0039] In some embodiments, the number and distribution of depth planes and / or depth of field may vary dynamically based on the pupil size and / or orientation of the viewer's eye. In some embodiments, an imaging system 452 (e.g., a digital camera) facing inward may be used to capture an image of the eye 410 and determine the size and / or orientation of the pupil of the eye 410. In some embodiments, the imaging system 452 facing inward may be attached to the frame 212 (as shown in FIG. 2) and may communicate electrically with processing modules 224 and / or 228 that process the image information from the imaging system 452 facing inward and can determine, for example, the pupil diameter and / or orientation of the user 204's eye.

[0040] In some embodiments, the inward-facing imaging system 452 (e.g., a digital camera) can observe user movements such as eye and face movements. The inward-facing imaging system 452 can capture an image of the eye 410 and may be used to determine the size and / or orientation of the pupil of the eye 410. The inward-facing imaging system 452 can be used to determine the direction in which the user is looking (e.g., eye pose), or to obtain an image for user biometric identification (e.g., via iris identification). The image obtained by the inward-facing imaging system 452 may be analyzed to determine the user's eye pose and / or mood and can be used by the display system 400 to determine the audio or visual content to be presented to the user. The display system 400 may also use sensors such as an inertial measurement unit (IMU), an accelerometer, a gyroscope, etc. to determine the head pose (e.g., head position or head orientation). The head pose may be used, alone or in combination with the eye pose, to interact with the support tracking and / or to present audio content.

[0041] In some embodiments, one camera may be utilized per eye to separately determine the pupil size and / or orientation of each eye, thereby allowing the presentation of image information to each eye to be dynamically adjusted to that eye. In some embodiments, at least one camera may be utilized per eye to independently determine the pupil size and / or eye pose of each eye separately, thereby allowing the presentation of image information to each eye to be dynamically adjusted to that eye. In some other embodiments, only the pupil diameter and / or orientation of one eye 410 (e.g., using only a single camera per pair of eyes) is determined and assumed to be similar for both eyes of the viewer 204.

[0042] For example, the depth of field may vary inversely with the pupil size of the viewer. As a result, as the size of the pupil of the viewer's eye decreases, one plane that is indistinguishable because its location in the plane exceeds the depth of focus of the eye becomes distinguishable, and as the pupil size decreases and the depth of field increases correspondingly, it increases so as to appear more in focus. Similarly, the number of spaced-apart depth planes used to present different images to the viewer may be decreased with the decreased pupil size. For example, the viewer may not be able to clearly perceive the details of both the first depth plane and the second depth plane at one pupil size without adjusting the focusing of the eye from one depth plane to the other. However, these two depth planes may be sufficient to focus on the user at another pupil size without changing the focusing.

[0043] In some embodiments, the display system may vary the number of waveguides that receive image information based on the determination of pupil size and / or orientation, or in response to receiving an electrical signal indicating a particular pupil size and / or orientation. For example, if the user's eye is indistinguishable between two depth planes associated with two waveguides, the controller 450 may be configured or programmed to stop providing image information to one of these waveguides. Advantageously, this may reduce the processing burden on the system, thereby increasing the responsiveness of the system. In embodiments where the DOE for the waveguide is switchable between on and off states, the DOE may be switched to the off state when the waveguide receives image information.

[0044] In some embodiments, it may be desirable to satisfy the condition that the emitted beam has a diameter less than the diameter of the viewer's eye. However, satisfying this condition can be difficult in light of the variability of the size of the viewer's pupil. In some embodiments, this condition is satisfied over a wide range of pupil sizes by varying the size of the emitted beam in response to a determination of the size of the viewer's pupil. For example, as the pupil size decreases, the size of the emitted beam may also decrease. In some embodiments, the emitted beam size may be varied using a variable aperture.

[0045] The display system 400 can include an outward-facing imaging system 454 (e.g., a digital camera) that images a portion of the world 456. This portion of the world 456 can be referred to as the field of view (FOV), and the imaging system 454 can also be referred to as the FOV camera. The entire area available for viewing or imaging by the viewer 204 can be referred to as the field of regard (FOR). The FOR may include a solid angle of 4π steradians surrounding the display system 400. In some implementations of the display system 400, the FOR may include substantially all of the solid angle around the user 204 of the display system 400 so that the user 204 can move their head and eyes to view objects surrounding the user (in front of, behind, above, below, or to the side of the user). The image obtained from the outward-facing imaging system 454 can be used to track gestures made by the user (e.g., hand or finger gestures) and to detect objects within the world 456 in front of the user.

[0046] The display system 400 can include a user input device 466 through which a user can input commands to the controller 450 and interact with the display system 400. For example, the user input device 466 can include a trackpad, a touch screen, a joystick, a multi-degree-of-freedom (DOF) controller, a capacitive sensing device, a game controller, a keyboard, a mouse, a directional pad (D-pad), a wand, a haptic device, a totem (e.g., functioning as a virtual user input device), and the like. In some cases, the user may use a finger (e.g., a thumb) to press or swipe on a touch sensor-based input device to provide input to the display system 400 (e.g., to provide the user input to a user interface provided by the display system 400). The user input device 466 may be held by the user's hand during use of the display system 400. The user input device 466 can communicate with the display system 400 either wired or wirelessly.

[0047] FIG. 5 shows an embodiment of an output beam output by a waveguide. One waveguide is shown, but other waveguides within waveguide assembly 405 may function similarly, and it should be understood that waveguide assembly 405 includes a plurality of waveguides. Light 505 is introduced into waveguide 420 at input edge 510 of waveguide 420 and propagates within waveguide 420 by TIR. At the point where light 505 impinges on DOE 460, a portion of the light exits the waveguide as output beam 515. Output beams 515 are shown as being substantially parallel, but they may also be redirected to propagate at an angle to eye 410 depending on the depth plane associated with waveguide 420 (e.g., to form a diverging output beam). It should be understood that a substantially parallel output beam may represent a waveguide with an optical extraction optical element that externally couples the light and forms an image that appears to be set on a depth plane at a long distance (e.g., optical infinity) from eye 410. Other waveguides or other sets of optical extraction optical elements may output a more diverging output beam pattern that requires eye 410 to focus at a closer distance and be interpreted by the brain as light from a distance closer to eye 410 than optical infinity.

[0048] FIG. 6 shows another embodiment of a display system 400 that includes a waveguide device, an optical coupler subsystem for optically coupling light to or from the waveguide device, and a control subsystem. The display system 400 can be used to generate multi-focus stereoscopic, images, or light fields. The display system 400 can include one or more primary planar waveguides 604 (only one is shown in FIG. 6) and one or more DOEs 608 associated with at least some of the respective primary waveguides 604. The planar waveguide 604 can be similar to the waveguides 420, 422, 424, 426, 428 discussed with reference to FIG. 4. The optical system can employ a diffractive waveguide device to relay light along a first axis (vertical or Y-axis in the view of FIG. 6) and expand the effective exit pupil of the light along the first axis (e.g., the Y-axis). The diffractive waveguide device can include, for example, a diffractive planar waveguide 612 and at least one DOE 616 (illustrated by the dashed line) associated with the diffractive planar waveguide 612. The diffractive planar waveguide 612 can be similar or identical to the primary planar waveguide 604 having a different orientation at at least some points. Similarly, at least one DOE 616 can be similar or identical to the DOE 608 at at least some points. For example, the diffractive planar waveguide 612 and / or the DOE 616 can each be made of the same material as the primary planar waveguide 604 and / or the DOE 608. The optical system shown in FIG. 6 can be integrated into the wearable display system 200 shown in FIG. 2.

[0049] The relayed light with an expanded exit pupil is optically coupled into one or more primary planar waveguides 604 from the diffractive waveguide device. The primary planar waveguide 662 preferably relays light along a second axis (e.g., horizontal or X-axis in the view of FIG. 6) that is orthogonal to the first axis. It should be noted that the second axis can be a non-orthogonal axis with respect to the first axis. The primary planar waveguide 604 expands the effective exit path of the light along its second axis (e.g., X-axis). For example, the diffractive planar waveguide 612 can pass the light through the primary planar waveguide 604 that relays and expands the light along the vertical or Y-axis and relays and expands the light along the horizontal or X-axis.

[0050] The display system 400 may include one or more colored light sources (e.g., red, green, and blue laser light) 620 that can be optically coupled into the proximal end of the single-mode optical fiber 624. The distal end of the optical fiber 624 may be screwed or received through the hollow tube 628 of the piezoelectric material. The distal end projects from the tube 628 as a flexible cantilever 632 that is not fixed. The piezoelectric tube 628 can be associated with four quadrant electrodes (not shown). The electrodes may be deposited, for example, on the outside, outer surface or outer periphery, or diameter of the tube 628. A core electrode (not shown) is also located at the core, center, inner periphery, or inner diameter of the tube 628.

[0051] For example, the drive electronics 636, which are electrically coupled via the wire 640, drive the opposing pair of electrodes and bend the piezoelectric tube 628 independently about two axes. The protruding distal tip of the optical fiber 624 has a mechanical resonance mode. The resonance frequency may depend on the diameter, length, and material properties of the optical fiber 624. By vibrating the piezoelectric tube 628 near the first mechanical resonance mode of the fiber cantilever 632, the fiber cantilever 632 can be vibrated and swept through a large deflection.

[0052] By stimulating resonant vibrations along two axes, the tip of the fiber cantilever 632 is scanned in two axial directions within the area filling a two-dimensional (2-D) scan. By modulating the intensity of the light source 620 in synchronization with the scan of the fiber cantilever 632, light emitted from the fiber cantilever 632 forms an image. An explanation of such a setup is provided in U.S. Patent Publication No. 2014 / 0003762, which is incorporated herein by reference in its entirety.

[0053] The component 644 of the optical coupler subsystem collimates the light emitted from the scanning fiber cantilever 632. The collimated light is reflected by the mirrored surface 648 into the narrow-dispersion planar waveguide 612 containing at least one diffractive optical element (DOE) 616. The collimated light propagates vertically (with respect to the view in FIG. 6) along the dispersion planar waveguide 612 by total internal reflection, thereby repeatedly intersecting the DOE 616. The DOE 616 preferably has a low diffraction efficiency. This diffracts a portion of the light (e.g., 10%) towards the edge of the larger primary planar waveguide 604 at each point of intersection with the DOE 616 and allows a portion of the light to continue on its original trajectory along the length of the dispersion planar waveguide 612 via TIR.

[0054] At each point of intersection with the DOE 616, additional light is diffracted towards the entrance of the primary waveguide 612. By splitting the incident light into a plurality of external coupling sets, the exit pupil of the light is vertically expanded by the DOE 616 within the dispersion planar waveguide 612. The vertically expanded light externally coupled from the dispersion planar waveguide 612 enters the edge of the primary planar waveguide 604.

[0055] The light entering the primary waveguide 604 propagates horizontally along the primary waveguide 604 (with respect to the figure in FIG. 6) via TIR. As the light intersects the DOE 608 at multiple points, it propagates horizontally along at least a portion of the length of the primary waveguide 604 via TIR. The DOE 608 preferably has a phase profile that is advantageously the sum of a linear diffraction pattern and a radially symmetric diffraction pattern and can be designed or configured to produce both deflection and focusing of the light. The DOE 608 preferably has a low diffraction efficiency (e.g., 10%) such that only a portion of the light of the beam is deflected towards the viewer's eye at each intersection of the DOE 608, while the remainder of the light continues to propagate through the primary waveguide 604 via TIR.

[0056] At each point of intersection between the propagating light and the DOE 608, a portion of the light is diffracted towards the adjacent surface of the primary waveguide 604, allowing the light to escape from TIR and be emitted from the surface of the primary waveguide 604. In some embodiments, the radially symmetric diffraction pattern of the DOE 608 additionally imparts a certain focal level to the diffracted light and shapes (e.g., imparts curvature to) the wavefronts of the individual beams and steers the beams to an angle that matches the designed focal level.

[0057] Thus, these different paths can couple light outside the primary planar waveguide 604 by resulting in different multiplicity of the DOE 608, focus levels, and / or filling patterns at different angles and in the exit pupil. The different filling patterns in the exit pupil can advantageously be used to generate a light field display with multiple depth planes. Each layer within the waveguide assembly or a set of layers within a stack (e.g., three layers) may be employed to generate an individual color (e.g., red, blue, green). Thus, for example, a first set of three adjacent layers may be employed to generate red, blue, and green light, respectively, at a first depth of focus. A second set of three adjacent layers may be employed to generate red, blue, and green light, respectively, at a second depth of focus. Multiple sets may be employed to generate a full 3D or 4D color image light field with various depths of focus. (Exemplary optical system for imaging the eye using an off-axis imager)

[0058] The eyes of a wearer of a head-mounted display (HMD) (e.g., the wearable display system 200 shown in FIG. 2) can be imaged using a reflective off-axis diffractive optical element (DOE) (which may be a holographic optical element (HOE) in some implementations). The resulting image can be used for, e.g., tracking one or both eyes, imaging the retina, reconstructing the eye shape in three dimensions, extracting biometric information from the eye (e.g., iris identification), etc.

[0059] There are various reasons why a head-mounted display (HMD) may use information about the wearer's eye condition. For example, this information can be used to estimate the wearer's line of sight or for biometric identification. However, this problem is difficult due to the short distance between the HMD and the wearer's eye. This is further complicated by the fact that gaze tracking requires a larger field of view while biometric identification requires a relatively high pixel count on the iris target. For an imaging system that will attempt to perform both of these objectives, the requirements of the two tasks are significantly opposed. Finally, both problems are further complicated by occlusion by the eyelids and eyelashes. Embodiments of the imaging system described herein address some or all of these problems. Various embodiments of the imaging system 700 described herein with reference to FIGS. 7A-7H can be used in conjunction with an HMD, including the display devices described herein (e.g., the wearable display system 200 shown in FIG. 2, the display systems 400 shown in FIGS. 4 and 6).

[0060] FIG. 7A schematically illustrates an embodiment of an imaging system 700 that is used to view the eye 304 and is mounted adjacent to the wearer's temple (e.g., on the ear support of the wearable display system 200, e.g., the frame 212) and includes an imager 702b. In other embodiments, a second imager is used for the other eye 302 of the wearer such that each eye is imaged separately. The imager 702b can include an infrared digital camera that is sensitive to infrared radiation. The imager 702b is mounted so as to be forward-facing (in the direction of the wearer's vision) and not rearward-facing and is directed towards the eye 304 (like the camera 452 shown in FIG. 4). By positioning the imager 702b closer to the wearer's ear, the weight of the imager 702b also gets closer to the ear, and the HMD can be more comfortable to wear compared to an HMD where the imager is rearward-facing and positioned closer to the front of the HMD (e.g., proximate to the display 208). Additionally, by installing the forward-facing imager 702b in the vicinity of the wearer's temple, the distance from the wearer's eye 304 to the imager is approximately doubled compared to a rearward-facing imager positioned in the vicinity of the front of the HMD (e.g., compared to the camera 452 shown in FIG. 4). Since the depth of field of the image is approximately proportional to this distance, the depth of field for the forward-facing imager 702b is approximately doubled compared to the rearward-facing imager. The larger depth of field of the imager 702b can be advantageous for imaging the wearer's eye region having a large or protruding nose, supraorbital arch, etc.

[0061] Imager 702b is positioned to view the inner surface 704 of the optical element 706, which is transparent in other respects. The optical element 706 can be part of the display 208 of the HMD (or lenses within a pair of glasses). The optical element can be transmissive to at least 10%, 20%, 30%, 40%, 50%, or more of the visible light incident on the optical element. In other embodiments, the optical element 706 need not be transmissive (e.g., in a virtual reality display). The optical element 706 can comprise a reflective element 708. The reflective element 708 can be a surface that reflects a first range of wavelengths but is substantially transmissive to a second range of wavelengths (different from the first range of wavelengths). The first range of wavelengths can be within the infrared, and the second range of wavelengths can be within the visible. For example, the reflective element 708 can comprise a hot mirror that reflects infrared light but transmits visible light. In such embodiments, infrared light 710a, 712a, 714a from the wearer propagates to the optical element 706 and is reflected therefrom, resulting in reflected infrared light 710b, 712b, 714b, which can be imaged by the imager 702b. In some embodiments, the imager 702b can be sensitive to, or capable of capturing, at least a subset (non-empty subset and / or less than all subset, etc.) of the first range of wavelengths reflected by the reflective element 708. For example, the reflective element 708 can reflect infrared light in the range of 700 nm to 1.5 μm, and the imager 702b can be sensitive to, or capable of capturing, near-infrared light at wavelengths in the range of 700 nm to 900 nm. As another example, the reflective element 708 can reflect infrared light in the range of 700 nm to 1.5 μm, and the imager 702b can include a filter that filters out infrared light in the range of 900 nm to 1.5 μm so that the imager 702b can capture near-infrared light at wavelengths in the range of 700 nm to 900 nm.

[0062] Visible light from the outside world 456 is transmitted through the optical element 706 and can be perceived by the wearer. In fact, the imaging system 700 shown in FIG. 7A acts like a virtual imager 702c that is retro-directed towards the wearer's eye 304. The virtual imager 702c can image virtual infrared light 710c, 712c, 714c (shown as dotted lines) that propagates from the wearer's eye 304 through the optical element 706. A hot mirror (or other DOE described herein) can be disposed on the inner surface 704 of the optical element 706, but this is not limiting. In other embodiments, the hot mirror or DOE can be disposed on the outer surface of the optical element 706 or within the optical element 706 (e.g., a volumetric HOE).

[0063] FIG. 7B schematically illustrates another example of the imaging system 700. In this embodiment, perspective distortion can be reduced or eliminated by the combined use of a perspective control lens assembly 716b (e.g., a shift lens assembly, a tilt lens assembly, or a tilt-shift lens assembly) and the imager 702b. In some embodiments, the perspective control lens assembly 716b can be part of the lens of the imager 702b. The perspective control lens 716b can be configured such that the normal to the imager 702b is substantially parallel to the normal to the region of the surface 704 that includes a DOE (or HOE) or a hot mirror. In fact, the imaging system 700 shown in FIG. 7B acts like a virtual imager 702c with a virtual perspective control lens assembly 716c that is retro-directed towards the wearer's eye 304.

[0064] Additionally, or alternatively, as schematically shown in FIG. 7C, the reflective element 708 of the optical element 706 may have an off-axis holographic mirror (OAHM) that is used to facilitate viewing of the eye 304 by the camera imager 702b that reflects the light 710a, 712a, 714a on its surface 704 and captures the reflected light 710b, 712b, 714b. The OAHM 708 may also have refractive power, in which case it can be an off-axis volume diffractive optical element (OAVDOE) as schematically shown in FIG. 7D. In the embodiment shown in FIG. 7D, the effective location of the virtual camera 702c is at infinity (not shown in FIG. 7D).

[0065] In some embodiments, the HOE (e.g., OAHM or OAVDOE) can be divided into a plurality of segments. Each of these segments can have different optical properties or characteristics, including, for example, the reflection angle or refractive power at which the segment reflects the incident (infrared) light. The segments can be configured such that light is reflected from each segment toward the imager 702b. As a result, the image acquired by the imager 702b is also divided into a corresponding number of segments, each effectively viewing the eye from a different angle. FIG. 7E schematically illustrates an embodiment of the display system 700 having an OAHM with three segments 718a1, 718a2, 718a3 that act as individual virtual cameras 702c1, 702c2, 702c3 that image the eye 304 at different angular locations, respectively.

[0066] FIG. 7F schematically illustrates another embodiment of display system 700 having an OAHM with three segments 718a1, 718a2, 718a3, each having a refractive power (e.g., a segmented OAVDOE), where each segment generates a virtual camera at infinity and images the eye 304 at different angular locations. The three segments are schematically illustrated in FIGS. 7E and 7F, which is by way of example and not limitation. In other embodiments, two, four, five, six, seven, eight, nine, or more segments can be utilized. Any of these segments of the HOE may have no refractive power, or some or all of them may have refractive power.

[0067] The three segments 718a1, 718a2, 718a3 are shown spaced apart horizontally across the optical element 706 in FIGS. 7E and 7F. In other embodiments, the segments can be spaced apart vertically on the optical element 706. For example, FIG. 7G schematically shows a DOE 718 having two vertically spaced segments 718a1 and 718a2, where segment 718a1 is configured to retroreflect light toward imager 702b (which may be in the same substantially horizontal plane as segment 718a1), and segment 718a2 is configured to reflect light upward toward imager 702b. Similar to a bifocal lens, the arrangement shown in FIG. 7G can be advantageous in enabling the imaging system 700 to use a reflected image (schematically shown via the solid arrow lines) obtained by imager 702b from the upper segment 718a1 when the wearer is looking straight ahead through the upper portion of the HMD, and to use a reflected image (schematically shown via the dashed arrow lines) from the lower segment 718a2 when the wearer is looking downward through the lower portion of the HMD.

[0068] A mixture of horizontally separated segments and vertically separated segments can also be used in other embodiments. For example, FIG. 7H shows another embodiment of the HOE718 with a 3×3 array of segments. The imager 702b can acquire reflection data representing light rays originating from different areas and angular directions of the eye region from each of these nine segments. Two exemplary light rays propagating from the eye region of the HOE718 and retroreflected to the imager 702b are shown as solid and dashed lines. The imaging system 700 (or the processing module 224 or 228) can analyze the reflection data from the plurality of segments and computationally determine the three-dimensional shape of the eye or the line-of-sight direction of the eye (e.g., eye pose) in a multi-focal manner.

[0069] Embodiments of the optical system 700 utilizing segments can have a plurality of advantages. For example, the segments can be used individually by selecting a particular segment that is best suited for a particular task, or they can be used collectively to computationally estimate the three-dimensional shape or pose of the eye. In the former case, this selectivity can be used, for example, to select an image of the wearer's iris with the least occlusion by the eyelids or eyelashes. In the latter case, the three-dimensional reconstruction of the eye can be used to estimate the orientation (e.g., by estimating the location of corneal bulging) or the accommodation state (e.g., by estimating lens-induced distortion on the apparent location of the pupil). (Additional aspect)

[0070] In a first aspect, a head-mounted display (HMD) configured to be worn on a user's head is disclosed. The HMD includes a frame having a pair of ear supports, a pair of optical elements supported by the frame such that each of the pair of optical elements can be disposed in front of a user's eye, a forward imager mounted on one of the pair of ear supports, and a reflective element disposed within or on one of the pair of optical elements and configured to reflect infrared light toward the forward imager so as to receive the infrared light reflected by the reflective element.

[0071] In a second aspect, each of the pair of optical elements is the HMD according to aspect 1, which is transparent to visible light.

[0072] In a third aspect, each of the pair of optical elements is the HMD according to aspect 1 or 2, configured to display an image to the user.

[0073] In a fourth aspect, each of the pair of optical elements is the HMD according to aspect 3, including a light field display.

[0074] In a fifth aspect, the light field display is the HMD according to aspect 4, including a waveguide stack configured to output an image to the user.

[0075] In a sixth aspect, the reflective element is the HMD according to any one of aspects 1 to 5, including a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis volumetric diffractive optical element (OAVDOE).

[0076] In a seventh aspect, the reflective element is the HMD according to any one of aspects 1 to 6, which is transparent to visible light.

[0077] In an eighth aspect, the reflective element comprises a plurality of segments, each segment within the plurality of segments having an optical property that is different from the optical property of at least one other segment within the plurality of segments, the HMD according to any one of aspects 1 to 7.

[0078] In a ninth aspect, the optical property includes a reflection angle or a refractive power, the HMD according to aspect 8.

[0079] In a tenth aspect, the plurality of segments comprises 2, 3, 4, 5, 6, 7, 8, or 9 segments, the HMD according to aspect 8 or aspect 9.

[0080] In an eleventh aspect, the forward imager is mounted on a temple portion of one of a pair of ear supports, the HMD according to any one of aspects 1 to 10.

[0081] In a twelfth aspect, the imager comprises a focus control lens assembly, the HMD according to any one of aspects 1 to 11.

[0082] In a thirteenth aspect, the focus control lens assembly comprises an offset lens, a tilt lens, or an offset-tilt lens, the HMD according to aspect 12.

[0083] In a fourteenth aspect, a display system is disclosed. The display system comprises an optical element configured to display an image to a user, the optical element being configured to be positioned in front of the user's eyes, a forward imager, and a reflective element disposed within or on the optical element and configured to reflect infrared light received from the user's eyes towards the forward imager.

[0084] In a fifteenth aspect, the optical element comprises a light field display, the display system according to aspect 14.

[0085] In the 16th aspect, the reflective element comprises a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis volumetric diffractive optical element (OAVDOE), and the display system is as described in aspect 14 or 15.

[0086] In the 17th aspect, the reflective element comprises a plurality of segments having different refractive powers or different reflection angles, and the display system is as described in any one of aspects 14 to 16.

[0087] In the 18th aspect, the display system as described in any one of aspects 14 to 17 further comprises a non-transitory memory configured to store an image of the user's eye obtained by a forward imager, and a hardware processor communicating with the non-transitory memory, the hardware processor being programmed to access the image of the eye, track the user's eye, extract biometric information associated with the user's eye, reconstruct a part of the shape of the user's eye, estimate the accommodation state of the user's eye, or image the retina, iris, or other elements of the user's eye, and perform one or more of these actions.

[0088] In the 19th aspect, a head-mounted display system is disclosed. The HDM comprises a frame configured to support the display system as described in any one of aspects 14 to 18 such that the optical element is positioned in front of the user's first eye.

[0089] In the 20th aspect, the frame supports a second display system as described in any one of aspects 14 to 18 such that the optical element of the second display system is positioned in front of the user's second eye, and the head-mounted display system is as described in aspect 19.

[0090] In the 21st aspect, an imaging system is disclosed. The imaging system includes a reflective element that reflects light within a first wavelength range, and an imager that is sensitive to light within a non-empty subset less than all of the first wavelength range and is configured to be oriented to capture the light reflected by the reflective element.

[0091] In the 22nd aspect, the reflective element of the imaging system according to aspect 21 includes a hot mirror, a holographic optical element (HOE), an off-axis holographic mirror (OAHM), or an off-axis volume diffractive optical element (OAVDOE).

[0092] In the 23rd aspect, the imaging system according to any one of aspects 21 - 22, wherein the first wavelength range includes an infrared wavelength range.

[0093] In the 24th aspect, the imaging system according to any one of aspects 21 - 23 includes an optical element, the optical element includes a reflective element, and the optical element is transparent to at least 50% of the visible light incident on the optical element.

[0094] In the 25th aspect, the reflective element of the imaging system according to any one of aspects 21 - 24 includes a plurality of segments.

[0095] In the 26th aspect, the first segment within the plurality of segments has an optical property different from the optical property of the second segment within the plurality of segments, of the imaging system according to aspect 25.

[0096] In the 27th aspect, the optical property of the first segment within the plurality of segments or the optical property of the second segment within the plurality of segments includes a reflection angle or a refractive power, of the imaging system according to aspect 26.

[0097] In a 28th aspect, the imaging system described in any one of aspects 25 - 27, wherein the plurality of segments comprises at least two segments.

[0098] In a 29th aspect, the imaging system described in any one of aspects 25 - 28, wherein two of the plurality of segments are arranged horizontally.

[0099] In a 30th aspect, the imaging system described in any one of aspects 25 - 29, wherein two of the plurality of segments are arranged vertically.

[0100] In a 31st aspect, the imaging system described in any one of aspects 25 - 30, wherein some of the plurality of segments are arranged within a grid.

[0101] In a 32nd aspect, the imager further comprises a focus control lens assembly, the imaging system described in any one of aspects 21 - 31.

[0102] In a 33rd aspect, the focus control lens assembly comprises an offset lens, a tilt lens, or an offset - tilt lens, the imaging system described in aspect 32.

[0103] In a 34th aspect, an imaging system for indirectly capturing an image of a user's eye is disclosed. The imaging system comprises a reflective element that reflects light within a first wavelength range, the reflective element comprising an off - axis holographic mirror (OAHM) or an off - axis volume diffractive optical element (OAVDOE), the reflective element being oriented to reflect light propagating from the user's eye when the imaging system is placed in front of the user's eye, and an imager sensitive to light within a non - empty subset less than all of the first wavelength range, the imager being oriented to image an image of the user's eye by capturing light propagating from the user's eye reflected by the reflective element.

[0104] In the 35th aspect, the imaging system described in aspect 34 is such that the image of the user's eye captured by the imager and the image of the user's eye captured by the camera installed in front of the user's eye are indistinguishable.

[0105] In the 36th aspect, the imaging system described in aspect 35 is such that the image of the user's eye captured by the imager is virtually the same as the image of the user's eye captured by the camera installed in front of the user's eye.

[0106] In the 37th aspect, the imaging system described in any one of aspects 35 - 36 is such that the effective location of the camera installed in front of the user's eye is at infinity.

[0107] In the 38th aspect, the imaging system described in any one of aspects 35 - 37 is such that the first wavelength range includes the infrared wavelength range.

[0108] In the 39th aspect, the imaging system described in any one of aspects 35 - 38 includes an optical element, the optical element includes a reflective element, and the optical element is transmissive to at least 50% of the visible light incident on the optical element.

[0109] In the 40th aspect, the imaging system described in any one of aspects 35 - 39 is such that the reflective element includes a plurality of segments.

[0110] In the 41st aspect, the imaging system described in aspect 40 is such that the first segment within the plurality of segments has an optical property different from the optical property of the second segment within the plurality of segments.

[0111] In the 42nd aspect, the imaging system described in aspect 41 is such that the optical property of the first segment within the plurality of segments or the optical property of the second segment within the plurality of segments includes a reflection angle or a refractive power.

[0112] In the 43rd aspect, the imaging system described in any one of aspects 40-42, wherein the plurality of segments comprises at least two segments.

[0113] In the 44th aspect, the imaging system described in any one of aspects 40-43, wherein two of the plurality of segments are arranged horizontally.

[0114] In the 45th aspect, the imaging system described in any one of aspects 40-44, wherein two of the plurality of segments are arranged vertically.

[0115] In the 46th aspect, the imaging system described in any one of aspects 40-45, wherein some of the plurality of segments are arranged within a grid.

[0116] In the 47th aspect, the imager further comprises a focus control lens assembly, the imaging system described in any one of aspects 34-46.

[0117] In the 48th aspect, the focus control lens assembly comprises an offset lens, a tilt lens, or an offset-tilt lens, the imaging system described in aspect 47.

[0118] In the 49th aspect, an imaging system is disclosed. The imaging system comprises a reflective element that reflects light within a first wavelength range, the reflective element comprising a hot mirror, an off-axis holographic mirror (OAHM), or an off-axis volume diffractive optical element (OAVDOE), and an imager sensitive to light within the first wavelength range, the imager being configured to be oriented to capture at least the light reflected by the reflective element, the imaging system comprising a display.

[0119] In the 50th aspect, the first wavelength range comprises an infrared wavelength range, the imaging system described in aspect 49.

[0120] In the 51st aspect, the display is the imaging system described in aspect 49 or aspect 50, which is substantially transmissive to visible light.

[0121] In the 52nd aspect, the reflective element comprises a plurality of segments, and each segment within the plurality of segments has an optical property that is different from the optical property of at least one other segment within the plurality of segments. The imaging system is described in any one of aspects 49 - 51.

[0122] In the 53rd aspect, the optical property includes a reflection angle or a refractive power. The imaging system is described in aspect 52.

[0123] In the 54th aspect, the plurality of segments comprises 2, 3, 4, 5, 6, 7, 8, or 9 segments. The imaging system is described in aspect 52 or aspect 53.

[0124] In the 55th aspect, the imager further comprises a focus control lens assembly. The imaging system is described in any one of aspects 49 to 54.

[0125] In the 56th aspect, the focus control lens assembly comprises a shift lens, a tilt lens, or a shift - tilt lens. The imaging system is described in aspect 55.

[0126] In the 57th aspect, the imaging system further comprises a non - transitory data storage device configured to store an image acquired by the imager, and a hardware processor that communicates with the non - transitory data storage device. The hardware processor is programmed with executable instructions to analyze the image and perform one or more of eye tracking, biometric identification, multi - viewpoint reconstruction of the shape of the eye, estimation of the accommodation state of the eye, or discrimination of the retina, iris, or other imaging patterns of the eye. The imaging system is described in any one of aspects 21 to 56.

[0127] In the 58th aspect, a head-mounted display (HMD) is disclosed. The HMD includes the imaging system described in any one of aspects 21 to 57.

[0128] In the 59th aspect, the HMD includes a frame having a portion configured to be worn near the ear, and the imager is disposed near that portion, the HMD described in aspect 58.

[0129] In the 60th aspect, the imaging system is configured to image the wearer's first eye, the HMD includes a second imaging system described in any one of aspects 21 to 57, and the second imaging system is configured to image the wearer's second eye, the HMD described in aspect 58 or aspect 59.

[0130] In the 61st aspect, the HMD is an augmented reality device (ARD), the HMD described in any one of aspects 58 - 60.

[0131] In the 62nd aspect, a method for creating a virtual camera is disclosed. The method includes providing an imaging system in front of an object to be imaged and creating a virtual camera in front of the object, the imaging system being a reflective element that reflects light within a first wavelength range, the reflective element comprising an off-axis holographic mirror (OAHM) or an off-axis volumetric diffractive optical element (OAVDOE), the reflective element being oriented to reflect light propagating from the object when the imaging system is installed in front of the object, and an imager sensitive to light within a non-empty subset less than all of the first wavelength range, the imager being oriented to image an object by capturing light propagating from the object reflected by the reflective element, and the image of the object imaged by the imager and the image of the object imaged by a camera in front of the object being indistinguishable, the method including the step of.

[0132] In the 63rd aspect, the method according to aspect 62, wherein the first wavelength range comprises an infrared wavelength range.

[0133] In the 64th aspect, the imaging system comprises an optical element, the optical element comprises a reflective element, and the optical element is transmissive to at least 50% of the visible light incident on the optical element, according to any one of aspects 62 - 63.

[0134] In the 65th aspect, the reflective element comprises a plurality of segments, according to any one of aspects 62 - 64.

[0135] In the 66th aspect, the first segment within the plurality of segments has an optical property different from the optical property of the second segment within the plurality of segments, according to aspect 65.

[0136] In the 67th aspect, the optical property of the first segment within the plurality of segments or the optical property of the second segment within the plurality of segments comprises a reflection angle or a refractive power, according to aspect 66.

[0137] In the 68th aspect, the plurality of segments comprises at least two segments, according to any one of aspects 65 - 67.

[0138] In the 69th aspect, two of the plurality of segments are arranged horizontally, according to any one of aspects 65 - 68.

[0139] In the 70th aspect, two of the plurality of segments are arranged vertically, according to any one of aspects 65 - 69.

[0140] In the 71st aspect, some of the plurality of segments are arranged within a grid, according to any one of aspects 65 - 70.

[0141] In the 72nd aspect, the imager further comprises a perspective control lens assembly, according to any one of aspects 62 - 71.

[0142] In the 73rd aspect, the perspective control lens assembly is the method described in aspect 72, comprising an offset lens, a tilt lens, or an offset-tilt lens.

[0143] In the 74th aspect, a method of imaging an object using a virtual camera is disclosed. The method includes providing an imaging system in front of the object to be imaged and creating a virtual camera in front of the object. The imaging system is a reflective element that reflects light within a first wavelength range, and includes an off-axis holographic mirror (OAHM) or an off-axis volumetric diffractive optical element (OAVDOE). When the imaging system is installed in front of the object, the reflective element is oriented to reflect light propagating from the object. The method also includes an imager that is sensitive to light within a non-empty subset less than all of the first wavelength range, and is oriented to image the object by capturing the light propagating from the object reflected by the reflective element. The method further includes imaging the object using the virtual camera, including imaging the object by capturing the light propagating from the object reflected by the reflective element. The image of the object imaged by the imager and the image of the object imaged by the camera in front of the object are indistinguishable.

[0144] In the 75th aspect, the method described in aspect 74, wherein the first wavelength range comprises an infrared wavelength range.

[0145] In the 76th aspect, the imaging system comprises an optical element, the optical element comprises a reflective element, and the optical element is transmissive to at least 50% of the visible light incident on the optical element, according to any one of aspects 74-75.

[0146] In the 77th aspect, the reflective element comprises a plurality of segments, according to any one of aspects 74-76.

[0147] In the 78th aspect, the method according to aspect 77, wherein the first segment among the plurality of segments has an optical property different from the optical property of the second segment among the plurality of segments.

[0148] In the 79th aspect, the method according to aspect 78, wherein the optical property of the first segment among the plurality of segments or the optical property of the second segment among the plurality of segments includes a reflection angle or a refractive power.

[0149] In the 80th aspect, the method according to any one of aspects 77 - 79, wherein the plurality of segments includes at least two segments.

[0150] In the 81st aspect, the method according to any one of aspects 77 - 80, wherein two of the plurality of segments are arranged horizontally.

[0151] In the 82nd aspect, the method according to any one of aspects 77 - 81, wherein two of the plurality of segments are arranged vertically.

[0152] In the 83rd aspect, the method according to any one of aspects 77 - 82, wherein some of the plurality of segments are arranged in a grid.

[0153] In the 84th aspect, the method according to any one of aspects 74 - 83, wherein the imager further includes a focus control lens assembly.

[0154] In the 85th aspect, the method according to aspect 84, wherein the focus control lens assembly includes a shift lens, a tilt lens, or a shift - tilt lens.

[0155] In the 86th aspect, an imaging assembly is disclosed. The imaging assembly includes a see - through element (e.g., a display), a viewing camera installed to view the display, a lens associated with the camera, and a reflective element on the display that makes the display reflective to all or part of the wavelengths to which the display is sensitive.

[0156] In the 87th aspect, the reflective element comprises a hot mirror, an off-axis holographic mirror (OAHM) or an off-axis volume diffraction optical element (OAVDOE), and the assembly described in aspect 86.

[0157] In the 88th aspect, the assembly is the assembly described in any one of aspects 86 - 87, integrated into a wearable structure such as a pair of glasses or a helmet.

[0158] In the 89th aspect, the reflective element is segmented, and the assembly described in any one of aspects 86 - 88.

[0159] In the 90th aspect, the assembly is configured for the use of a segmented OAHM to select the best possible viewing angle for a particular task (e.g., gaze tracking or biometric identification), and the assembly described in aspect 89.

[0160] In the 91st aspect, the assembly is configured for the use of a plurality of segmented sub - images for stereoscopic or multi - viewpoint three - dimensional reconstruction of the eye shape, and the assembly described in any one of aspects 89 - 90.

[0161] In the 92nd aspect, the three - dimensional reconstruction of the eye shape is used to estimate the accommodation state of the eye, and the assembly described in aspect 91.

[0162] In the 93rd aspect, the estimation of the accommodation state of the eye includes comparing the apparent location and shape of the pupil and iris of the eye across multiple images of the same wearer of the assembly, and the assembly described in aspect 92.

[0163] In the 94th aspect, the estimation of the accommodation state of the eye is used to determine the dilation state of the lens, and the assembly described in any one of aspects 92 - 93.

[0164] In the 95th aspect, the assembly is the assembly described in any one of aspects 86 - 94, configured for the use of image segments as input to an information fusion algorithm.

[0165] In the 96th aspect, the information fusion algorithm is the assembly described in aspect 95, used to improve the apparent resolution of the eye or the quality of information extraction therefrom.

[0166] In the 97th aspect, the information fusion algorithm is the assembly described in any one of aspects 95 - 96, with image super - resolution techniques.

[0167] In the 98th aspect, the information fusion algorithm is the assembly described in any one of aspects 95 - 97, used to improve the image of the eye's iris.

[0168] In the 99th aspect, the information fusion algorithm is the assembly described in any one of aspects 95 - 98, with iris code extraction (e.g., John Daugman, et al. 2006) and subsequent fusion of the resulting iris codes to form a single estimate of the wearer's iris code.

[0169] In the 100th aspect, the assembly is the assembly described in any one of aspects 86 - 99, configured for the use of image segments to improve eye pose estimation or tracking.

[0170] In the 101st aspect, the three - dimensional reconstruction of the eye, iris, pupil, and cornea (or any subset thereof) is used directly in combination with image segments to improve the eye coverage rate in pose estimation, for the assembly described in aspect 100.

[0171] In the 102nd aspect, the reflective element includes a refractive power for adding or reducing beam divergence, with an OAVDOE, for the assembly described in any one of aspects 86 - 101.

[0172] In the 103rd aspect, the reflective element is an assembly as described in any one of the side surfaces 86 - 102, including any number of segments (e.g., 2, 3, 6, or 9 segments).

[0173] In the 104th aspect, the reflective element is an assembly as described in any one of the side surfaces 86 - 103, configured to reflect infrared light, and the vision camera is sensitive to infrared light.

[0174] In the 105th aspect, the reflective element is an assembly as described in side surface 104, comprising a hot mirror configured to reflect infrared light but otherwise be transmissive to visible light.

[0175] In the 106th aspect, the assembly as described in any one of the side surfaces 86 - 105 further comprises an offset lens (e.g., as in an oblique offset photograph) involving the normal to the vision camera parallel to the normal to the surface with the reflective element.

[0176] In the 107th aspect, a head - mounted display (HMD) is disclosed. The HMD comprises a pair of displays, each display comprising an imaging assembly as described in any one of the side surfaces 86 - 106, and one of the pair of assemblies is configured for each eye of the wearer. (Conclusion)

[0177] The processes, methods, and algorithms described in this specification and / or depicted in the accompanying figures are each embodied in code modules that are executed by one or more physical computing systems, hardware computer processors, application-specific circuits, and / or electronic hardware configured to execute specific and particular computer instructions, whereby they can be fully or partially automated. For example, a computing system can include a general-purpose computer (e.g., a server) or a dedicated computer, a dedicated circuit, etc., programmed with specific computer instructions. The code modules can be installed in a dynamic link library that is compiled and linked into an executable program, or can be written in a programming language that is interpreted. In some implementations, certain operations and methods can be implemented by circuits specific to a given function.

[0178] Furthermore, the functional implementations of the present disclosure are sufficiently mathematical, computer, or technically complex that a special-purpose hardware or one or more physical computing devices (utilizing appropriate specialized executable instructions) may be required to implement the functionality, e.g., due to the amount or complexity of the calculations involved, or to provide the results substantially in real time. For example, a video can include many frames, each frame can have millions of pixels, and specifically programmed computer hardware is required to process the video data to provide the desired image processing tasks or applications in a commercially reasonable amount of time.

[0179] A code module or any type of data can be stored on any type of non-transitory computer-readable medium such as a physical computer storage device including a hard drive, solid state memory, random access memory (RAM), read only memory (ROM), optical disk, volatile or non-volatile storage device, combinations of the same, and / or equivalents. The methods and modules (or data) can also be transmitted as data signals generated on various computer-readable transmission media including wireless-based and wired / cabled-based media (e.g., as part of a carrier wave or other analog or digital propagated signal), and can take various forms (e.g., as part of a single or multiplexed analog signal or as multiple discrete digital packets or frames). The results of the disclosed process or process steps can be persistently or otherwise stored within any type of non-transitory tangible computer storage device or communicated via a computer-readable transmission medium.

[0180] Any process, block, state, step, or functionality in the flow diagrams described in and / or depicted in the accompanying figures should be understood as potentially representing a code module, segment, or part of code that includes one or more executable instructions for implementing a specific function (e.g., logical or arithmetic) or step in the process. The various processes, blocks, states, steps, or functionality can be combined, rearranged, added, deleted, modified, or otherwise changed from the exemplary embodiments provided herein. In some embodiments, additional or different computing systems or code modules can implement some or all of the functionality described herein. The methods and processes described herein are also not limited to any particular sequence, and the associated blocks, steps, or states can be implemented in a suitable other sequence, e.g., sequentially, in parallel, or in some other manner. Tasks or events can be added to or removed from the disclosed exemplary embodiments. Further, the separation of the various system components in the implementations described herein is for illustrative purposes and should not be understood as requiring such separation in all implementations. It should be understood that the described program components, methods, and systems can generally be integrated together in a single computer product or packaged in multiple computer products. Many implementation variations are possible.

[0181] The present process, method, and system can be implemented in a network (or distributed) computing environment. The network environment can include an enterprise-wide computer network, an intranet, a local area network (LAN), a wide area network (WAN), a personal area network (PAN), a cloud computing network, a cloud source computing network, the Internet, and the World Wide Web. The network can be a wired or wireless network or any other type of communication network.

[0182] The systems and methods of the present disclosure each have several innovative aspects, none of which alone contribute to or are required for the desirable attributes disclosed herein. The various features and processes described above can be used independently of each other or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of the present disclosure. Various modifications of the implementations described in the present disclosure may be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other implementations without departing from the spirit or scope of the present disclosure. Accordingly, the claims are not intended to be limited to the implementations shown herein, but should be accorded the widest scope consistent with the present disclosure, the principles, and the novel features disclosed herein.

[0183] Certain features described herein in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, the various features described in the context of a single implementation can also be implemented separately in multiple implementations or in any suitable sub-combination. Further, features may be described above as acting in a certain combination and may further be claimed as such, but one or more features from the claimed combination can in some cases be deleted from the combination, and the claimed combination can be directed to a sub-combination or variation of a sub-combination. No single feature or group of features is necessary or essential to any embodiment.

[0184] In particular, conditional statements used herein such as "can", "could", "might", "may", "e.g.", and equivalents, generally convey that while one embodiment includes certain features, elements, and / or steps, other embodiments do not include them, unless specifically stated otherwise or understood otherwise within the context in which they are used. Thus, such conditional statements are not generally intended to imply that features, elements, and / or steps are required in any way for one or more embodiments, or that one or more embodiments necessarily include logic for determining whether these features, elements, and / or steps should be included or implemented in any particular embodiment, regardless of the author's input or prompting. The terms "comprising", "including", "having", and equivalents are synonyms and are used inclusively in a non-limiting manner, without excluding additional elements, features, acts, operations, etc. Also, the term "or" is used in its inclusive sense (and not in its exclusive sense), and thus, for example, when used to connect a list of elements, the term "or" means one, some, or all of the elements in the list. Additionally, the articles "a", "an", and "the" as used in this application and the appended claims should be construed to mean "one or more than one" or "at least one" unless otherwise defined.

[0185] As used herein, the phrase referring to a list of items "at least one of" refers to any combination of those items, including a single element. As an example, "at least one of A, B, or C" is intended to cover A, B, C, A and B, A and C, B and C, and A, B, and C. Connective phrases such as "at least one of X, Y, and Z" are generally understood in a context such that, unless otherwise specifically stated, the items, terms, etc. can be at least one of X, Y, or Z, and are used to convey that. Thus, such connective phrases are generally not intended to suggest that an embodiment requires the presence of at least one of X, at least one of Y, and at least one of Z, respectively.

[0186] Similarly, operations may be depicted in the drawings in a particular order, but it should be recognized that such operations are performed in the particular order shown, or in a sequential order, or that all of the illustrated operations need not be performed, in order to achieve a desired result. Further, the drawings may schematically depict one or more exemplary processes in the form of flowcharts. However, other operations not depicted may also be incorporated into the exemplary methods and processes schematically illustrated. For example, one or more additional operations may be performed before, after, simultaneously with, or between any of the illustrated operations. Additionally, operations may be rearranged or re-ordered in other implementations. In some situations, multitasking and parallel processing may be advantageous. Further, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the program components and systems described generally may be integrated together in a single software product or packaged into multiple software products. Additionally, other implementations are within the scope of the following claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve a desired result.

Claims

1. An imaging system, comprising: an optical element positioned in front of the user's eye; a reflective element that reflects light within a first wavelength range; a forward-facing imager sensitive to a subset of light in the first wavelength range, the forward-facing imager oriented to capture the subset of light in the first wavelength range reflected by the reflective element, the forward-facing imager being on the same side of the optical element as a user's eye; a virtual rear-facing imager formed by the infrared light propagating from the eye of the user through the optical element corresponding to the infrared light reflected by the reflective element and captured by the forward-facing imager; Equipped with an imaging system, wherein the virtual rear-facing imager and the front-facing imager are positioned on opposite sides of the optical element, and wherein a depth of field varies dynamically based on pupil size and / or orientation of the user's eyes.

2. The imaging system of claim 1, wherein the reflective element comprises a hot mirror, a holographic optical element (HOE), an off-axis holographic mirror (OAHM), or an off-axis volumetric diffractive optical element (OAVDOE).

3. The imaging system of claim 1, wherein the first wavelength range comprises an infrared wavelength range.

4. The imaging system of claim 1, wherein the optical element comprises the reflective element, and the optical element is transparent to at least 50% of visible light incident on the optical element.

5. The imaging system of claim 1, wherein the reflective element comprises a plurality of segments.

6. The imaging system of claim 5, wherein a first segment in the plurality of segments has optical properties that are different from the optical properties of a second segment in the plurality of segments.

7. The imaging system of claim 6, wherein the optical properties of the first segment within the plurality of segments or the optical properties of the second segment within the plurality of segments comprise a reflection angle or a refractive power.

8. An imaging system as described in claim 5, wherein two of the plurality of segments are arranged horizontally.

9. An imaging system as described in claim 5, wherein two of the plurality of segments are arranged vertically.

10. The imaging system of claim 1, wherein the forward-facing imager further comprises a perspective control lens assembly comprising a shifting lens, a tilting lens, or a shifting-tilting lens.

11. The imaging system of claim 1, wherein the reflective element is transparent to a second wavelength range different from the first wavelength range.

12. The imaging system of claim 1, wherein the optical element comprises the reflective element, and the optical element is non-transparent to visible light incident on the optical element.

13. The imaging system of claim 1, wherein the depth of field for the forward-facing imager is the same as the depth of field of the virtual rear-facing imager based on the virtual rear-facing imager and the forward-facing imager being equidistant from the optical element.

14. The imaging system of claim 1, wherein the depth of field for the virtual rear-facing imager and the forward-facing imager is the same.

15. The imaging system of claim 1, wherein the depth of field varies inversely with the user's pupil size.

16. The imaging system of claim 1, wherein the number of spaced depth planes used to present different images to the user decreases with decreasing pupil size.

Images