Eyepiece imaging assembly for head-mounted display

Abstract

To provide a suitable eyepiece imaging assembly for a head-mounted display.SOLUTION: A head-mounted display may include a frame, an eyepiece, an image injection device, a sensor array, a reflector, and an off-axis optical element. The frame may be configured to be supported on a user's head. The eyepiece may be configured to be coupled to the frame and positioned in front of the user's eyes. The eyepiece may include multiple layers. The image injection device may be configured to provide image content to the eyepiece for viewing by the user. The sensor array may be integrated within or on the eyepiece. The reflector may be disposed within or on the eyepiece and configured to reflect light received from an object for imaging by the sensor array. The off-axis optical element may be disposed within or on the eyepiece. The off-axis optical element may be configured to receive light reflected from the reflector and direct at least a portion of the light toward the sensor array.SELECTED DRAWING: Figure 11A

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Application No. 63 / 130,274 (Attorney Docket No. MLEAP.317PR), entitled "EYEPIECE IMAGING ASSEMBLIES FOR A HEAD MOUNTED DISPLAY," filed December 23, 2020. This application is related to U.S. patent application Ser. No. 15 / 271,802 (Attorney Docket No. MLEAP.011A2), filed September 21, 2016, entitled "EYE IMAGING WITH AN OFF-AXIS IMAGER," U.S. patent application Ser. No. 15 / 925,505 (Attorney Docket No. MLEAP.099A3), filed March 19, 2018, entitled "EYE-IMAGING APPARATUS USING DIFFRACTIVE OPTICAL ELEMENTS," and International Application No. PCT / US2020 / 044107 (Attorney Docket No. MLEAP.247WO), filed July 29, 2020, entitled "ANGULARLY SEGMENTED HOT MIRROR FOR EYE TRACKING." The entirety of each application referenced in this paragraph is incorporated herein by reference.

[0002] The present disclosure relates to virtual reality and augmented reality imaging and visualization systems, and in particular to imaging assemblies integrated into or on the eyepieces of head-mounted displays. [Background technology]

[0003] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or portions thereof are presented to a user as if they were real or perceived as real. Virtual reality, or "VR," scenarios typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual input, while augmented reality, or "AR," scenarios typically involve the presentation of digital or virtual image information as an augmentation to the visualization of the real world around the user. Mixed reality, or "MR," scenarios are a type of AR scenario, typically involving augmented objects that are integrated with and responsive to the natural world. For example, in MR scenarios, AR image content may be occluded by or otherwise perceived as interacting with objects in the real world.

[0004] Referring to FIG. 1 , an augmented reality scene 10 is depicted in which a user of the AR technology views a real-world park-like setting 20 featuring people, trees, buildings, and a concrete platform 30 in the background. In addition to these items, the user of the AR technology also perceives that he or she is "seeing" "virtual content," such as a robotic figure 40 standing on the real-world platform 30 and a cartoonish avatar character 50 that appears to be an anthropomorphic bumblebee, even though these elements 40, 50 do not exist in the real world. Due to the complexity of the human visual perception system, it is a challenge to create AR technology that facilitates a comfortable, natural-feeling, and rich presentation of virtual image elements, among other virtual or real-world image elements.

[0005] The systems and methods disclosed herein address various challenges associated with AR and VR technologies. Summary of the Invention [Means for solving the problem]

[0006] Various implementations of methods and apparatus within the scope of the appended claims each have several aspects, no one of which is solely responsible for the desirable attributes described herein. Without limiting the scope of the appended claims, some prominent features will be described herein.

[0007] One aspect of the present disclosure provides for imaging an object, such as an eye, with an imaging assembly that does not view the object directly from a position directly in front of the object. Rather, in various designs discussed herein, light from the object is reflected to a sensor from an at least partially reflective optical element or reflector. This reflector may be partially reflective and partially transmissive or transparent, allowing a user to see through the partially reflective / transmissive optical element. Thus, various optical devices according to various implementations described herein include such reflectors that direct light from the object to the imaging assembly, thereby capturing an image of the object as if the imaging assembly were directly in front of the object. Various implementations disclosed herein are configured, for example, to direct light from the eye to the imaging assembly and capture an image of the eye as if the imaging camera or sensor were directly in front of the eye, but instead the sensor is offset from the line of sight (e.g., a line of sight looking directly ahead), the field of view of the user's eye (e.g., the central field of view), or any combination thereof. Advantages of some implementations include imaging an object as if viewed from a position directly in front of the object, but without interfering with and / or obstructing the user's view of the object directly in front of the viewer (e.g., by temporarily positioning the imaging assembly relative to the eye). In various examples, the imaging assembly is integrated into or on the eyepiece of a head-mounted display. Advantages of some such designs include reduced camera form factor and improved alignment between optical components in or on the eyepiece. The imaging assembly can be used to image objects within the user's eye or the environment.

[0008] Details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Neither this summary nor the following detailed description is intended to define or limit the scope of the inventive subject matter. The present specification also provides, for example, the following: (Item 1) A head-mounted display, a frame configured to be supported on a user's head; an eyepiece coupled to the frame and configured to be positioned in front of the user's eye, the eyepiece comprising a plurality of layers; an image injection device configured to provide image content to the eyepiece for viewing by the user; a sensor array integrated in or on the eyepiece; a reflector disposed in or on the eyepiece configured to reflect light received from an object for imaging by the sensor array; a transmissive diffractive or transmissive refractive optical element disposed within or on the eyepiece and configured to receive light reflected from the reflector and diffract or refract at least a portion of the light toward the sensor array. (Item 2) 10. A head-mounted display as described in any one of the preceding items, wherein the sensor array comprises a plurality of detector pixels formed on at least one of the layers. (Item 3) 10. The head-mounted display of claim 1, wherein the sensor array includes wafer-scale optics. (Item 4) A head-mounted display as described in any one of the preceding items, wherein the diffractive or refractive optical element comprises at least one lens aligned with the sensor array such that light from the reflector passes through the at least one lens to the sensor array and forms an image on the sensor array. (Item 5) Item 5. A head-mounted display as described in item 4, wherein the sensor array is arranged within or on a first layer of the plurality of layers of the eyepiece, and the at least one lens is arranged within or on a second, different layer of the plurality of layers of the eyepiece. (Item 6) Item 5. A head-mounted display as described in item 4, wherein the sensor array is arranged in or on a first side of a layer of the plurality of layers of the eyepiece, and the at least one lens is arranged in or on a second, opposite side of the layer. (Item 7) 7. The head-mounted display according to any one of items 4 to 6, wherein at least one lens includes a wafer-scale optical system. (Item 8) 10. The head mounted display of claim 1, wherein the reflector comprises a hot mirror. (Item 9) 10. A head-mounted display as described in any one of the preceding items, wherein the reflector is configured to reflect light in a first range of infrared (IR) or near-infrared wavelengths while transmitting light in a second range of visible wavelengths. (Item 10) 10. A head-mounted display according to any one of the preceding claims, wherein the reflector is formed on at least one of the plurality of layers. (Item 11) 10. A head-mounted display according to any one of the preceding claims, wherein the reflector comprises one of the plurality of layers. (Item 12) 10. A head mounted display according to any one of the preceding items, wherein the diffractive or refractive optical element is arranged in an optical path between the reflector and the sensor array. (Item 13) 10. A head-mounted display as described in any one of the preceding items, wherein the reflector is arranged on a first layer of the plurality of layers, the diffractive or refractive optical element is arranged on a second layer of the plurality of layers, and the sensor array is arranged on a third layer of the plurality of layers, and the second layer is arranged between the first layer and the third layer. (Item 14) 10. A head mounted display according to any one of the preceding claims, wherein the diffractive or refractive optical element comprises an off-axis optical element. (Item 15) 10. A head mounted display according to any one of the preceding items, wherein the diffractive or refractive optical element has optical power. (Item 16) 10. A head mounted display according to any one of the preceding claims, wherein the diffractive or refractive optical element comprises an off-axis lens. (Item 17) A head-mounted display as described in any one of the preceding items, wherein at least a portion of the diffractive or refractive optical element is engraved on one of the layers of the eyepiece portion. (Item 18) A head-mounted display as described in any one of the preceding items, wherein the sensor array is a forward-facing camera configured to image at least a portion of the object based at least in part on light received from a reflector disposed in front of the sensor array, and the at least a portion of the object is disposed behind the sensor array and includes the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. (Item 19) 10. A head-mounted display according to any one of the preceding claims, wherein the sensor array is further configured to image objects in an environment in front of the eyepiece. (Item 20) 20. The head-mounted display of item 19, wherein the sensor array is a rear-facing camera configured to image at least a portion of the object based at least in part on light received from the reflector behind the sensor array, and the at least a portion of the object is positioned in front of the sensor array in an environment in front of the user. [Brief explanation of the drawings]

[0009] [Figure 1] 1 shows a user's view of an augmented reality (AR) device. [Figure 2] 1 illustrates an example of a wearable display system. [Figure 3] 1 illustrates a conventional display system that simulates a three-dimensional image for a user. [Figure 4] 1 illustrates aspects of an approach for simulating a three-dimensional image using multiple depth planes. [Figure 5A] 1 shows the relationship between the radius of curvature and the radius of focus. [Figure 5B] 1 shows the relationship between the radius of curvature and the radius of focus. [Figure 5C] 1 shows the relationship between the radius of curvature and the radius of focus. [Figure 6] 1 illustrates an example of a waveguide stack for outputting image information to a user. [Figure 7] 1 shows an example of an exit beam output by a waveguide. [Figure 8] 1 illustrates an example of a stacked waveguide assembly in which each depth plane contains an image formed using multiple different component colors. [Figure 9A]1 shows a cross-sectional side view of an example of a set of stacked waveguides, each including an incoupling optical element. [Figure 9B] 9B shows a perspective view of an example of the multiple stacked waveguides of FIG. 9A. [Figure 9C] FIG. 9C is a top plan view of an example of the multiple stacked waveguides of FIGS. 9A and 9B. [Figure 10A] 1 illustrates a schematic diagram of an exemplary imaging system with partially transmissive and partially reflective optical elements and an eye-tracking camera assembly positioned so as not to obstruct the central field of view or straight ahead gaze of a user's eye. [Figure 10B] 1 illustrates a schematic diagram of an exemplary imaging system with partially transmissive and partially reflective optical elements and an eye-tracking camera assembly positioned so as not to obstruct the central field of view or straight ahead gaze of a user's eye. [Figure 11A] 1A and 1B illustrate schematic diagrams of exemplary imaging systems integrated into or on the eyepiece and configured to image objects in the eye or the environment, respectively, arranged so as not to obstruct the central field of view of the user's eye or a straight-ahead line of sight. [Figure 11B] 1A and 1B illustrate schematic diagrams of exemplary imaging systems integrated into or on the eyepiece and configured to image objects in the eye or the environment, respectively, arranged so as not to obstruct the central field of view of the user's eye or a straight-ahead line of sight. [Figure 12A] 10A and 10B show schematic diagrams of an example sensor array and optical lens integrated in or on the eyepiece, with lenses on the same layer but on either side of the layer, or with the lens and sensor array on a different layer than the sensor array. [Figure 12B] 10A and 10B show schematic diagrams of an example sensor array and optical lens integrated in or on the eyepiece, with lenses on the same layer but on either side of the layer, or with the lens and sensor array on a different layer than the sensor array. [Figure 13] 10 illustrates a schematic diagram of another exemplary imaging system with a sensor array having wafer-scale optics integrated into the eyepiece layer. [Figure 14]FIG. 1 is a process flow diagram of an example method for imaging an object using a camera positioned so as not to obstruct the central field of view or straight ahead gaze of a user's eye.

[0010] Reference numbers may be repeated throughout the drawings to indicate correspondence between referenced elements. The drawings are provided to illustrate exemplary embodiments described herein and are not intended to limit the scope of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] overview Head-mounted displays (HMDs) often use information about the state of a wearer's eyes for various purposes. For example, this information can be used to estimate the wearer's gaze direction, for biometric identification, for vision studies, and to assess the wearer's physiological state. However, imaging the eyes can be challenging. The distance between the HMD and the wearer's eyes is short. Furthermore, gaze tracking requires a large field of view (FOV), and biometric identification requires a relatively large number of pixels on the iris target. These requirements are highly conflicting when an imaging system attempts to achieve both of these objectives. Furthermore, occlusions by eyelids and eyelashes can further complicate both issues. Some current implementations for tracking eye movements use cameras mounted on the HMD and pointed directly at the eyes to capture direct images of the eyes. However, to achieve the desired FOV and pixel count, the cameras must be mounted within the wearer's FOV, which tends to obscure or interfere with the wearer's ability to view the surrounding world. Other implementations image the eye directly but move the camera away so as not to obstruct the wearer's view, resulting in the eye being imaged from a high angle, distorting the image and reducing the field of view available for imaging the eye. Similarly, using a camera pointed directly at the object to image an object in the environment, such as in front of the wearer's eye, may obstruct the wearer's view if aligned with the wearer's forward line of sight or central field of view. Furthermore, moving the camera away and / or pointing the camera at a large angle relative to the object may distort the image of the object and / or provide a different perspective than the eye.

[0012] Implementations of imaging systems described herein address some or all of these issues. Various implementations described herein provide, for example, devices and systems capable of imaging an object (e.g., at least a portion of the eye, a portion of the tissue surrounding the eye, or an object in the environment in front of the wearer's eye) while allowing the wearer to see the surrounding world. For example, an imaging system may include a sensor array integrated in or on an eyepiece positioned in front of the user's eye. The eyepiece may include one or more optical elements (e.g., a partially reflective and partially transmissive reflector, and a transmissive diffractive or refractive optical element that redirects light to the sensor array) configured to direct light from the object to the sensor array. The sensor array may receive at least a portion of the light to capture an image of the object from a distanced position directly in front of the object, such as the eye, as if it were in a direct line of sight position.

[0013] In some implementations, the imaging system described herein may be part of the display optics of an HMD (or a pair of glasses or other eyewear lenses). One or more reflective optical elements may be selected to reflect a first range of wavelengths while allowing unimpeded propagation of a second range of wavelengths (e.g., a range of wavelengths different from the first range) through the eyepiece. The first range of wavelengths may be, for example, infrared (IR) light, and the second range of wavelengths may be visible light. For example, the eyepiece may include a reflective optical element (e.g., a reflector) that reflects infrared light while transmitting visible light. In effect, the imaging system behaves as if there is a virtual camera assembly facing away from the object (e.g., the wearer's eye). In this way, the virtual camera assembly may form an image using infrared light reflected from the object (e.g., the wearer's eye) while allowing visible light from the surrounding world to pass through and be perceived by the wearer.

[0014] Without adhering to any particular scientific theory, the embodiments described herein may include several non-limiting advantages. Some embodiments may increase the physical distance between the camera assembly and the eye, which may facilitate positioning the camera assembly outside the wearer's field of view or central field of view, thereby allowing for capturing images of the eye equivalent to looking directly at the eye, without obstructing the wearer's field of view, such as the field of view directly in front of the wearer. Some of the embodiments described herein may also be configured to allow for eye tracking using a larger field of view than conventional systems, thereby enabling eye tracking over a wide range of positions. Some implementations described herein may also be configured to image objects in the environment in front of the wearer's eye without obstructing the wearer's field of view. For example, using IR imaging may facilitate imaging the eye without interfering with the wearer's ability to see through the eyepiece or observe the environment.

[0015] Reference will now be made to the drawings, wherein like reference numerals refer to like parts throughout.

[0016] Exemplary HMD Devices FIG. 2 illustrates an example of a wearable display system 60. The display system 60 includes, for example, a display 70 with an eyepiece and various mechanical and electronic modules and systems that support the functionality of the display 70. The display 70 may be coupled to a frame 80, which is wearable by a user or viewer 90 of the display system and configured to position the display 70 in front of the user's 90 eyes. In some embodiments, the display 70 may be considered eyewear. In some embodiments, a speaker 100 is coupled to the frame 80 and configured to be positioned adjacent to the user's 90 ear canal (in some embodiments, a separate speaker, not shown, is positioned adjacent the user's other ear canal to provide stereo / shapeable sound control). In some embodiments, the display system may also include one or more microphones 110 or other devices for detecting sound. In some embodiments, the microphone may be configured to allow a user to provide input or commands to the system 60 (e.g., voice menu commands, natural language questions, etc.) and / or enable voice communication with others (e.g., with other users of similar display systems). The microphone may also be configured as an ambient sensor for collecting audio data (e.g., sounds from the user and / or the environment). In some embodiments, the display system may also include an ambient sensor 120a, which may be separate from the frame 80 and attached to the body of the user 90 (e.g., the head, torso, limbs, etc. of the user 90). The ambient sensor 120a, in some embodiments, may be configured to acquire data characterizing a physiological state of the user 90. For example, the sensor 120a may be an electrode.

[0017] 2 , the display 70 is operably coupled by a communication link 130, such as a wired lead or a wireless connection, to a local data processing module 140, which may be mounted in a variety of configurations, such as fixedly attached to the frame 80, fixedly attached to a helmet or hat worn by the user, built into headphones, or otherwise removably attached to the user 90 (e.g., in a backpack-style configuration, a belt-connected-style configuration). Similarly, the sensor 120 a may be operably coupled to the local processor and data module 140 by a communication link 120 b, e.g., a wired lead or a wireless connection. The local processing and data module 140 may include a hardware processor and digital memory, such as non-volatile memory (e.g., flash memory or a hard disk drive), both of which may be used to assist in processing, caching, and storing data. The data may include a) data captured from sensors (e.g., which may be operatively coupled to frame 80 or otherwise attached to user 90), such as image capture devices (e.g., cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, gyros, and / or other sensors disclosed herein, and / or b) data obtained and / or processed using remote processing module 150 and / or remote data repository 160 (including data related to virtual content), possibly for passage to display 70 after such processing or retrieval. Local processing and data module 140 may be operatively coupled to remote processing module 150 and remote data repository 160 by communications links 170, 180, such as via wired or wireless communication links, such that these remote modules 150, 160 are operatively coupled to each other and available as resources to local processing and data module 140. In some embodiments, the local processing and data module 140 may include one or more of an image capture device, a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a wireless device, and / or a gyro.In some other embodiments, one or more of these sensors may be mounted on the frame 80 or may be stand-alone structures that communicate with the local processing and data module 140 via a wired or wireless communication path.

[0018] 2 , in some embodiments, remote processing module 150 may comprise one or more processors configured to analyze and process data and / or image information. In some embodiments, remote data repository 160 comprises a digital data storage facility, which may be available over the Internet or other networking configuration in a “cloud” resource configuration. In some embodiments, remote data repository 160 may include one or more remote servers that provide information, such as information for generating augmented reality content, to local processing and data module 140 and / or remote processing module 150. In some embodiments, all data is stored and all computations are performed in the local processing and data module, allowing for fully autonomous use from the remote module.

[0019] The perception of an image as "three-dimensional," or "3D," can be achieved by providing a slightly different image representation to each of the viewer's eyes. FIG. 3 illustrates a conventional display system for simulating a three-dimensional image for a user. Two distinct images 190, 200, one for each eye 210, 220, are output to the user. The images 190, 200 are spaced apart from the eyes 210, 220 by a distance 230 along the optical axis, or Z-axis, parallel to the viewer's line of sight. The images 190, 200 are flat, and the eyes 210, 220 may focus on them by assuming a single viewing position. Such a 3D display system relies on the human visual system to combine the images 190, 200 to perceive depth and / or scale of the combined image.

[0020] However, it will be appreciated that the human visual system is more complex, and providing a realistic perception of depth is even more challenging. For example, many viewers of conventional "3D" display systems find such systems uncomfortable or may not perceive any sense of depth at all. Without being limited by theory, a viewer may perceive an object as "three-dimensional" due to a combination of convergence and accommodation. The movement of the two eyes relative to one another (e.g., the movement of the pupils toward or away from one another, resulting in the eyes' rotation to converge and fixate on an object) is closely related to the focusing of the eye's lens and pupil (i.e., "accommodation"). Under normal conditions, changing the focus of the eye's lens, i.e., the eye's accommodation to change focus from one object at a different distance to another, along with pupil dilation or constriction, automatically changes convergence to the same distance, under a relationship known as the "accommodative convergence reflex." Similarly, changes in convergence trigger matching accommodation changes in the lens shape and pupil size under normal conditions. As described herein, many stereoscopic or "3D" display systems display a scene using slightly different presentations (i.e., slightly different images) to each eye, thereby enabling the human visual system to perceive a three-dimensional perspective. However, such systems, among other things, simply provide different presentations of the scene, with the eyes viewing all of the image information in a single state of accommodation, countering the "accommodative convergence reflex" and making them uncomfortable for many users. A display system with a better match between accommodation and convergence may produce a more realistic and comfortable simulation of three-dimensional images, contributing to longer wear duration and, therefore, compliance with diagnostic and treatment protocols.

[0021] FIG. 4 illustrates aspects of an approach for simulating a three-dimensional image using multiple depth planes. Referring to FIG. 4 , objects at various distances from the eyes 210, 220 on the z-axis are accommodated by the eyes 210, 220 to focus on these objects. The eyes 210, 220 adopt specific accommodation states that focus on objects at different distances along the z-axis. As a result, a specific accommodation state may be said to be associated with a particular one of the depth planes 240, which has an associated focal length such that an object or portion of an object within the particular depth plane is in focus when the eye is in an accommodation state relative to that depth plane. In some embodiments, a three-dimensional image may be simulated by providing different image presentations to each of the eyes 210, 220, and further by providing different image presentations corresponding to each of the depth planes. While shown separately for clarity of illustration, the fields of view of the eyes 210, 220 may overlap, for example, as the distance along the z-axis increases. Furthermore, although shown flat for ease of illustration, it will be understood that the contours of the depth plane may be curved in physical space so that any feature in the depth plane is in focus with the eye in a particular state of accommodation.

[0022] The distance between an object and the eye 210 or 220 can also change the amount of divergence of light from the object as viewed by the eye. Figures 5A-5C show the relationship between distance and divergence of light rays. The distances between the object and the eye 210 are represented in descending order of distance: R1, R2, and R3. As shown in Figures 5A-5C, the shorter the distance to the object, the more divergent the light rays become. The longer the distance, the more collimated the light rays become. In other words, the bright field generated at a point (an object or a portion of an object) has a spherical wavefront curvature, which is a function of how far the point is from the user's eye. The curvature increases as the distance between the object and the eye 210 decreases. As a result, the degree of divergence of light rays varies at different depth planes, with the degree of divergence increasing as the distance between the depth plane and the viewer's eye 210 decreases. 5A-5C, for clarity of illustration, only a single eye 210 is shown, however, it will be understood that discussion regarding eye 210 may apply to both eyes 210 and 220 of the viewer.

[0023] Without being limited by theory, it is believed that the human eye can typically interpret a finite number of depth planes to provide depth perception. As a result, a highly believable simulation of perceived depth can be achieved by providing the eye with different image presentations corresponding to each of these limited number of depth planes. The different presentations may be focused differently by the viewer's eyes, thereby helping to provide depth cues to the user based on the eye accommodations required to focus on different image features for a scene located on different depth planes and / or based on observing that different image features on different depth planes are out of focus.

[0024] Example of a waveguide stack assembly FIG. 6 illustrates an example of a waveguide stack for outputting image information to a user. Display system 250 includes a stack of waveguides, or stacked waveguide assembly 260, that can be utilized to provide a three-dimensional perception to the eye / brain using multiple waveguides 270, 280, 290, 300, 310. In some embodiments, display system 250 is system 60 of FIG. 2, and FIG. 6 illustrates several portions of system 60 in further detail. For example, waveguide assembly 260 can be part of display 70 of FIG. 2 (e.g., can form at least a portion of the eyepiece). It will be understood that display system 250 can be considered a bright field display in some embodiments.

[0025] Continuing with reference to FIG. 6, the waveguide assembly 260 may also include multiple features 320, 330, 340, 350 between the waveguides. In some embodiments, the features 320, 330, 340, 350 may be one or more lenses. The waveguides 270, 280, 290, 300, 310 and / or multiple lenses 320, 330, 340, 350 may be configured to transmit image information to the eye with various levels of wavefront curvature or ray divergence. Each waveguide level may be associated with a particular depth plane and configured to output image information corresponding to that depth plane. The image injection devices 360, 370, 380, 390, 400 may act as light sources for the waveguides and may be utilized to inject image information into the waveguides 270, 280, 290, 300, 310, and each may be configured to distribute incoming light across wavelengths as described herein for output toward the eye 210. Light exits output faces 410, 420, 430, 440, 450 of the image injection devices 360, 370, 380, 390, 400 and is injected into corresponding input faces 460, 470, 480, 490, 500 of the waveguides 270, 280, 290, 300, 310. In some embodiments, each input face 460, 470, 480, 490, 500 may be an edge of the corresponding waveguide or may be part of a major face of the corresponding waveguide (i.e., one of the waveguide faces directly facing the world 510 or the viewer's eye 210). In some embodiments, a single light beam (e.g., a collimated beam) may be injected into each waveguide to output an entire field of view of the cloned collimated beam directed toward the eye 210 at a particular angle (and amount of divergence) corresponding to the depth plane associated with the particular waveguide. In some embodiments, one of the image injection devices 360, 370, 380, 390, 400 may be associated with and inject light into multiple (e.g., three) waveguides 270, 280, 290, 300, 310.

[0026] In some embodiments, image injection devices 360, 370, 380, 390, 400 are each separate displays that generate image information for injection into each of the corresponding waveguides 270, 280, 290, 300, 310. In other embodiments, image injection devices 360, 370, 380, 390, 400 are outputs of a single multiplexed display that may, for example, pipe image information to each of image injection devices 360, 370, 380, 390, 400 via one or more optical conduits (such as fiber optic cables). It will be understood that the image information provided by image injection devices 360, 370, 380, 390, 400 may include light of different wavelengths or colors (e.g., different component colors, as discussed herein).

[0027] In some embodiments, the light injected into the waveguides 270, 280, 290, 300, 310 is provided by a light projector system 520, which includes a light module 530, which may include a light emitter such as a light emitting diode (LED). The light from the light module 530 is directed toward a modulator 540, such as a spatial light modulator, where it may be modified by the light modulator 540 via a beam splitter 550. The light modulator 540 may be configured to change the perceived intensity of the light injected into the waveguides 270, 280, 290, 300, 310. Examples of spatial light modulators include liquid crystal displays (LCDs), including liquid crystal on silicon (LCOS).

[0028] In some embodiments, the display system 250 may be a scanning fiber display comprising one or more scanning fibers configured to project light in various patterns (e.g., raster scan, spiral scan, Lissajous pattern, etc.) into one or more waveguides 270, 280, 290, 300, 310 and ultimately into the viewer's eye 210. In some embodiments, the illustrated image injection devices 360, 370, 380, 390, 400 may generally represent a single scanning fiber or a bundle of scanning fibers configured to inject light into one or more waveguides 270, 280, 290, 300, 310. In other embodiments, the illustrated image injection devices 360, 370, 380, 390, 400 may generally represent multiple scanning fibers or a bundle of scanning fibers, each configured to inject light into an associated one of the waveguides 270, 280, 290, 300, 310. It will be appreciated that one or more optical fibers may be configured to transmit light from optical module 530 to one or more of waveguides 270, 280, 290, 300, and 310. It will be appreciated that one or more intervening optical structures may be provided between this or these scanning fibers and one or more of waveguides 270, 280, 290, 300, 310, for example, to redirect light exiting the scanning fibers into one or more of waveguides 270, 280, 290, 300, 310.

[0029] Controller 560 controls the operation of one or more of stacked waveguide assembly 260, including the operation of image injection devices 360, 370, 380, 390, 400, light source 530, and light modulator 540. In some embodiments, controller 560 is part of local data processing module 140. Controller 560 includes programming (e.g., instructions in a non-transitory medium) that regulates the timing and provision of images to waveguides 270, 280, 290, 300, 310, for example, according to any of the various approaches disclosed herein. In some embodiments, the controller may be a single, integrated device or a distributed system connected by a wired or wireless communication channel. Controller 560 may, in some embodiments, be part of processing module 140 or 150 (FIG. 2).

[0030] Continuing with reference to FIG. 6 , the waveguides 270, 280, 290, 300, and 310 can be configured to propagate light within each respective waveguide via TIR. The waveguides 270, 280, 290, 300, and 310 can each be planar or have other shapes (e.g., curved) with major top and bottom surfaces and edges extending between the major surfaces. In the illustrated configuration, the waveguides 270, 280, 290, 300, and 310 can each include outcoupling optical elements 570, 580, 590, 600, and 610 configured to extract light from the waveguide by redirecting the light and propagating within the respective waveguide to output image information to the eye 210. The extracted light can also be referred to as outcoupled light, and the light of the outcoupling optical elements can also be referred to as light extraction optical elements. The extracted light beam may be output by the waveguide where light propagating within the waveguide encounters a light extraction optical element. The outcoupling optical element 570, 580, 590, 600, 610 may be, for example, a grating including diffractive optical features, as discussed further herein. While shown disposed on the bottom major surface of the waveguides 270, 280, 290, 300, 310 for ease of explanation and clarity of the drawings, in some embodiments, the outcoupling optical element 570, 580, 590, 600, 610 may be disposed on the top and / or bottom major surface and / or directly within the volume of the waveguides 270, 280, 290, 300, 310, as discussed further herein. In some embodiments, the outcoupling optical elements 570, 580, 590, 600, 610 may be formed in a layer of material attached to a transparent substrate to form the waveguides 270, 280, 290, 300, 310. In other embodiments, the waveguides 270, 280, 290, 300, 310 may be a monolithic piece of material, and the outcoupling optical elements 570, 580, 590, 600, 610 may be formed on a surface of and / or within the piece of material.

[0031] Continuing with reference to FIG. 6, as discussed herein, each waveguide 270, 280, 290, 300, 310 is configured to output light for forming an image corresponding to a particular depth plane. For example, the waveguide 270 closest to the eye may be configured to send collimated light (injected into such waveguide 270) to the eye 210. The collimated light may represent the optical infinity focal plane. The next waveguide 280 may be configured to send collimated light that passes through a first lens 350 (e.g., a negative lens) before reaching the eye 210. Such a first lens 350 may be configured to produce a slightly convex wavefront curvature, causing the eye / brain to interpret the light coming from the next waveguide 280 as coming from the first focal plane, approaching inward from optical infinity toward the eye 210. Similarly, the third up waveguide 290 may be configured to pass its output light through both the first lens 350 and the second lens 340 before reaching the eye 210, and the combined refractive powers of the first lens 350 and the second lens 340 may be configured to produce an additional incremental amount of wavefront curvature so that the eye / brain interprets the light coming from the third waveguide 290 as coming from a second focal plane that is closer further inward from optical infinity towards the person than the light from the next waveguide up 280.

[0032] The remaining waveguide layers 300, 310 and lenses 330, 320 are similarly configured, with the highest waveguide 310 in the stack sending its output through all lenses between it and the eye for aggregate optical power representing the focal plane closest to the person. When viewing / interpreting light coming from the world 510 on the other side of the stacked waveguide assembly 260, a contributing lens layer 620 can be placed on top of the stack to contribute to the aggregate power of the lens stacks 320, 330, 340, 350 below. This configuration allows the perception of as many focal planes as there are available waveguides / lenses. Both the outcoupling optical elements of the waveguides and the focusing aspects of the lenses may be static (i.e., not dynamic or electro-active). In some alternative embodiments, either or both of these may be dynamic using electro-active features.

[0033] In some embodiments, two or more of the waveguides 270, 280, 290, 300, 310 may have the same associated depth plane. For example, multiple waveguides 270, 280, 290, 300, 310 may be configured to output a set of images at the same depth plane, or multiple subsets of waveguides 270, 280, 290, 300, 310 may be configured to output a set of images at the same depth planes, one set for each depth plane. This may be advantageous for forming tiled images to provide an extended field of view at those depth planes.

[0034] Continuing with reference to FIG. 6 , the outcoupling optical elements 570, 580, 590, 600, 610 can be configured to both redirect light exiting each of these waveguides and output this light with an appropriate amount of divergence or collimation for the particular depth plane associated with the waveguide. As a result, waveguides with different associated depth planes may have differently configured outcoupling optical elements 570, 580, 590, 600, 610, which output light with different amounts of divergence depending on the associated depth plane. In some embodiments, the light-extraction optical elements 570, 580, 590, 600, 610 can be volume or surface features that can be configured to output light at a particular angle. For example, the light-extraction optical elements 570, 580, 590, 600, 610 can be volume holograms, surface holograms, and / or diffraction gratings. In some embodiments, features 320, 330, 340, 350 may not be lenses; rather, they may simply be spacers (eg, cladding layers and / or structures that form air gaps).

[0035] In some embodiments, the outcoupling optical elements 570, 580, 590, 600, 610 are diffractive features, or "diffractive optical elements" (also referred to herein as "DOEs"), that form a diffraction pattern. Preferably, the DOEs have sufficiently low diffraction efficiency that each intersection of the DOEs deflects only a portion of the light beam toward the eye 210, while the remainder continues traveling through the TIR. Thus, the light carrying the image information is split into multiple associated exit beams that exit the waveguide at a large number of locations, resulting in a fairly uniform pattern of exit emission toward the eye 210, as this particular collimated beam bounces around within the waveguide.

[0036] In some embodiments, one or more DOEs may be switchable between an "on" state in which they actively diffract and an "off" state in which they do not significantly diffract. For example, a switchable DOE may include a layer of polymer-dispersed liquid crystal in which microdroplets comprise a diffractive pattern in a host medium, and the refractive index of the microdroplets may be switched to substantially match the refractive index of the host material (in which case the pattern does not appreciably diffract incident light), or the microdroplets may be switched to a refractive index that does not match the refractive index of the host medium (in which case the pattern actively diffracts incident light).

[0037] In some embodiments, a camera assembly 630 (e.g., a digital camera, including a visible light and infrared light camera) may be provided to capture images of the eye 210, a portion of the eye 210, or at least a portion of the tissue surrounding the eye 210, for example, detecting user input, extracting biometric information from the eye, estimating and tracking the eye's gaze direction, monitoring the user's physiological state, etc. As used herein, a camera may refer to any image capture device. In some embodiments, the camera assembly 630 may include an image capture device and a light source for projecting light (e.g., IR or near-infrared light) onto the eye, which may then be reflected by the eye and detected by the image capture device. In some embodiments, the light source includes an IR or near-infrared emitting light emitting diode (“LED”). While the light source is illustrated as being attached to the camera assembly 630, it will be understood that the light source may be located in other areas relative to the camera assembly such that light emitted by the light source is directed toward the wearer's eye (e.g., light source 530 described herein). In some embodiments, camera assembly 630 may be mounted on frame 80 (FIG. 2) and may be in electronic communication with processing module 140 or 150, which may process image information from camera assembly 630 to make various determinations, such as those related to the user's physiological state, the wearer's gaze direction, iris identification, etc., as discussed herein. Information related to the user's physiological state may be used to determine the user's behavioral or emotional state. Examples of such information include the user's movements or the user's facial expressions. The user's behavioral or emotional state may then be triangulated with collected environmental or virtual content data to determine the relationship between the behavioral or emotional state, the physiological state, and the environmental or virtual content data. In some embodiments, one camera assembly 630 may be utilized for each eye to monitor each eye separately.

[0038] Referring now to FIG. 7 , an example of an exit beam output by a waveguide is shown. While one waveguide is illustrated, it will be understood that if the waveguide assembly 260 includes multiple waveguides, the other waveguides in the waveguide assembly 260 ( FIG. 6 ) may function similarly. Light 640 is injected into the waveguide 270 at the input face 460 of the waveguide 270 and propagates within the waveguide 270 by TIR. At the point where the light 640 impinges on the DOE 570, a portion of the light exits the waveguide as exit beam 650. Although the exit beams 650 are illustrated as being substantially parallel, as discussed herein, they may be redirected to propagate to the eye 210 at an angle (e.g., forming a diverging exit beam) depending on the depth plane associated with the waveguide 270. A substantially parallel exit beam may indicate a waveguide with outcoupling optics that outcouples light to form an image that appears to be set on a depth plane at a long distance (e.g., optical infinity) from the eye 210. Other waveguides or sets of outcoupling optics may output a more divergent exit beam pattern, which requires the eye 210 to accommodate to closer distances to focus on the retina, which is interpreted by the brain as light from a distance closer to the eye 210 than optical infinity.

[0039] In some embodiments, a full-color image can be formed at each depth plane by overlaying images in each of the component colors, e.g., three or more component colors. FIG. 8 illustrates an example of a stacked waveguide assembly in which each depth plane includes an image formed using multiple different component colors. While the illustrated embodiment shows depth planes 240a-240f, a greater or lesser number of depth planes may be contemplated. Each depth plane may have three or more component color images associated with it, including a first image in a first color, G, a second image in a second color, R, and a third image in a third color, B. Different depth planes are indicated in the figure by different numbers of diopters (dpt) following the letters G, R, and B. By way of example only, the number following each of these letters indicates the diopter (1 / m), or inverse distance of the depth plane from the viewer, and each box in the figure represents an individual component color image. In some embodiments, the exact positioning of the depth planes for the different component colors may vary, taking into account differences in how the eye focuses light of different wavelengths. For example, different component color images for a given depth plane may be positioned on depth planes corresponding to different distances from the user, which may increase visual acuity and user comfort or reduce chromatic aberration.

[0040] In some embodiments, light for each component color may be output by a single dedicated waveguide, resulting in each depth plane having multiple waveguides associated with it. In such embodiments, each box containing a G, R, or B letter in the diagram may be understood to represent an individual waveguide, and three waveguides may be provided per depth plane, resulting in three component color images per depth plane. For ease of illustration, the waveguides associated with each depth plane are shown adjacent to each other in this drawing, but it will be understood that in a physical device, these waveguides may all be arranged in a stack, one waveguide per level. In other embodiments, multiple component colors may be output by the same waveguide, resulting in, for example, only a single waveguide per depth plane.

[0041] 8, in some embodiments, G is green, R is red, and B is blue. In other embodiments, other colors associated with other wavelengths of light may be used, including magenta and cyan, in addition to red, green, or blue, or may substitute for one or more of red, green, or blue. In some embodiments, features 320, 330, 340, and 350 may be active or passive optical filters configured to block light from the surrounding environment or selectively pass it through to the viewer's eyes.

[0042] It will be appreciated that references throughout this disclosure to a given color of light will be understood to encompass light of one or more wavelengths within the wavelength range of light perceived by a viewer as being of that given color. For example, red light may include one or more wavelengths of light within the range of about 620-780 nm, green light may include one or more wavelengths of light within the range of about 492-577 nm, and blue light may include one or more wavelengths of light within the range of about 435-493 nm.

[0043] In some embodiments, light source 530 (FIG. 6) may be configured to emit light at one or more wavelengths outside the range of visual perception of a viewer, such as IR or ultraviolet wavelengths. Infrared light may include light having wavelengths in the range of 700 nm to 10 μm. In some embodiments, infrared light may include near-infrared light having wavelengths in the range of 700 nm to 1.5 μm. Waveguide incoupling, outcoupling, and other light redirecting structures in display 250 may further be configured to direct and emit this light from the display toward the user's eye 210, for example, for imaging or user stimulation.

[0044] Referring now to FIG. 9A , in some embodiments, light impinging on a waveguide may need to be redirected to incouple the light into the waveguide. An incoupling optical element may be used to redirect and incouple the light into its corresponding waveguide. FIG. 9A shows a cross-sectional side view of an example of a plurality of stacked waveguides, or stacked waveguide set 660, each including an incoupling optical element. The waveguides may each be configured to output light of one or more different wavelengths or one or more different wavelength ranges. It will be understood that stack 660 may correspond to stack 260 ( FIG. 6 ), and that the illustrated waveguide of stack 660 may correspond to a portion of plurality of waveguides 270, 280, 290, 300, 310, except that light from one or more of image injection devices 360, 370, 380, 390, 400 is injected into the waveguide from a location requiring light redirection for incoupling.

[0045] The illustrated stacked waveguide set 660 includes waveguides 670, 680, and 690. Each waveguide includes an associated incoupling optical element (which may also be referred to as a light input area on the waveguide), such that incoupling optical element 700 is disposed on a major surface (e.g., the top major surface) of waveguide 670, incoupling optical element 710 is disposed on a major surface (e.g., the top major surface) of waveguide 680, and incoupling optical element 720 is disposed on a major surface (e.g., the top major surface) of waveguide 690. In some embodiments, incoupling optical elements 700, 710, 720 may be disposed on the bottom major surface of each waveguide 670, 680, 690 (particularly where one or more of the incoupling optical elements is a reflective, deflecting optical element). As shown, incoupling optical elements 700, 710, 720 may be disposed on the top major surface of each of these waveguides 670, 680, 690 (or on top of the next lower waveguide), and in this particular case, these incoupling optical elements are transmissive polarizing optical elements. In some embodiments, incoupling optical elements 700, 710, 720 may be disposed within the body of each of the waveguides 670, 680, 690. In some embodiments, as discussed herein, the incoupling optical elements 700, 710, 720 are wavelength selective, such that they selectively redirect light of one or more wavelengths while transmitting light of other wavelengths. Although illustrated on one side or corner of each of these waveguides 670, 680, 690, the incoupling optical elements 700, 710, 720 may be disposed within other areas of each of these waveguides 670, 680, 690 in some embodiments.

[0046] As shown, the incoupling optical elements 700, 710, 720 may be laterally offset from one another. In some embodiments, each incoupling optical element may be offset to receive light without passing that light through another incoupling optical element. For example, each incoupling optical element 700, 710, 720 may be configured to receive light from a different image injection device 360, 370, 380, 390, and 400, as shown in FIG. 6 , and may be spaced apart (e.g., laterally spaced) from the other incoupling optical elements 700, 710, 720 so as to substantially not receive light from other ones of the incoupling optical elements 700, 710, 720.

[0047] Each waveguide also includes an associated light distributing element, e.g., light distributing element 730 disposed on a major surface (e.g., the top major surface) of waveguide 670, light distributing element 740 disposed on a major surface (e.g., the top major surface) of waveguide 680, and light distributing element 750 disposed on a major surface (e.g., the top major surface) of waveguide 690. In some other embodiments, light distributing element 730, 740, 750 may be disposed on the bottom major surface of each associated waveguide 670, 680, 690. In some other embodiments, light distributing element 730, 740, 750 may be disposed on both the top and bottom major surfaces of each associated waveguide 670, 680, 690, or light distributing element 730, 740, 750 may be disposed on different ones of the top and bottom major surfaces of each different associated waveguide 670, 680, 690.

[0048] The waveguides 670, 680, 690 may be spaced apart and separated, for example, by a layer of gas, liquid, or solid material. For example, as shown, layer 760a may separate waveguides 670 and 680, and layer 760b may separate waveguides 680 and 690. In some embodiments, layers 760a and 760b are formed of a low refractive index material (i.e., a material having a lower refractive index than the material forming the immediately adjacent ones of waveguides 670, 680, 690). Preferably, the refractive index of the material forming layers 760a and 760b is 0.05 or greater, or 0.10 or less than the refractive index of the material forming waveguides 670, 680, 690. Advantageously, the lower refractive index layers 760a, 760b may act as cladding layers to facilitate TIR of light through the waveguides 670, 680, 690 (e.g., TIR between the top and bottom major surfaces of each waveguide). In some embodiments, the layers 760a, 760b are formed of air. Although not shown, it will be understood that the top and bottom of the illustrated waveguide set 660 may include immediately adjacent cladding layers.

[0049] Preferably, for ease of manufacturing and other considerations, the materials forming waveguides 670, 680, 690 are similar or identical, and the materials forming layers 760a, 760b are similar or identical. In some embodiments, the materials forming waveguides 670, 680, 690 may vary between one or more waveguides, or the materials forming layers 760a, 760b may differ while still maintaining the various refractive index relationships described above.

[0050] 9A, light rays 770, 780, 790 are incident on waveguide set 660. It will be appreciated that light rays 770, 780, 790 may be injected into waveguides 670, 680, 690 by one or more image injection devices 360, 370, 380, 390, 400 (FIG. 6).

[0051] In some embodiments, the light beams 770, 780, 790 have different characteristics, such as different wavelengths or different wavelength ranges, that may correspond to different colors. The incoupling optical elements 700, 710, 720 each deflect the incident light so that the light propagates through a respective one of the waveguides 670, 680, 690 by TIR.

[0052] For example, incoupling optical element 700 may be configured to deflect light ray 770, which has a first wavelength or range of wavelengths. Similarly, transmitted light ray 780 impinges on and is deflected by incoupling optical element 710, which is configured to deflect light of a second wavelength or range of wavelengths. Similarly, light ray 790 is deflected by incoupling optical element 720, which is configured to selectively deflect light of a third wavelength or range of wavelengths.

[0053] 9A , the deflected light rays 770, 780, 790 are deflected to propagate through the corresponding waveguides 670, 680, 690. That is, the incoupling optical element 700, 710, 720 of each waveguide deflects the light into the corresponding waveguide 670, 680, 690, incoupling the light into the corresponding waveguide. The light rays 770, 780, 790 are deflected at an angle that causes the light to propagate through the respective waveguides 670, 680, 690 by TIR. The light rays 770, 780, 790 propagate through the respective waveguides 670, 680, 690 by TIR until they impinge on the corresponding light distributing element 730, 740, 750 of the waveguide.

[0054] Referring now to Figure 9B, a perspective view of an example of the multiple stacked waveguides of Figure 9A is shown. As described above, incoupled light rays 770, 780, and 790 are deflected by incoupling optical elements 700, 710, and 720, respectively, and then propagate by TIR within waveguides 670, 680, and 690, respectively. Light rays 770, 780, and 790 then impinge on light distributing elements 730, 740, and 750, respectively. Light distributing elements 730, 740, and 750 deflect light rays 770, 780, and 790 so that they propagate toward outcoupling optical elements 800, 810, and 820, respectively.

[0055] In some embodiments, the light distributing elements 730, 740, 750 are orthogonal pupil expanders (OPEs). In some embodiments, the OPEs both deflect or distribute light toward the outcoupling optical elements 800, 810, 820, and also increase the beam or spot size of this light as it propagates to the outcoupling optical elements. In some embodiments, for example, if the beam size is already the desired size, the light distributing elements 730, 740, 750 may be omitted, and the incoupling optical elements 700, 710, 720 may be configured to deflect light directly toward the outcoupling optical elements 800, 810, 820. For example, referring to FIG. 9A , the light distributing elements 730, 740, 750 may replace the outcoupling optical elements 800, 810, 820, respectively. In some embodiments, the outcoupling optical element 800, 810, 820 is an exit pupil (EP) or exit pupil expander (EPE) that directs light into the viewer's eye 210 (FIG. 7). It will be understood that the OPE may be configured to increase the dimension of the eyebox in at least one axis, and the EPE may expand the eyebox in an axis that intersects, e.g., is perpendicular to, the axis of the OPE.

[0056] 9A and 9B, in some embodiments, a waveguide set 660 includes, for each component color, waveguides 670, 680, 690, incoupling optical elements 700, 710, 720, light distribution elements (e.g., OPEs) 730, 740, 750, and outcoupling optical elements (e.g., EPs) 800, 810, 820. The waveguides 670, 680, 690 may be stacked with air gaps / cladding layers between them. The incoupling optical elements 700, 710, 720 redirect or deflect incident light into their respective waveguides (different incoupling optical elements receive different wavelengths of light). The light then propagates at angles that result in TIR within the respective waveguides 670, 680, 690. In the illustrated example, after being deflected by the first incoupling optical element 700, light ray 770 (e.g., blue light) continues to bounce down the waveguide, interacting with the light distributing element (e.g., OPE) 730 and thus the outcoupling optical element (e.g., EP) 800 as previously described. Light rays 780 and 790 (e.g., green and red light, respectively) pass through the waveguide 670, where light ray 780 strikes and is deflected by the incoupling optical element 710. Light ray 780 then bounces down the waveguide 680 via TIR toward its light distributing element (e.g., OPE) 740 and thus the outcoupling optical element (e.g., EP) 810. Finally, light ray 790 (e.g., red light) passes through the waveguide 690 and strikes the light incoupling optical element 720 of the waveguide 690. The light incoupling optical element 720 deflects the light ray 790 so that it propagates by TIR to the light distribution element (e.g., OPE) 750 and then by TIR to the outcoupling optical element (e.g., EP) 820. Finally, the outcoupling optical element 820 outcouples the light ray 790 towards a viewer, who also receives outcoupled light from the other waveguides 670, 680.

[0057] Figure 9C is a top plan view of an example of multiple stacked waveguides of Figures 9A and 9B. As shown, waveguides 670, 680, 690 may be vertically aligned, along with each waveguide's associated light distribution element 730, 740, 750 and associated outcoupling optical elements 800, 810, 820. However, as discussed herein, it is preferred that the incoupling optical elements 700, 710, 720 are not vertically aligned; instead, the incoupling optical elements are non-overlapping (e.g., laterally spaced apart, as seen in the top view). As discussed further herein, this non-overlapping spatial arrangement facilitates injecting light from different sources into different waveguides on a one-to-one basis, thereby ensuring that a particular light source is uniquely coupled to a particular waveguide. In some embodiments, arrangements including non-overlapping, spatially separated incoupling optical elements may be referred to as shift pupil systems, and the incoupling optical elements in these arrangements may correspond to sub-pupils.

[0058] Exemplary Imaging System The eye or tissue surrounding the eye of a wearer of an HMD (e.g., wearable display system 200 shown in FIG. 2) can be imaged using one or more optical elements to direct light from the eye to a camera assembly. The resulting images can be used for monocular or binocular tracking, retinal imaging, three-dimensional reconstruction of the eye shape, extraction of biometric information from the eye (e.g., iris identification), etc.

[0059] As outlined above, there are various reasons why an HMD may use information about the state of a wearer's eyes. For example, this information may be used to estimate the wearer's gaze direction or for biometric identification. However, the short distance between the HMD and the wearer's eyes makes this problem challenging. This is further complicated by the fact that gaze tracking requires a wider field of view, while biometric identification requires a relatively large number of pixels for a target on the iris. For an imaging system that attempts to achieve both of these objectives, the requirements of these two tasks are in strong conflict. Finally, occlusions by eyelids and eyelashes further complicate both issues. Embodiments of the imaging system described herein may address at least some or all of these issues.

[0060] 10A and 10B schematically illustrate an example imaging system 1000a configured to image one or both eyes 210, 220 of a wearer 90. The imaging system 1000a includes a substrate 1070 and a camera assembly 1030 positioned to view the eye 220. The embodiment of the imaging system 1000a described herein with reference to FIGS. 10A and 10B can be used with an HMD that includes a display device described herein (e.g., the wearable display system 60 shown in FIG. 2, the display system 250 shown in FIGS. 6 and 7, and the stack 660 of FIGS. 9A-9C). For example, in some implementations in which imaging system 1000a is part of display system 250 of FIG. 6, substrate 1070 may comprise or replace one of waveguides 270, 280, 290, 300, or 310, may be disposed between two of waveguides 270, 280, 290, 300, or 310, may be disposed between waveguide stack 260 and eye 210, or may be disposed between waveguide stack 260 and world 510.

[0061] In some embodiments, camera assembly 1030 may be mounted near the wearer's eye, such as on frame 80 of wearable display system 60 of FIG. 2 (e.g., on temples 82 near the wearer's temples), around the edge of display (e.g., eyepiece) 70 of FIG. 2 (as shown in FIG. 10B ), and / or integrated into display (e.g., eyepiece) 70 of FIG. 2. Camera assembly 1030 may be substantially similar to camera assembly 630 of FIG. 6. In other embodiments, a second camera assembly may be used to separately image the wearer's other eye 210. Camera assembly 1030 may include an IR digital camera sensitive to IR radiation. The camera assembly 1030 can be mounted so that it faces forward (e.g., toward the wearer's field of view), as shown in FIG. 10A, or the camera assembly 1030 can be mounted so that it faces backward and is oriented backward, e.g., toward the eye 220 or to the side of the eye (e.g., FIG. 10B).

[0062] In some embodiments, the camera assembly 1030 may include an image capture device and a light source 1032 that projects light onto the eye 220, which may then be reflected back to the eye 220 and detected by the camera assembly 1030. Although the light source 1032 is shown as being attached to the camera assembly 1030, the light source 1032 may be disposed in other areas relative to the camera assembly, such that light emitted by the light source is directed towards the wearer's eye and reflected back towards the camera assembly 1030. For example, the imaging system 1000a may be part of the display system 250 ( FIG. 6 ), the substrate 1070 may include or replace one of the waveguides 270, 280, 290, 300, or 310, and the light source 1032 may be one of the light emitters 360, 370, 380, 390 or the light source 530.

[0063] 10A , the camera assembly 1030 is positioned to be directed toward a substrate, such as a proximal surface 1074 of the substrate 1070. The substrate 1070 may be, for example, part of the display (e.g., eyepiece) 70 of FIG. 2 or a lens of a pair of eyeglasses. The substrate 1070 may be transmissive to at least 10%, 20%, 30%, 40%, or 50% or more of the visible light incident on the substrate 1070, and possibly up to 80%, 90%, 95%, 98%, 99%, 99.9%, 99.99% or more, or within any range formed by any of these values. In other embodiments, the substrate 1070 need not be transparent (e.g., in a virtual reality display). The substrate 1070 may comprise one or more reflective optical elements 1078. In some embodiments, the one or more reflective optical elements 1078 can be selected to reflect a first range of wavelengths while being substantially transmissive to a second range of wavelengths different from the first range of wavelengths. In some embodiments, the first range of wavelengths can be IR wavelengths, and the second range of wavelengths can be visible wavelengths. Thus, in some implementations, the one or more reflective optical elements can include a hot mirror or reflector 1078. The substrate 1070 can include a polymer or plastic material, such as polycarbonate, or other lightweight material with desired optical properties. Without being bound to a particular scientific theory, plastic materials may be less rigid and therefore less susceptible to breakage or failure during use. Plastic materials can also be lightweight, which, when combined with plastic materials, can allow for thinner substrates, facilitating the production of compact and lightweight imaging systems. While the substrate 1070 is described as including a polymer, such as polycarbonate or other plastic, with desired optical properties, other materials, such as fused silica, with desired optical properties are also possible.

[0064] The one or more reflective optical elements 1078 may include reflective optical elements configured to reflect or redirect light in a first range of wavelengths (e.g., infrared light) while transmitting light in a second range of wavelengths (e.g., visible light). In such embodiments, infrared light 1010a, 1012a, and 1014a from the eye 220 reflects off the one or more optical elements 1078, resulting in reflected infrared light 1010b, 1012b, 1014b that can be used to form an image by the camera assembly 1030. In some embodiments, the camera assembly 1030 may be sensitive to or capable of capturing at least a subset (a non-empty subset, or a subset less than all) of the wavelengths in the first range reflected by the one or more reflective optical elements 1078. For example, if one or more reflective optical elements 1078 may be partially transmissive, the one or more optical elements 1078 may reflect infrared light in the range of 700 nm to 1.5 μm, and the camera assembly 1030 may be sensitive to or capable of capturing near-infrared light with wavelengths in the range of 700 nm to 900 nm. As another example, the one or more reflective optical elements 1078 may reflect infrared light in the range of 700 nm to 1.5 μm. In some implementations, the camera assembly 1030 may include a filter that filters infrared light in the range of 900 nm to 1.5 μm, such that the camera assembly 1030 is capable of capturing near-infrared light with wavelengths in the range of 700 nm to 900 nm.

[0065] Visible light from the outside world (e.g., world 510 in FIG. 6 ) can pass through the substrate 1070 and be perceived by the wearer. Nevertheless, light can be reflected from objects on the opposite side of the substrate (e.g., the eye side) and enter the camera assembly 1030. In effect, as a result of this reflection caused by the reflective optical element(s) 1078, the imaging system 1000a can behave as if there were a virtual camera assembly 1030c facing away from the wearer's eye 220, capturing images seen directly by the eye 220. The virtual camera assembly 1030c is labeled with a "c" reference because it may include a virtual image formed by virtual infrared light rays 1010c, 1012c, and 1014c propagating from the wearer's eye 220 to the substrate 1070. The reflective optical element(s) 1078 are shown as being disposed on a proximal surface 1074 of the substrate 1070, although other configurations are possible. For example, one or more reflective optical elements 1078 may be disposed on a distal surface 1076 of the substrate 1060 or within the substrate 1070 .

[0066] Thus, although an exemplary arrangement of imaging system 1000a is shown in Figures 10A and 10B, other arrangements are possible.

[0067] Exemplary Imaging System Integrated in or on the Eyepiece FIG. 11A schematically illustrates another exemplary imaging system 2000 usable with an HMD (e.g., eyewear) including a display system described herein (e.g., the wearable display system 60 shown in FIG. 2 , the display system 250 shown in FIGS. 6 and 7 , and the stack 660 of FIGS. 9A-9C ). For example, in some implementations, the eyepiece 2010 shown in FIG. 11A can be coupled to a frame (e.g., the frame 80 shown in FIG. 2 ), which can be configured to be supported on a user's head. Furthermore, the eyepiece 2010 can be configured to be positioned in front of the user's eye 2020. The eyepiece 2010 can include at least a portion that is transparent to transmit light from a portion of the environment 2021 in front of the eyepiece 2010 to the user's eye 2020 so as to view through the eyepiece and the environment in front of the user's eye. The eyepiece 2010 can also provide virtual content to the user's eye 2020 for viewing. In various implementations, the eyepiece 2010 can include one or more layers (e.g., one or more of the waveguides 270, 280, 290, 300, and 310 shown in FIG. 6). Additionally, one or more of the image injection devices (e.g., image injection devices 360, 370, 380, 390, and 400 shown in FIG. 6) can be configured to provide image content to the eyepiece 2010 for viewing by the user. For example, the eyepiece 2010 can be configured to direct light from one or more image injection devices to the user's eye 2020 to present image content to the user.

[0068] The imaging system 2000 may also include a sensor array 2030, a refractive or transmissive diffractive optical element 2040, and a reflector 2050. In various implementations, the imaging system 2000 and / or various portions thereof (e.g., the sensor array 2030, the refractive or transmissive diffractive optical element 2040, and the reflector 2050) may be disposed in or on or integrated with the eyepiece 2010. The reflector 2050 can be configured to reflect light 2060a received from an object (e.g., a user's eye 2020, a portion of the eye 2020, or a portion of tissue surrounding the eye 2020) for imaging by the sensor array 2030. The reflector or reflective optical element 2050 may be partially reflective and partially transmissive so that the user can view the environment in front of the user through it. The refractive or transmissive diffractive optical element 2040 can be configured to receive light 2060b reflected from the reflector 2050 and redirect at least a portion of the light 2060c towards the sensor array 2030. The refractive or transmissive diffractive optical element 2040 may include, for example, a transmissive element such as a transmissive diffractive optical element or a diffraction grating configured to transmit and diffract light from the reflector 2050 incident thereon, thereby deflecting and redirecting at least a portion of the light to the sensor array 2030. The refractive or transmissive diffractive optical element 2040 may also include a transmissive element such as a refractive optical element configured to refract light from the reflector 2050 incident thereon, thereby deflecting and redirecting at least a portion of the light to the sensor array 2030.

[0069] In some implementations, the refractive or transmissive diffractive optical element 2040 can include an off-axis optical element. Such an off-axis optical element can deflect a beam so that it propagates parallel to or along an axis, such as the optical axis of the optical element or optical system, when the incident beam is directed in a different, "skewed" direction rather than propagating along that axis. In this case, the optical axis can be the optical axis of a camera with a sensor array or can include the optical axis of the off-axis optical element. The off-axis optical element can have a mechanical center that is offset from the optical axis or center of rotational symmetry. In various implementations, the off-axis optical element is configured to have an axis, e.g., an optical axis, that is perpendicular to the off-axis optical element and / or the eyepiece 2010 or layer in or on which the off-axis optical element is disposed. This optical axis can, in some cases, correspond to an axis of symmetry for an optical surface or optical feature contained within or on the off-axis optical element. Light from the reflector 2050 may be deflected, diffracted, refracted, redirected, or a combination thereof, so that it propagates further along the axis or optical axis to the sensor array 2030.

[0070] As a result of the reflector, imaging system 2000 can effectively behave as if there is a virtual camera assembly 2030d facing away from object 2020, capturing a direct view of object 2020. Virtual camera assembly 2030d may be thought of as capturing images of object 2020 via virtual infrared light rays 2060d propagated from the object through eyepiece 2010, for example.

[0071] As described herein, the eyepiece portion 2010 may include one or more layers. In FIG. 11A, three layers 2011, 2012, and 2013 are shown. However, in some implementations, only one or two layers may be included. In other implementations, more than two layers (e.g., three, four, five, six, seven, eight, nine, ten, etc., or any range formed by such values) may be included. The layers 2011, 2012, and 2013 may include one or more waveguides configured to guide light therein by total internal reflection. For example, layers 2011, 2012, 2013 may include one or more waveguides (e.g., one or more of waveguides 270, 280, 290, 310 shown in FIG. 6) configured to receive light from one or more image injection devices (e.g., one or more of image injection devices 360, 370, 380, 390, 400 shown in FIG. 6). The one or more waveguides may be configured to propagate light therein by total internal reflection. The one or more waveguides may include one or more incoupling optical elements (e.g., 700, 710, 720 shown in FIG. 9A ), outcoupling optical elements (e.g., 570, 580, 590, 600, 610 shown in FIG. 6 ), or other light redirecting optical elements (e.g., 730, 740, 750 shown in FIG. 9A ) configured to distribute and / or extract light exiting the one or more waveguides to output image information to the user's eye 2020. In some implementations, the waveguides may include one or more compound pupil expanders (CPEs) configured to expand the pupil and output light guided within the waveguide. Any one or more of these incoupling optical elements, outcoupling optical elements, redirecting optical elements, and compound pupil expanders may include a diffractive optical element, such as a diffraction grating or a holographic optical element. In some cases, different waveguides can be arranged to provide different color image content.

[0072] Other optical elements (e.g., lenses, polarizers, prisms, etc.) may be used to, for example, focus, correct aberrations, direct, etc. light as desired for a particular application. For example, the eyepiece 2010 (e.g., one or more layers 2011, 2012, 2013) may include one or more lenses (e.g., lenses 320, 330, 340, 350 shown in FIG. 6) configured to direct image information to the eye 2020 with various levels of wavefront curvature or beam divergence (e.g., in a collimated beam), or lack thereof. As described herein, the eyepiece 2010 may include different waveguides or groups of waveguides configured to project light into the user's eye 2020 to display image content with different amounts of divergence and / or collimation, and in some cases, to provide light that is divergent or collimated as if projected from different distances from the user's eye 2020. In some cases, the eyepiece 2010 may include an anterior or posterior lens. For example, a contributing lens layer 620, as shown in FIG. 6, may be used to contribute to the collective power of the lenses in layers 2011, 2012, and 2013. As another example, the lenses may be used to extend the depth of focus. In some cases, prescription lenses may be used to provide refractive correction for users with refractive errors. For example, the lenses may be used to correct myopia, hyperopia, presbyopia, astigmatism, etc., or any combination thereof.

[0073] In some implementations, the eyepiece 2010 (e.g., one or more layers 2011, 2012, 2013) may include an outer lens or window, such as a decorative (or cosmetic) lens. For example, the outer or decorative lens may have a shape, such as a curved shape, that affects the appearance of the eyewear. In some implementations, the outer or decorative lens may be a lens used like sunglasses to filter sunlight. In another example, the outer or decorative lens may be a color filtering lens for use in goggles. In yet another example, the outer or decorative lens may have a colored visual appearance that is visible to others not wearing the lens (e.g., a lens that appears blue, red, etc. to others). The outer or decorative lens may also include a color layer that is visible by people other than the user. In some cases, one or more lenses or surfaces may be provided with an anti-reflective and / or scratch-resistant coating.

[0074] In some implementations, the eyepiece 2010 (e.g., one or more layers 2011, 2012, 2013) can include an illumination layer and / or a dimmer. The illumination layer and / or dimmer may be used to adjust the brightness and / or contrast of the eyepiece 2010 and, therefore, the projected image for viewing. As one example, the illumination layer can provide light (e.g., from one or more light sources) to illuminate the eye with infrared light, for example, to be imaged and / or, in some cases, to provide a flash of light to the eye for eye tracking. As another example, the dimmer can reduce the amount of light to match the image being viewed. In some cases, the dimmer can include one or more films (e.g., one or more rainbow films or filters to block certain wavelengths or amounts of light). In some cases, the dimmer can be a curved layer (e.g., a curved lens).

[0075] In some implementations, the eyepiece 2010 (e.g., one or more layers 2011, 2012, 2013) may include a material having desired optical properties. As described herein, when positioned in front of a user's eye 2020 when the user is wearing the HMD, the eyepiece 2010 may include a portion that transmits light from a portion of the environment 2021 in front of the user and the eyepiece 2010 to the user's eye 2020 to provide a view of that portion of the environment 2021. In various implementations, one or more of the layers 2011, 2012, 2013 of the eyepiece 2010 may include at least a portion thereof that is transparent. In some examples, the eyepiece 2010 may include at least one glass or plastic / polymer layer that is transparent to visible light. In some cases, the eyepiece 2010 may be transmissive and / or transparent to at least 10%, 20%, 30%, 40%, 50% or more of visible light, and in some cases up to 80%, 90%, 95%, 98%, 99%, 99.9%, 99.99% or more, or within any range formed by any of these values.

[0076] 11A , the imaging system 2000 may include a sensor array 2030 integrated in or on the eyepiece. While the sensor array 2030 is illustrated in layer 2011, the sensor array 2030 may be integrated in or on any one or more of layers 2011, 2012, and 2013 of the eyepiece 2010, such as one or more of the waveguide, illumination layer, dimmer layer, front lens, rear lens, cosmetic or decorative lens, etc. The sensor array 2030 may be configured to image an object. For example, the sensor array 2030 may be positioned to face forward (e.g., toward the wearer's line of sight) as shown in FIG. 11A , and may be configured to view the wearer's eye 2020 (or a portion of the eye 2020 or a portion of tissue surrounding the eye 2020) and capture an image of the eye 2020 by receiving light from the eye that is reflected by a reflector 2050 onto the sensor array. As another example, as shown in FIG. 11B, the sensor array 2030 can be arranged to face backward (e.g., in the opposite direction from that shown in FIG. 11A) and directed toward the eye 2020, allowing the wearer to capture an image of an object 2022 in the environment 2021 being viewed by the wearer by receiving light from the object reflected onto the sensor array by the reflector 2050.

[0077] The sensor array 2030 can be integrated in or on the eyepiece 2010 at a location anywhere around the wearer's eyes 2020 so that the sensor array 2030 does not obstruct the wearer's view of the surrounding world, or central field of vision, or forward line of sight, or interfere with the operation of the HMD. In some implementations, the sensor array 2030 can be integrated in or on the eyepiece 2010, for example, adjacent to the temples or temples, adjacent to the bridge of the nose, above the eyes, or below the eyes and above the cheeks. In some implementations, the sensor array 2030 can be integrated in or on the eyepiece 2010 closer to the edges of the eyepiece, or temples, or temples, or nose, or any combination thereof, than the center of the eyepiece, or at a location on the eyepiece through which the forward line of sight is directed. In some implementations, additional sensor arrays can be used for additional objects, such as the wearer's other eye so that each eye can be imaged separately, or other objects in the environment 2021 so that multiple objects can be imaged separately.

[0078] In some cases, the sensor array 2030 may include a CMOS detector array. In some cases, the sensor array 2030 may include a CCD detector array. In some examples, the sensor array 2030 may include detector pixels formed (e.g., engraved) or disposed (e.g., glued) on at least one of the layers 2011, 2012, and 2013 of the eyepiece 2010. The detector pixels may be formed or disposed on at least one of the layers 2011, 2012, and 2013, at least a portion of which may be transparent and disposed in front of the user's eye 2020 when the user wears the HMD, such that the transparent portion transmits light from a portion of the environment 2021 in front of the user and the eyepiece 2011 to the user's eye 2020, allowing the user to view that portion of the environment 2021. The sensor array 2030 may include wafer-scale optics. The wafer-scale optics may be integrated into or on the eyepiece. In various implementations, the wafer-scale optics is not contained within a housing that excludes the eyepiece 2010. For example, the wafer-scale optics is not contained in a housing separate from the housing that houses the eyepiece 2010. Rather, the wafer-scale optics may be contained within or on one of layers 2011, 2012, 2013 of the eyepiece 2010. The wafer-scale optics may comprise, for example, an imaging lens configured to form an image of an object on the sensor array. Thus, the wafer-scale optics may include optical elements having refractive power, and the wafer-scale optics may include a refractive lens.

[0079] In various implementations, a light source (e.g., 1032 in FIGS. 10A-10B ) can be positioned such that light from the light source can be directed toward the object 2020 for imaging and reflected toward the sensor array 2030. In some designs, the light source may project light in the infrared (IR) or near-infrared spectrum, and the sensor array 2030 may be sensitive to such light or a subset of such light. In some implementations, the light source faces forward toward the eyepiece and the environment in front of the eye, causing light to reflect off a reflector toward the eye. In some implementations, the light source faces forward toward the eye, causing light to illuminate the eye.

[0080] In some implementations, the sensor array 2030 may be sensitive to or capable of imaging at least a subset (a non-empty subset, or a subset less than all) of the wavelengths in the first range reflected by the reflector 2050. For example, the reflector 2050 may reflect infrared light in the 700 nm to 1.5 μm range, and the sensor array 2030 may be sensitive to or capable of imaging near-infrared light in the 700 nm to 900 nm wavelength range. As another example, the reflector 2050 may reflect infrared light in the 700 nm to 1.5 μm range, and the sensor array 2030 may include a filter that filters infrared light in the 900 nm to 1.5 μm range, allowing the sensor array 2030 to image near-infrared light in the 700 nm to 900 nm wavelength range. However, other configurations are possible, and the sensor array 2030 may in some cases be sensitive to a wider range of wavelengths than that reflected by the reflector 2050.

[0081] In various implementations, light from the light source may be reflected toward the sensor array 2030 by a reflector 2050. As shown in FIG. 11A , the reflector 2050 may be disposed within or on the eyepiece 2010. While the reflector 2050 is illustrated on layer 2013, it may be formed on any one or more of layers 2011, 2012, 2013 of the eyepiece 2010 (e.g., a waveguide, an illumination layer, a dimmer layer, a front lens, a rear lens, a cosmetic or decorative lens, etc.), or the reflector 2050 may include one of layers 2011, 2012, 2013 of the eyepiece 2010. In some cases, the reflector 2050 may be disposed in or on at least one layer 2011, 2012, 2013 of the eyepiece 2010, at least a portion of which is transparent and disposed at a location in front of the user's eyes 2020 when the user is wearing the HMD, such that the transparent portion transmits light from a portion of the environment 2021 in front of the user and the eyepiece 2011 to the user's eyes 2020, making that portion of the environment 2021 visible. In some cases, the reflector 2050 may include a hot mirror. In some cases, the reflector 2050 may include an optical coating.

[0082] The reflector 2050 may be configured to reflect or redirect light in a first range of wavelengths (e.g., infrared or near-infrared light) while transmitting light in a second range of wavelengths (e.g., visible light). In some such implementations, infrared light 2060a from the eye 2020 propagates toward and reflects off the reflector 2050, resulting in reflected infrared light 2060b, at least a portion of which may be received and imaged by the sensor array 2030.

[0083] In some implementations, the reflector 2050 may include a diffractive optical element (DOE), such as, for example, the diffractive optical elements described herein. In some designs, for example, the reflector 2050 may include a diffraction grating. In some cases, the reflector 2050 may be a reflective diffractive optical element that diffracts light of a first wavelength (e.g., infrared or near-infrared light) through reflective diffraction, where light reflected from the reflector is diffracted while transmitting light of a second range of wavelengths (e.g., visible light). In some such implementations, infrared light 2060a from the eye 2020 propagates toward the reflector 2050 and is diffracted from the reflector 2050 (e.g., infrared light reflected from the reflector is diffracted), resulting in diffracted infrared light 2060b, at least a portion of which may be received and imaged by the sensor array 2030.

[0084] In some implementations, reflector 2050 includes a DOE, such as a holographic optical element (HOE), a holographic mirror (HM), or a volume diffractive optical element (VDOE). In some implementations, any of these may have refractive power. In some cases, reflector 2050 includes a diffractive optical element, such as a cholesteric liquid crystal diffractive optical element or a diffraction grating, which may be configured to increase and / or optimize any one or more of polarization selectivity, bandwidth, phase profile, spatial variation of diffractive properties, spectral selectivity, and / or diffraction efficiency, among others. For example, any of the CLCs or CLCGs described in U.S. Patent Application No. 15 / 835,108, filed December 7, 2017, entitled "Diffractive Devices Based On Cholesteric Liquid Crystal," can be implemented as reflector 2050 or a cholesteric diffractive optical element described herein, which is incorporated by reference in its entirety for all of its disclosure. In some embodiments, reflector 2050 may be a switchable DOE that can be switched between an actively diffracting "on" state and a non-significantly diffracting "off" state.

[0085] In some implementations, the reflector 2050 or diffractive optical element may comprise any liquid crystal grating or structure. The CLC or CLCG described above may be an example of a liquid crystal grating or structure. Other liquid crystal gratings or structures may also include liquid crystal features and / or patterns having sizes smaller than the wavelength of visible light, including what are referred to as Pancharatnam-Berry phase effect (PBPE) structures, metasurfaces, or metamaterials. For example, any of the PBPE structures, metasurfaces, or metamaterials described in U.S. Patent Publication No. 2017 / 0010466, entitled "Display System With Optical Elements For In-Coupling Multiplexed Light Streams," U.S. Patent Application No. 15 / 879,005, filed January 24, 2018, entitled "Antireflection Coatings For Metasurfaces," or U.S. Patent Application No. 15 / 841,037, filed December 13, 2017, entitled "Patterning Of Liquid Crystals Using Soft-Imprint Replication Of Surface Alignment Patterns," each of which is incorporated by reference in its entirety for all purposes. Such structures may be configured to manipulate light for beam steering, wavefront shaping, wavelength and / or polarization separation, and for combining different wavelengths and / or polarizations, can include liquid crystal gratings with metasurfaces, which are otherwise referred to as metamaterial liquid crystal gratings or liquid crystal gratings with PBPE structures. In some implementations, liquid crystal gratings with PBPE structures can combine the high diffraction efficiency and low sensitivity to angle of incidence of liquid crystal gratings while retaining the high wavelength sensitivity of PBPE structures.

[0086] 11A , the refractive or transmissive diffractive optical element 2040 is disposed on layer 2012, but may be disposed within or on any one of layers 2011, 2012, 2013 of the eyepiece 2010 (e.g., within or on any one or more of the waveguide, illumination layer, dimmer layer, front lens, rear lens, cosmetic or decorative lens, etc.). In some cases, the refractive or transmissive diffractive optical element 2040 may be inscribed on one of layers 2011, 2012, 2013 of the eyepiece 2010.

[0087] As an example, the sensor array 2030 may be arranged in or on a first layer 2011 of the eyepiece portion 2010, and the refractive or transmissive diffractive optical element 2040 may be arranged in or on a second, different layer 2012 of the eyepiece layer 2010. The first layer 2011 and / or the second layer 2012 may include portions thereof that are transparent, through which light is transmitted.

[0088] In various implementations, a refractive or transmissive diffractive optical element 2040 may be disposed in the optical path between the reflector 2050 and the sensor array 2030. For example, the reflector 2050 may be disposed on layer 2013, the refractive or transmissive diffractive optical element 2040 may be disposed on layer 2012, and the sensor array 2030 may be disposed on layer 2011. Layer 2012 may be disposed between layers 2013 and 2011. Layers 2011, 2012, and 2013 may be transparent and allow light to pass through from a portion of the environment 2021 in front of the user.

[0089] In some cases, the refractive or transmissive diffractive optical element 2040 can comprise at least one lens aligned with the sensor array 2030 such that light from the reflector 2050 passes through the at least one lens to the sensor array 2030 to form an image on the sensor array 2030. As an example, the at least one lens can include at least one diffractive optical element. As another example, the at least one lens can include at least one refractive lens. The at least one lens can include at least one off-axis refractive lens. In various implementations, the at least one lens can include wafer-scale optics. In some implementations, the at least one lens can have refractive power.

[0090] In some cases, the refractive or transmissive diffractive optical element 2040 may include a transmissive diffractive optical element (DOE) or a transmissive diffraction grating. In some implementations, the transmissive DOE or diffraction grating may include a holographic optical element (HOE), an off-axis holographic mirror (OAHM), or a volume diffractive optical element (OAVDOE). In some implementations, the transmissive DOE may include an off-axis DOE, an off-axis diffraction grating, an off-axis holographic optical element (HOE), an off-axis holographic mirror (OAHM), or an off-axis volume diffractive optical element (OAVDOE). In some implementations, any of these may have refractive power. In some embodiments, the OAHM may also have refractive power, in which case it may be an off-axis volume diffractive optical element (OAVDOE). In some cases, the transmissive diffractive optical element 2040 may be a cholesteric liquid crystal diffractive optical element or grating, such as an off-axis cholesteric liquid crystal grating (OACLCG), which may be configured to increase and / or optimize any one or more of polarization selectivity, bandwidth, phase profile, spatial variation of diffractive properties, spectral selectivity, and / or diffraction efficiency, among others. For example, any of the CLCs or CLCGs described in U.S. Patent Application No. 15 / 835,108, filed December 7, 2017, entitled "Diffractive Devices Based On Cholesteric Liquid Crystal," may be implemented as the cholesteric diffractive optical element 2040 described herein, which is incorporated by reference in its entirety for all of its disclosure. In some embodiments, the transmissive diffractive optical element 2040 may be a switchable DOE that may be switched between an actively diffracting "on" state and a non-significantly diffracting "off" state.

[0091] In some implementations, the transmissive diffractive optical element 2040 can be an optically transmissive liquid crystal grating. The CLC or CLCG described above can be an example of a liquid crystal grating. Other liquid crystal gratings can include liquid crystal features and / or patterns having sizes smaller than the wavelength of visible light, and can include what are referred to as Pancharatnam-Berry phase effect (PBPE) structures, metasurfaces, or metamaterials. For example, any of the PBPE structures, metasurfaces, or metamaterials described in U.S. Patent Publication No. 2017 / 0010466, entitled "Display System With Optical Elements For In-Coupling Multiplexed Light Streams," U.S. Patent Application No. 15 / 879,005, filed January 24, 2018, entitled "Antireflection Coatings For Metasurfaces," or U.S. Patent Application No. 15 / 841,037, filed December 13, 2017, entitled "Patterning Of Liquid Crystals Using Soft-Imprint Replication Of Surface Alignment Patterns," each of which is incorporated by reference in its entirety for all purposes. Such structures can be configured to manipulate light for beam steering, wavefront shaping, wavelength and / or polarization separation, etc., and for combining different wavelengths and / or polarizations, they can include liquid crystal gratings with metasurfaces, which are otherwise referred to as metamaterial liquid crystal gratings or liquid crystal gratings with PBPE structures. In some implementations, liquid crystal gratings with PBPE structures can combine the high diffraction efficiency and low sensitivity to angle of incidence of liquid crystal gratings while retaining the high wavelength sensitivity of PBPE structures.

[0092] As described herein, the imaging system 2000 can be used in a head-mounted display to track a user's gaze based on images of at least one of the user's eye, a portion of the user's eye, or a portion of tissue surrounding the eye. For example, as shown in FIG. 11A , the sensor array 2030 can be a forward-facing camera configured to image at least a portion of an object (e.g., the user's eye 2020, a portion of the eye 2020, or a portion of tissue surrounding the eye 2020) based, at least in part, on light received from a reflector 2050 disposed in front of the sensor array 2030. At least a portion of the object 2020 can be disposed behind the sensor array 2030.

[0093] In some implementations, the imaging system 2000 can be used to image an object in an environment 2021 in front of the eyepiece 2020. For example, as shown in FIG. 11B , the sensor array 2030 can be a rear-facing camera configured to image at least a portion of the object 2022 based at least in part on light received from a reflector 2050 disposed behind the sensor array 2030. At least a portion of the object 2022 can be disposed in front of the sensor array 2030 in the environment 2021 in front of the user. In the illustrated configuration, a refractive or transmissive diffractive optical element is disposed between the reflector 2050 and the sensor array 2030.

[0094] As disclosed herein, as shown in FIG. 11A , the sensor array 2030 may be disposed in or on a first layer 2011 of the eyepiece 2010, and the refractive or transmissive diffractive optical element 2040 may be disposed in or on a second, different layer 2012 of the eyepiece 2010. In various implementations, the refractive or transmissive diffractive optical element 2040 may include at least one imaging lens aligned with the sensor array 2030 to form an image on the sensor array 2030. In some implementations, the refractive or transmissive element may include an off-axis optical element, such as an off-axis lens. In some cases, the refractive or transmissive diffractive optical element 2040 may be disposed on the same layer as the sensor array 2030. For example, as shown in FIG. 12A , the sensor array 2030 may be disposed in or on a first side 2011 a of a layer 2011 of the eyepiece 2010, and the refractive or transmissive diffractive optical element 2040 a may be disposed in or on a second, opposite side 2011 b of the layer 2011. In various implementations, the refractive or transmissive diffractive optical element 2040 a may comprise at least one imaging lens aligned with the sensor array 2030 to form an image on the sensor array 2030. In some designs, the refractive or transmissive diffractive optical element includes an off-axis optical element, such as an off-axis lens. In some implementations, the layer 2011 in which the sensor array 2030 and the refractive or transmissive diffractive optical element 2040 a are disposed may include a transparent portion that transmits light.

[0095] In some implementations, the eyepiece 2010 may include more than one refractive or transmissive diffractive optical element 2040. The refractive or transmissive diffractive optical elements 2040 may be on the same and / or different layers 2011, 2012, 2013 from the sensor array 2030 to the eyepiece 2010. For example, as shown in FIG. 12B , the eyepiece 2010 includes refractive or transmissive diffractive optical elements 2040a, 2040b on both the same layer 2011 and different layers 2012 from the sensor array 2030 to the eyepiece 2010. In various implementations, the refractive or transmissive diffractive optical elements 2040a, 2040b may include an imaging lens aligned with the sensor array 2030 to form an image on the sensor array 2030. The layers 2011, 2012 in or on which the sensor array 2030 and refractive or transmissive diffractive optical elements 2040a, 2040b are disposed may each include portions thereof that are transparent and transmit light. In some implementations, the refractive or transmissive diffractive optical elements may include off-axis optical elements for redirecting light beams and imaging lenses for forming an image on the sensor array 2030.

[0096] In some cases, the multiple refractive or transmissive diffractive optical elements 2040a, 2040b may be on one or more layers different from the sensor array 2030. For example, the refractive or transmissive diffractive optical elements 2040a, 2040b may be on opposite sides 2012a, 2012b of the same layer 2012, but on different sides of the layer 2011 from the sensor array 2030. As another example, the refractive or transmissive diffractive optical elements 2040a, 2040b may be on separate layers (e.g., 2012, 2013 in FIGS. 11A-11B) from each other and from the sensor array 2030. The refractive or transmissive diffractive optical elements 2040a, 2040b may include one or more lenses (e.g., a single lens or multiple lenses). In some cases, one or more lenses may be used in conjunction with the sensor array 2030 to form an image on the sensor array 2030. In some designs, a refractive or transmissive diffractive may comprise at least one refractive or transmissive diffractive configured to redirect a light beam and at least one imaging lens on the same or a different layer.

[0097] FIG. 13 schematically illustrates an exemplary implementation of another imaging system 2100. In various implementations, the eyepiece 2110 may be similar to the eyepiece 2010 described with reference to FIGS. 11A-11B. For example, the eyepiece 2010 may include one or more layers 2111, 2112, and 2113. The layers 2111, 2112, and 2113 may be similar to the layers 2011, 2012, and 2013 described with reference to FIGS. 11A-11B (e.g., one or more of a waveguide, an illumination layer, a dimmer layer, a front lens, a rear lens, a cosmetic or decorative lens, etc.). The one or more layers 2111, 2112, and 2113 may be planar. The imaging system 2100 may also include a reflector 2150 similar to the reflector 2050 described with reference to FIGS. 11A-11B. The imaging system 2100 may also include an imager 2130. The imager 2130 may include a sensor array (similar to the sensor array 2030 described with respect to FIGS. 11A-11B) and wafer-scale imaging optics. For example, the sensor array and wafer-scale imaging optics of the imager 2130 may be integrated within or on the eyepiece 2110. In various implementations, the wafer-scale imaging optics integrated within or on the eyepiece 2110 are not included in a housing other than the eyepiece 2110. For example, a camera need not include a sensor array including wafer-scale optics in a housing disposed above the eyepiece, which may also be included in the housing. Rather, the wafer-scale optics may be included on a layer of the eyepiece and aligned with the sensor array, which may be included on the same or a different layer of the eyepiece, which may be included in the housing.

[0098] The imaging system 2100 may be similar to the imaging system 2000 described with respect to FIGS. 11A-11B. In some such designs, the imaging system 2100 may or may not include a refractive or transmissive diffractive optical element 2040 with an off-axis optical element, as discussed above with respect to FIGS. 11A-11B. In some cases, the wafer-scale imaging optics may include one or more imaging lenses (which may be similar, for example, to the optical elements 2040a, 2040b described with respect to FIGS. 12A-12B) disposed on the same or a different layer as the sensor array. The imaging lenses are aligned with the sensor array so that light from the reflector 2150 can pass through the sensor array to form an image on the sensor array.

[0099] Exemplary Routine for Imaging an Object Figure 14 is a process flow diagram of an exemplary routine for imaging an object (e.g., a user's eye or an object in an environment) using an off-axis camera (or sensor array), such as camera assembly 630 of Figure 6, camera assembly 1030 of Figure 10A, sensor array 2030 of Figures 11A, 11B, or imager 2130 of Figure 13. Routine 1900 describes how light from an object can be directed toward a camera assembly positioned at a distance from or offset to the side of the object (e.g., eye) to image the object as if the camera assembly were directly in front of the object and pointing directly at the object.

[0100] At block 1910, an imaging system configured to receive light from the object and direct the light to a camera assembly is provided. The imaging system may be one or more of the imaging systems described above with respect to Figures 10A, 10B, 11A, 11B, and 13.

[0101] At block 1920, the light is captured by a camera assembly (e.g., camera assembly 630 of FIG. 6, camera assembly 1030 of FIGS. 10A-10B, sensor array 2030 of FIGS. 11A-11B, or imager 2130 of FIG. 13). The camera assembly may be located at the eye, the pupil of the eye, or on the line of sight toward the front of the eye. The camera assembly may be located to the side of the eyepiece (e.g., on the side of the head). The camera assembly may be a forward-facing or rear-facing camera. At block 1930, an image of the object may be generated based on the captured light, as described herein and throughout this disclosure.

[0102] In some embodiments, routine 1900 may include an optional step (not shown) of illuminating the object with light from a light source (e.g., the light source of FIG. 6 or light source 1032 of FIGS. 10A-10B). In some embodiments, the light may include a range of wavelengths including infrared light.

[0103] In some embodiments, the images generated in block 1930 may be processed and analyzed, for example, using image processing techniques. In some implementations, the analyzed images may be used to perform one or more of eye tracking, biometric identification, multi-scopic reconstruction of eye shape, estimation of eye accommodation state, imaging of the retina, iris, or other characteristic patterns of the eye, and assess a physiological state of the user based in part on the analyzed off-axis images, as described above and throughout this disclosure. In some implementations, the analyzed images may be used to track objects in an environment.

[0104] In various embodiments, the routine 1900 may be implemented by a hardware processor (e.g., local processing and data module 140 of FIG. 2) configured to execute instructions stored in memory. In other embodiments, a remote computing device (in network communication with the display device) equipped with computer-executable instructions can cause the display device to implement aspects of the routine 1900. [Example]

[0105] The present disclosure provides various embodiments of head mounted displays, including but not limited to the following examples: Part I 1. A head-mounted display, a frame configured to be supported on a user's head; an eyepiece coupled to the frame and configured to be positioned in front of a user's eye, the eyepiece comprising a plurality of layers; an image injection device configured to provide image content to the eyepiece for viewing by a user; a sensor array integrated in or on the eyepiece; a reflector disposed in or on the eyepiece configured to reflect light received from an object for imaging by the sensor array; and a transmissive diffractive optical element disposed in or on the eyepiece and configured to receive light reflected from the reflector, transmit at least a portion of the light, and diffract it toward the sensor array. 2. The head-mounted display of Example 1, wherein the eyepiece directs light from the image injection device toward the user's eye to present image content to the user. 3. A head-mounted display as described in any of the preceding examples, wherein the plurality of layers comprises one or more waveguides configured to receive light from the image injection device and guide at least a portion of the light therein by total internal reflection to provide image content to a user. 4. A head mounted display according to any preceding embodiment, wherein the plurality of layers comprises a plurality of waveguides, the different waveguides being arranged to provide different color image content. 5. A head-mounted display as described in any of the preceding examples, wherein the multiple layers comprise multiple waveguides, and different waveguides or groups of waveguides are configured to project light into the user's eye to display image content with different amounts of divergence, as if projected from different distances from the user's eye. 6. A head mounted display as described in any preceding example, wherein the plurality of layers comprises one or more of a decorative or cosmetic lens, a front lens, a dimmer, a rear lens, an illumination layer, or a prescription lens configured to provide refractive correction for a user with refractive error. 7. A head-mounted display as described in any of the preceding embodiments, wherein the eyepiece is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 8. A head mounted display according to any preceding embodiment, wherein the plurality of layers comprises at least one transparent glass or plastic layer. 9. A head-mounted display according to any preceding embodiment, wherein the sensor array comprises a plurality of detector pixels formed on at least one of the layers. 10. A head-mounted display as described in any of the preceding embodiments, wherein the sensor array comprises a plurality of detector pixels arranged on at least one layer that is at least partially transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, the transparent portion transmitting light from a portion of the environment in front of the user and the eyepiece to the user's eyes to provide a view of that portion of the environment in front of the user and the eyepiece. 11. A head mounted display according to any preceding embodiment, wherein the sensor array comprises wafer scale optics. 12. A head-mounted display as described in any of the preceding embodiments, wherein the transmissive diffractive optical element comprises at least one diffractive lens aligned with the sensor array, such that light from the reflector passes through the at least one diffractive lens to the sensor array and forms an image on the sensor array. 13. A head-mounted display as described in Example 12, wherein the sensor array is arranged within or on a first layer of the plurality of layers of the eyepiece section, and the at least one diffractive lens is arranged within or on a second, different layer of the plurality of layers of the eyepiece section. 14. A head-mounted display as described in Example 13, wherein the first and second layers each have at least a portion that is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from a portion of the environment in front of the user and the eyepieces to the user's eyes, providing a view of that portion of the environment in front of the user and the eyepieces. 15. A head-mounted display as described in Example 12, wherein the sensor array is arranged within or on a first side of a layer of the plurality of layers of the eyepiece portion, and the at least one diffractive lens is arranged within or on a second opposite side of the layer. 16. A head-mounted display as described in Example 15, wherein the layer in or on which the sensor array and at least one lens are arranged is transparent and has at least a portion that is positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from a portion of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that portion of the environment in front of the user and the eyepiece. 17. A head mounted display according to any preceding embodiment, wherein the reflector comprises a hot mirror. 18. A head-mounted display as described in any of the preceding examples, wherein the reflector is configured to reflect light in a first range of infrared (IR) or near-infrared wavelengths while transmitting light in a second range of visible wavelengths. 19. A head mounted display according to any preceding embodiment, wherein the reflector is formed on at least one of the plurality of layers. 20. A head-mounted display according to any preceding embodiment, wherein the reflector comprises one of the plurality of layers. 21. A head mounted display according to any preceding embodiment, wherein the reflector includes an optical coating. 22. A head-mounted display as described in any of the preceding embodiments, wherein the reflector is arranged on at least one layer, at least a portion of which is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 23. A head mounted display according to any preceding embodiment, wherein the transmissive diffractive optical element is disposed in an optical path between the reflector and the sensor array. 24. A head-mounted display described in any of the preceding embodiments, wherein the reflector is arranged on a first layer of the plurality of layers, the transparent diffractive optical element is arranged on a second layer of the plurality of layers, the sensor array is arranged on a third layer of the plurality of layers, and the second layer is arranged between the first and third layers. 25. A head-mounted display as described in Example 24, wherein at least a portion of the first, second, and third layers are transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portions transmit light from part of the environment in front of the user and the eyepieces to the user's eyes, providing a view of that part of the environment in front of the user and the eyepieces. 26. A head mounted display according to any preceding embodiment, wherein the transmissive diffractive optical element comprises a transmissive diffraction grating. 27. A head mounted display according to any preceding embodiment, wherein the transmissive diffractive optical element comprises a transmissive holographic optical element, a transmissive volume diffractive optical element (OAVDOE), or a transmissive cholesteric liquid crystal grating (OACLCG). 28. A head mounted display according to any preceding embodiment, wherein the transmissive diffractive optical element has optical power. 29. A head-mounted display according to any preceding embodiment, wherein at least a portion of the transmissive diffractive optical element is engraved on one of the layers of the eyepiece. 30. A head-mounted display as described in any of the preceding examples, wherein the sensor array is a forward-facing camera configured to image at least a portion of a target based at least in part on light received from a reflector disposed in front of the sensor array, and wherein at least a portion of the target is disposed behind the sensor array and includes at least one of a user's eye, a portion of the eye, or a portion of tissue surrounding the eye. 31. A head-mounted display as described in any of the preceding examples, further comprising a light source that emits light in a first range of wavelengths toward at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 32. A head-mounted display as described in Example 31, wherein the first range of wavelengths includes at least one light in the infrared (IR) or near-infrared spectrum. 33. A head-mounted display according to any of the preceding examples, wherein the head-mounted display is configured to track the user's gaze based on an image of at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 34. A head-mounted display according to any preceding embodiment, wherein the sensor array is further configured to image objects in an environment in front of the eyepiece. 35. A head-mounted display as described in Example 34, wherein the sensor array is a rear-facing camera configured to image at least a portion of a target based at least in part on light received from the reflector behind the sensor array, and at least a portion of the target is positioned in front of the sensor array in the environment in front of the user. 36. The head-mounted display of any preceding embodiment, wherein the head-mounted display includes eyewear. Part II 1. A head-mounted display, a frame configured to be supported on a user's head; an eyepiece coupled to the frame and configured to be positioned in front of a user's eye, the eyepiece comprising a plurality of layers; an image injection device configured to provide image content to the eyepiece for viewing by a user; a sensor array integrated in or on the eyepiece; a reflector disposed in or on the eyepiece configured to reflect light received from an object for imaging by the sensor array; and a transmissive-refractive optical element disposed in or on the eyepiece, receiving light reflected from the reflector and refracting at least a portion of the light toward the sensor array. 2. The head-mounted display of Example 1, wherein the eyepiece directs light from the image injection device toward the user's eye to present image content to the user. 3. A head-mounted display as described in any of the preceding examples, wherein the plurality of layers comprises one or more waveguides configured to receive light from the image injection device and guide at least a portion of the light therein by total internal reflection to provide image content to a user. 4. A head mounted display according to any preceding embodiment, wherein the plurality of layers comprises a plurality of waveguides, the different waveguides being arranged to provide different color image content. 5. A head-mounted display as described in any of the preceding examples, wherein the multiple layers comprise multiple waveguides, and different waveguides or groups of waveguides are configured to project light into the user's eye to display image content with different amounts of divergence, as if projected from different distances from the user's eye. 6. A head mounted display as described in any preceding example, wherein the plurality of layers comprises one or more of a decorative or cosmetic lens, a front lens, a dimmer, a rear lens, an illumination layer, or a prescription lens configured to provide refractive correction for a user with refractive error. 7. A head-mounted display as described in any of the preceding embodiments, wherein the eyepiece is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 8. A head mounted display according to any preceding embodiment, wherein the plurality of layers comprises at least one transparent glass or plastic layer. 9. A head-mounted display according to any preceding embodiment, wherein the sensor array comprises a plurality of detector pixels formed on at least one of the layers. 10. A head-mounted display as described in any of the preceding embodiments, wherein the sensor array comprises a plurality of detector pixels arranged on at least one layer that is at least partially transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, the transparent portion transmitting light from a portion of the environment in front of the user and the eyepiece to the user's eyes to provide a view of that portion of the environment in front of the user and the eyepiece. 11. A head mounted display according to any preceding embodiment, wherein the sensor array comprises wafer scale optics. 12. A head-mounted display as described in any of the preceding embodiments, wherein the refractive optical element comprises at least one lens aligned with the sensor array such that light from the reflector passes through the at least one lens to the sensor array to form an image on the sensor array. 13. A head-mounted display as described in Example 12, wherein the sensor array is arranged within or on a first layer of the plurality of layers of the eyepiece, and the at least one lens is arranged within or on a second, different layer of the plurality of layers of the eyepiece. 14. A head-mounted display as described in Example 13, wherein the first and second layers each have at least a portion that is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from a portion of the environment in front of the user and the eyepieces to the user's eyes, providing a view of that portion of the environment in front of the user and the eyepieces. 15. A head-mounted display as described in Example 12, wherein the sensor array is arranged within or on a first side of a layer of the plurality of layers of the eyepiece portion, and the at least one lens is arranged within or on a second opposite side of the layer. 16. A head-mounted display as described in Example 15, wherein the layer in or on which the sensor array and at least one lens are arranged is transparent and has at least a portion that is positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from a portion of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that portion of the environment in front of the user and the eyepiece. 17. A head mounted display according to any one of Examples 12 to 16, wherein at least one lens comprises a diffractive optical element. 18. The head-mounted display of any one of Examples 12 to 16, wherein the at least one lens includes at least one refractive lens. 19. The head-mounted display of any one of Examples 12 to 16, wherein at least one lens includes a wafer-scale optical system. 20. A head mounted display according to any preceding embodiment, wherein the reflector comprises a hot mirror. 21. A head-mounted display as described in any of the preceding examples, wherein the reflector is configured to reflect light in a first range of infrared (IR) or near-infrared wavelengths while transmitting light in a second range of visible wavelengths. 22. A head-mounted display according to any preceding embodiment, wherein the reflector is formed on at least one of the plurality of layers. 23. A head-mounted display according to any preceding embodiment, wherein the reflector comprises one of the plurality of layers. 24. A head mounted display according to any preceding embodiment, wherein the reflector includes an optical coating. 25. A head-mounted display as described in any of the preceding examples, wherein the reflector is arranged on at least one layer, at least a portion of which is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 26. A head mounted display according to any preceding embodiment, wherein the refractive optical element is disposed in an optical path between the reflector and the sensor array. 27. A head-mounted display as described in any of the preceding embodiments, wherein the reflector is arranged on a first layer of the plurality of layers, the refractive optical element is arranged on a second layer of the plurality of layers, the sensor array is arranged on a third layer of the plurality of layers, and the second layer is arranged between the first and third layers. 28. A head-mounted display as described in Example 27, wherein at least a portion of the first, second, and third layers are transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portions transmit light from part of the environment in front of the user and the eyepieces to the user's eyes, providing a view of that part of the environment in front of the user and the eyepieces. 29. A head mounted display according to any preceding embodiment, wherein the refractive optical element comprises an off-axis optical element. 30. A head mounted display according to any preceding embodiment, wherein the refractive optical element has refractive power. 31. A head mounted display according to any preceding embodiment, wherein the refractive optical element comprises an off-axis lens. 32. A head-mounted display according to any preceding embodiment, wherein at least a portion of the refractive optical element is engraved on one of the layers of the eyepiece. 33. A head-mounted display as described in any of the preceding examples, wherein the sensor array is a forward-facing camera configured to image at least a portion of a target based at least in part on light received from a reflector disposed in front of the sensor array, and wherein at least a portion of the target is disposed behind the sensor array and includes a user's eye, a portion of the eye, or a portion of tissue surrounding the eye. 34. A head-mounted display as described in any of the preceding examples, further comprising a light source that emits light in a first range of wavelengths toward at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 35. A head-mounted display as described in the embodiment Error! Reference source not found., wherein the first range of wavelengths includes at least one light in the infrared (IR) or near-infrared spectrum. 36. A head-mounted display according to any of the preceding examples, wherein the head-mounted display is configured to track the user's gaze based on an image of at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 37. A head-mounted display according to any preceding embodiment, wherein the sensor array is further configured to image objects in an environment in front of the eyepiece. 38. A head-mounted display as described in Example 37, wherein the sensor array is a rear-facing camera configured to image at least a portion of a target based at least in part on light received from the reflector behind the sensor array, and at least a portion of the target is positioned in front of the sensor array in the environment in front of the user. 39. The head-mounted display of any preceding embodiment, wherein the head-mounted display includes eyewear. Part III 1. A head-mounted display, a frame configured to be supported on a user's head; an eyepiece coupled to the frame and configured to be positioned in front of a user's eye, the eyepiece comprising a plurality of layers; an image injection device configured to provide image content to the eyepiece for viewing by a user; a sensor array integrated in or on the eyepiece; a reflector disposed in or on the eyepiece configured to reflect light received from an object for imaging by the sensor array; and an off-axis optical element disposed in or on the eyepiece and configured to receive light reflected from the reflector and direct at least a portion of the light toward the sensor array. 2. The head-mounted display of Example 1, wherein the eyepiece directs light from the image injection device toward the user's eye to present image content to the user. 3. A head-mounted display as described in any of the preceding examples, wherein the plurality of layers comprises one or more waveguides configured to receive light from the image injection device and guide at least a portion of the light therein by total internal reflection to provide image content to a user. 4. A head mounted display according to any preceding embodiment, wherein the plurality of layers comprises a plurality of waveguides, the different waveguides being arranged to provide different color image content. 5. A head-mounted display as described in any of the preceding examples, wherein the multiple layers comprise multiple waveguides, and different waveguides or groups of waveguides are configured to project light into the user's eye to display image content with different amounts of divergence, as if projected from different distances from the user's eye. 6. A head mounted display as described in any preceding example, wherein the plurality of layers comprises one or more of a decorative or cosmetic lens, a front lens, a dimmer, a rear lens, an illumination layer, or a prescription lens configured to provide refractive correction for a user with refractive error. 7. A head-mounted display as described in any of the preceding embodiments, wherein the eyepiece is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 8. A head mounted display according to any preceding embodiment, wherein the plurality of layers comprises at least one transparent glass or plastic layer. 9. A head-mounted display according to any preceding embodiment, wherein the sensor array comprises a plurality of detector pixels formed on at least one of the layers. 10. A head-mounted display as described in any of the preceding embodiments, wherein the sensor array comprises a plurality of detector pixels arranged on at least one layer that is at least partially transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, the transparent portion transmitting light from a portion of the environment in front of the user and the eyepiece to the user's eyes to provide a view of that portion of the environment in front of the user and the eyepiece. 11. A head mounted display according to any preceding embodiment, wherein the sensor array comprises wafer scale optics. 12. A head-mounted display as described in any of the preceding embodiments, wherein the off-axis optical element comprises at least one lens aligned with the sensor array such that light from the reflector passes through the at least one lens to the sensor array and forms an image on the sensor array. 13. A head-mounted display as described in Example 12, wherein the sensor array is arranged within or on a first layer of the plurality of layers of the eyepiece, and the at least one lens is arranged within or on a second, different layer of the plurality of layers of the eyepiece. 14. A head-mounted display as described in Example 13, wherein the first and second layers each have at least a portion that is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from a portion of the environment in front of the user and the eyepieces to the user's eyes, providing a view of that portion of the environment in front of the user and the eyepieces. 15. A head-mounted display as described in Example 12, wherein the sensor array is arranged within or on a first side of a layer of the plurality of layers of the eyepiece portion, and the at least one lens is arranged within or on a second opposite side of the layer. 16. A head-mounted display as described in Example 15, wherein the layer in or on which the sensor array and at least one lens are arranged is transparent and has at least a portion that is positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from a portion of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that portion of the environment in front of the user and the eyepiece. 17. A head mounted display according to any one of Examples 12 to 16, wherein at least one lens comprises a diffractive optical element. 18. The head-mounted display of any one of Examples 12 to 16, wherein the at least one lens includes at least one refractive lens. 19. The head-mounted display of any one of Examples 12 to 16, wherein at least one lens includes a wafer-scale optical system. 20. A head mounted display according to any preceding embodiment, wherein the reflector comprises a hot mirror. 21. A head-mounted display as described in any of the preceding examples, wherein the reflector is configured to reflect light in a first range of infrared (IR) or near-infrared wavelengths while transmitting light in a second range of visible wavelengths. 22. A head-mounted display according to any preceding embodiment, wherein the reflector is formed on at least one of the plurality of layers. 23. A head-mounted display according to any preceding embodiment, wherein the reflector comprises one of the plurality of layers. 24. A head mounted display according to any preceding embodiment, wherein the reflector includes an optical coating. 25. A head-mounted display as described in any of the preceding examples, wherein the reflector is arranged on at least one layer, at least a portion of which is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 26. A head mounted display according to any preceding embodiment, wherein the off-axis optical element is disposed in an optical path between the reflector and the sensor array. 27. A head-mounted display described in any of the preceding examples, wherein the reflector is arranged on a first layer of the plurality of layers, the off-axis optical element is arranged on a second layer of the plurality of layers, the sensor array is arranged on a third layer of the plurality of layers, and the second layer is arranged between the first and third layers. 28. A head-mounted display as described in Example 27, wherein at least a portion of the first, second, and third layers are transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portions transmit light from part of the environment in front of the user and the eyepieces to the user's eyes, providing a view of that part of the environment in front of the user and the eyepieces. 29. A head mounted display according to any preceding embodiment, wherein the off-axis optical element comprises a diffractive optical element (DOE) or a diffraction grating. 30. A head-mounted display according to any preceding embodiment, wherein the off-axis optical element comprises an off-axis diffractive optical element (DOE), an off-axis diffraction grating, an off-axis holographic mirror (OAHM), an off-axis volume diffractive optical element (OAVDOE), or an off-axis cholesteric liquid crystal grating (OACLCG). 31. A head mounted display according to any preceding embodiment, wherein the off-axis optical element has optical power. 32. A head-mounted display according to any preceding embodiment, wherein at least a portion of the off-axis optical element is engraved on one of the layers of the eyepiece. 33. A head-mounted display as described in any of the preceding examples, wherein the sensor array is a forward-facing camera configured to image at least a portion of a target based at least in part on light received from a reflector disposed in front of the sensor array, and wherein at least a portion of the target is disposed behind the sensor array and includes at least one of a user's eye, a portion of the eye, or a portion of tissue surrounding the eye. 34. A head-mounted display as described in any of the preceding examples, further comprising a light source that emits light in a first range of wavelengths toward at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 35. A head-mounted display as described in the embodiment Error! Reference source not found., wherein the first range of wavelengths includes at least one light in the infrared (IR) or near-infrared spectrum. 36. A head-mounted display according to any of the preceding examples, wherein the head-mounted display is configured to track the user's gaze based on an image of at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 37. A head-mounted display according to any preceding embodiment, wherein the sensor array is further configured to image objects in an environment in front of the eyepiece. 38. A head-mounted display as described in Example 37, wherein the sensor array is a rear-facing camera configured to image at least a portion of a target based at least in part on light received from the reflector behind the sensor array, and at least a portion of the target is positioned in front of the sensor array in the environment in front of the user. 39. The head-mounted display of any preceding embodiment, wherein the head-mounted display includes eyewear. 40. A head-mounted display, a frame configured to be supported on a user's head; an eyepiece coupled to the frame and configured to be positioned in front of a user's eye, the eyepiece comprising a plurality of layers; an image injection device configured to provide image content to the eyepiece for viewing by a user; a sensor array integrated in or on a first one of the layers of the eyepiece; a head-mounted display comprising at least one imaging lens aligned with the sensor array to form an image on the sensor array, the at least one imaging lens being arranged in or on a second layer of the plurality of layers of the eyepiece. 41. A head-mounted display as described in Example 40, wherein the eyepiece directs light from the image injection device toward the user's eyes to present image content to the user. 42. A head-mounted display described in any of Examples 40 to 41, wherein the plurality of layers comprises one or more waveguides configured to receive light from the image injection device and guide at least a portion of the light therein by total internal reflection, thereby providing image content to a user. 43. A head-mounted display described in any of Examples 40 to 42, wherein the multiple layers comprise multiple waveguides, and different waveguides are arranged to provide different color image content. 44. A head-mounted display described in any of Examples 40 to 43, wherein multiple layers have multiple waveguides, and different waveguides or groups of waveguides are configured to project light into the user's eye to display image content with different amounts of divergence, as if projected from different distances from the user's eye. 45. A head-mounted display described in any of Examples 40 to 44, wherein the multiple layers include one or more of a cosmetic window, a front lens, a dimmer, a rear lens, an illumination layer, or a prescription lens configured to provide refractive correction for a user with refractive error. 46. ​​A head-mounted display described in any of Examples 40 to 45, wherein the eyepiece is transparent and positioned in front of the user's eyes when the user wears the head-mounted display, and the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 47. A head-mounted display according to any one of Examples 40 to 46, wherein the plurality of layers comprises at least one transparent glass or plastic layer. 48. A head-mounted display described in any of Examples 40 to 47, wherein the sensor array comprises a plurality of detector pixels formed on at least one of the layers. 49. A head-mounted display described in any of Examples 40 to 48, wherein the sensor array comprises a plurality of detector pixels arranged on at least one layer that is at least partially transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, the transparent portion transmitting light from part of the environment in front of the user and the eyepiece to the user's eyes to provide a view of that part of the environment in front of the user and the eyepiece. 50. A head-mounted display described in any of Examples 40 to 49, further comprising a reflector, wherein the at least one lens is aligned with the sensor array so that light from the reflector passes through the at least one lens to the sensor array and forms an image on the sensor array. 51. A head-mounted display described in any of Examples 40 to 50, wherein the first and second layers are each transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, and the transparent portions have at least a portion that transmits light from a portion of the environment in front of the user and the eyepieces to the user's eyes, thereby providing a view of that portion of the environment in front of the user and the eyepieces. 52. A head-mounted display according to any one of Examples 40 to 51, wherein the at least one lens comprises a diffractive optical element. 53. A head-mounted display according to any one of Examples 40 to 51, wherein the at least one lens includes at least one diffractive lens. 54. A head-mounted display according to any one of embodiments 40 to 53, wherein the at least one lens includes a wafer-scale optical system. 55. A head-mounted display described in any of Examples 40 to 54, further comprising a reflector arranged within or on the eyepiece, the reflector configured to reflect light received from a subject for imaging by the sensor array. 56. A head-mounted display as described in Example 55, wherein the reflector includes a hot mirror. 57. A head-mounted display as described in Example 55 or 56, wherein the reflector is configured to reflect light in a first range of infrared (IR) or near-infrared wavelengths while transmitting light in a second range of visible wavelengths. 58. A head-mounted display according to any one of Examples 55 to 57, wherein the reflector is formed on at least one of the plurality of layers. 59. A head-mounted display according to any one of embodiments 55 to 58, wherein the reflector includes one of the plurality of layers. 60. The head-mounted display of any one of Examples 55 to 59, wherein the reflector includes an optical coating. 61. A head-mounted display described in any of Examples 55 to 60, wherein the reflector is arranged on at least one layer, at least a portion of which is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, so that the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 62. A head-mounted display described in any of Examples 50 to 61, further comprising an off-axis optical element arranged within or on the eyepiece, the off-axis optical element configured to receive light reflected from the reflector and direct at least a portion of the light toward at least one imaging lens. 63. A head-mounted display as described in Example 62, wherein the off-axis optical element is arranged in the optical path between the reflector and the at least one imaging lens. 64. A head-mounted display described in any of Examples 50 to 63, wherein the reflector is arranged on a third layer of the plurality of layers, and the second layer is arranged between the first and third layers. 65. A head-mounted display as described in Example 64, wherein at least a portion of the first, second, and third layers are transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portions transmit light from a portion of the environment in front of the user and the eyepieces to the user's eyes, providing a view of that portion of the environment in front of the user and the eyepieces. 66. A head-mounted display according to any one of Examples 62 to 65, wherein the off-axis optical element comprises a diffractive optical element (DOE) or a diffraction grating. 67. A head-mounted display described in any of Examples 62 to 66, wherein the off-axis optical element comprises an off-axis diffractive optical element (DOE), an off-axis diffraction grating, an off-axis holographic mirror (OAHM), an off-axis volume diffractive optical element (OAVDOE), or an off-axis cholesteric liquid crystal diffraction grating (OACLCG). 68. A head-mounted display according to any one of Examples 62 to 67, wherein the off-axis optical element has refractive power. 69. A head-mounted display described in any of Examples 62 to 68, wherein at least a portion of the off-axis optical element is engraved on one of the multiple layers of the eyepiece. 70. A head-mounted display described in any of Examples 50 to 69, wherein the sensor array is a forward-facing camera configured to image at least a portion of a target based at least in part on light received from a reflector positioned in front of the sensor array, and at least a portion of the target is positioned behind the sensor array and includes at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 71. A head-mounted display described in any of Examples 40 to 70, further comprising a light source that emits light in a first range of wavelengths toward at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 72. A head-mounted display as described in Example 71, wherein the first range of wavelengths includes at least one light in the infrared (IR) or near-infrared spectrum. 73. A head-mounted display described in any of Examples 40 to 72, wherein the head-mounted display is configured to track the user's gaze based on an image of at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 74. A head-mounted display described in any of Examples 40 to 69, wherein the sensor array is configured to capture images of objects in the environment in front of the eyepiece. 75. A head-mounted display described in any of Examples 50 to 69 and 74, wherein the sensor array is a rear-facing camera configured to image at least a portion of a target based at least in part on light received from a reflector behind the sensor array, and at least a portion of the target is positioned in front of the sensor array in an environment in front of the user. 76. A head-mounted display according to any one of Examples 40 to 75, wherein the head-mounted display includes eyewear. 77. A head-mounted display, a frame configured to be supported on a user's head; an eyepiece coupled to the frame and configured to be positioned in front of a user's eye, the eyepiece comprising a plurality of layers; an image injection device configured to provide image content to the eyepiece for viewing by a user; a sensor array integrated in or on a first side of one of the layers of the eyepiece; a head-mounted display comprising: at least one imaging lens aligned with the sensor array to form an image on the sensor array, and arranged within or on a second side of one of the multiple layers of the eyepiece on which the sensor array is arranged. 78. A head-mounted display as described in Example 77, wherein the eyepiece directs light from the image injection device toward the user's eyes to present image content to the user. 79. A head-mounted display described in any of Examples 77-78, wherein the plurality of layers comprises one or more waveguides configured to receive light from the image injection device and guide at least a portion of the light therein by total internal reflection, thereby providing image content to a user. 80. A head-mounted display described in any of Examples 77 to 79, wherein multiple layers comprise multiple waveguides, and different waveguides are arranged to provide different color image content. 81. A head-mounted display described in any of Examples 77 to 80, wherein multiple layers have multiple waveguides, and different waveguides or groups of waveguides are configured to project light into the user's eye to display image content with different amounts of divergence, as if projected from different distances from the user's eye. 82. A head-mounted display described in any of Examples 77 to 81, wherein the multiple layers include one or more of a cosmetic window, a front lens, a dimmer, a rear lens, an illumination layer, or a prescription lens configured to provide refractive correction for a user with refractive error. 83. A head-mounted display described in any of Examples 77 to 82, wherein the eyepiece is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, and the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 84. A head-mounted display described in any of Examples 77 to 83, wherein the plurality of layers comprises at least one transparent glass or plastic layer. 85. A head-mounted display described in any of Examples 77 to 84, wherein the sensor array comprises a plurality of detector pixels formed on at least one of the layers. 86. A head-mounted display described in any of Examples 77 to 85, wherein the sensor array comprises a plurality of detector pixels arranged on at least one layer that is at least partially transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, the transparent portion transmitting light from part of the environment in front of the user and the eyepiece to the user's eyes to provide a view of that part of the environment in front of the user and the eyepiece. 87. A head-mounted display described in any of Examples 77 to 86, further comprising a reflector, wherein the at least one lens is aligned with the sensor array so that light from the reflector passes through the at least one lens to the sensor array and forms an image on the sensor array. 88. A head-mounted display described in any of Examples 77 to 87, wherein the layer in which the sensor array and at least one lens are arranged is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, and the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 89. A head-mounted display described in any of Examples 77 to 88, wherein at least one lens comprises a diffractive optical element. 90. A head-mounted display described in any one of Examples 77 to 88, wherein the at least one lens includes at least one diffractive lens. 91. A head-mounted display according to any one of Examples 77 to 90, wherein the at least one lens includes a wafer-scale optical system. 92. A head-mounted display described in any of Examples 77 to 91, further comprising a reflector arranged within or on the eyepiece, the reflector configured to reflect light received from the subject for imaging by the sensor array. 93. A head-mounted display as described in Example 92, wherein the reflector includes a hot mirror. 94. A head-mounted display as described in Example 92 or 93, wherein the reflector is configured to reflect light in a first range of infrared (IR) or near-infrared wavelengths while transmitting light in a second range of visible wavelengths. 95. A head-mounted display according to any one of Examples 92 to 94, wherein the reflector is formed on at least one of the plurality of layers. 96. A head-mounted display according to any one of Examples 92 to 95, wherein the reflector includes one of the plurality of layers. 97. The head-mounted display of any one of Examples 92 to 96, wherein the reflector includes an optical coating. 98. A head-mounted display described in any of Examples 92 to 97, wherein the reflector is arranged on at least one layer, at least a portion of which is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, so that the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 99. A head-mounted display described in any of Examples 87 to 98, further comprising an off-axis optical element arranged within or on the eyepiece, the off-axis optical element configured to receive light reflected from the reflector and direct at least a portion of the light toward at least one imaging lens. 100. A head-mounted display as described in Example 99, wherein the off-axis optical element is arranged in the optical path between the reflector and the at least one imaging lens. 101. A head-mounted display as described in Example 99 or 100, wherein the reflector is arranged on a first layer of the plurality of layers, the off-axis optical element is arranged on a second layer of the plurality of layers, the sensor array is arranged on a third layer of the plurality of layers, and the second layer is arranged between the first and third layers. 102. A head-mounted display as described in Example 101, wherein at least a portion of the first, second, and third layers are transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portions transmit light from a portion of the environment in front of the user and the eyepieces to the user's eyes, providing a view of that portion of the environment in front of the user and the eyepieces. 103. The head mounted display of any one of Examples 99 to 102, wherein the off-axis optical element comprises a diffractive optical element (DOE) or a diffraction grating. 104. A head-mounted display described in any of Examples 99 to 103, wherein the off-axis optical element comprises an off-axis diffractive optical element (DOE), an off-axis diffraction grating, an off-axis holographic mirror (OAHM), an off-axis volume diffractive optical element (OAVDOE), or an off-axis cholesteric liquid crystal diffraction grating (OACLCG). 105. The head-mounted display according to any one of Examples 99 to 104, wherein the off-axis optical element has refractive power. 106. A head-mounted display described in any of Examples 99 to 105, wherein at least a portion of the off-axis optical element is engraved on one of the multiple layers of the eyepiece. 107. A head-mounted display described in any of Examples 87 to 106, wherein the sensor array is a forward-facing camera configured to image at least a portion of a target based at least in part on light received from a reflector positioned in front of the sensor array, and at least a portion of the target is positioned behind the sensor array and includes at least one of the user's eye, a portion of the eye, or a portion of tissue surrounding the eye. 108. A head-mounted display described in any of Examples 77 to 107, further comprising a light source that emits light in a first range of wavelengths toward at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 109. A head-mounted display as described in Example 108, wherein the first range of wavelengths includes at least one light in the infrared (IR) or near-infrared spectrum. 110. A head-mounted display described in any of Examples 77 to 109, wherein the head-mounted display is configured to track the user's gaze based on an image of at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 111. A head-mounted display described in any of Examples 77 to 106, wherein the sensor array is configured to capture images of objects in the environment in front of the eyepiece. 112. A head-mounted display described in any of Examples 87 to 106 and 111, wherein the sensor array is a rear-facing camera configured to image at least a portion of a target based at least in part on light received from a reflector behind the sensor array, and at least a portion of the target is positioned in front of the sensor array in an environment in front of the user. 113. The head-mounted display of any one of Examples 77 to 112, wherein the head-mounted display includes eyewear. 114. A head-mounted display, a frame configured to be supported on a user's head; an eyepiece coupled to the frame and configured to be placed in front of a user's eye, the eyepiece including a plurality of layers placed in front of the eye; an image injection device configured to provide image content to the eyepiece for viewing by a user; an imager comprising a sensor array and wafer-scale imaging optics integrated in or on the eyepiece; a reflector disposed in or on the eyepiece and configured to reflect light received from an object for imaging by the imager; A head-mounted display, wherein the wafer-scale imaging optical system is not contained within a housing other than the eyepiece. 115. A head-mounted display as described in Example 114, wherein the plurality of layers of the eyepiece section comprises a plurality of planar layers, and the imager is inclined with respect to the normal to the planar layers. 116. A head-mounted display as described in embodiment 114 or 115, wherein the imager is inclined toward the reflector. 117. The head-mounted display according to any one of embodiments 114 to 116, wherein the imager is tilted to face the reflector and receives light therefrom. 118. The head-mounted display according to any one of Examples 114 to 117, wherein the imager is inclined with respect to the forward direction. 119. The head-mounted display of any one of Examples 114 to 118, wherein the eyepiece directs light from the image injection device toward the user's eye to present image content to the user. 120. A head-mounted display described in any of Examples 114 to 119, wherein the plurality of layers comprises one or more waveguides configured to receive light from the image injection device and guide at least a portion of the light therein by total internal reflection, thereby providing image content to a user. 121. A head-mounted display described in any of Examples 114 to 120, wherein the multiple layers comprise multiple waveguides, and different waveguides are arranged to provide different color image content. 122. A head-mounted display as described in any of Examples 114 to 121, wherein multiple layers have multiple waveguides, and different waveguides or groups of waveguides are configured to project light into the user's eye to display image content with different amounts of divergence, as if projected from different distances from the user's eye. 123. A head-mounted display described in any of Examples 114 to 122, wherein the multiple layers include one or more of a cosmetic window, a front lens, a dimmer, a rear lens, an illumination layer, or a prescription lens configured to provide refractive correction for a user with refractive error. 124. A head-mounted display described in any of Examples 114 to 123, wherein the eyepiece is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, and the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 125. A head-mounted display according to any one of Examples 114 to 124, wherein the plurality of layers comprises at least one transparent glass or plastic layer. 126. A head-mounted display described in any of Examples 114 to 125, wherein the sensor array comprises a plurality of detector pixels formed on at least one of the layers. 127. A head-mounted display described in any of Examples 114 to 126, wherein the sensor array comprises a plurality of detector pixels arranged on at least one layer that is at least partially transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, the transparent portion transmitting light from a portion of the environment in front of the user and the eyepiece to the user's eyes to provide a view of that portion of the environment in front of the user and the eyepiece. 128. A head-mounted display described in any of Examples 114 to 127, wherein the wafer-scale imaging optical system includes at least one lens aligned with the sensor array, such that light from the reflector passes through the at least one lens to the sensor array and forms an image on the sensor array. 129. A head-mounted display as described in Example 128, wherein the sensor array is arranged within or on a first layer of the plurality of layers of the eyepiece, and the at least one lens is arranged within or on a second, different layer of the plurality of layers of the eyepiece. 130. A head-mounted display as described in Example 129, wherein the first and second layers each have at least a portion that is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portion transmits light from a portion of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that portion of the environment in front of the user and the eyepiece. 131. A head-mounted display as described in Example 128, wherein the sensor array is arranged within or on a first side of a layer of the plurality of layers of the eyepiece portion, and the at least one lens is arranged within or on a second opposite side of the layer. 132. A head-mounted display as described in Example 131, wherein the layer in which the sensor array and at least one lens are arranged or on which the layer is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, the transparent portion transmitting light from a portion of the environment in front of the user and the eyepiece to the user's eyes to provide a view of that portion of the environment in front of the user and the eyepiece. 133. A head-mounted display according to any one of Examples 128 to 132, wherein at least one lens comprises a diffractive optical element. 134. A head-mounted display according to any one of Examples 128 to 132, wherein at least one lens comprises at least one refractive lens. 135. A head-mounted display described in any of Examples 128 to 134, wherein at least a portion of the at least one lens is engraved on one of the layers of the eyepiece. 136. The head-mounted display according to any one of embodiments 114 to 135, wherein the reflector includes a hot mirror. 137. A head-mounted display described in any of Examples 114 to 136, wherein the reflector is configured to reflect light in a first range of infrared (IR) or near-infrared wavelengths while transmitting light in a second range of visible wavelengths. 138. A head-mounted display according to any one of Examples 114 to 137, wherein the reflector is formed on at least one of the plurality of layers. 139. The head-mounted display of any one of Examples 114 to 138, wherein the reflector includes one of the plurality of layers. 140. The head-mounted display of any one of Examples 114 to 139, wherein the reflector includes an optical coating. 141. A head-mounted display described in any of Examples 114 to 140, wherein the reflector is arranged on at least one layer, at least a portion of which is transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, so that the transparent portion transmits light from part of the environment in front of the user and the eyepiece to the user's eyes, providing a view of that part of the environment in front of the user and the eyepiece. 142. A head-mounted display described in any of Examples 114 to 141, further comprising an off-axis optical element arranged within or on the eyepiece, the off-axis optical element configured to receive light reflected from the reflector and direct at least a portion of the light toward the imager. 143. A head-mounted display as described in Example 142, wherein the off-axis optical element is arranged in the optical path between the reflector and the imager. 144. A head-mounted display described in any of Examples 142 to 143, wherein the reflector is arranged on a first layer of the plurality of layers, the off-axis optical element is arranged on a second layer of the plurality of layers, the sensor array is arranged on a third layer of the plurality of layers, and the second layer is arranged between the first and third layers. 145. A head-mounted display as described in Example 144, wherein at least a portion of the first, second, and third layers are transparent and positioned in front of the user's eyes when the user is wearing the head-mounted display, such that the transparent portions transmit light from a portion of the environment in front of the user and the eyepieces to the user's eyes, providing a view of that portion of the environment in front of the user and the eyepieces. 146. The head mounted display of any one of Examples 142 to 145, wherein the off-axis optical element comprises a diffractive optical element (DOE) or a diffraction grating. 147. A head-mounted display described in any of Examples 142 to 146, wherein the off-axis optical element comprises an off-axis diffractive optical element (DOE), an off-axis diffraction grating, an off-axis holographic mirror (OAHM), an off-axis volume diffractive optical element (OAVDOE), or an off-axis cholesteric liquid crystal diffraction grating (OACLCG). 148. The head-mounted display according to any one of Examples 142 to 147, wherein the off-axis optical element has refractive power. 149. A head-mounted display described in any of Examples 142 to 148, wherein at least a portion of the off-axis optical element is engraved on one of the multiple layers of the eyepiece. 150. A head-mounted display described in any of Examples 114 to 149, wherein the imager is a forward-facing camera configured to image at least a portion of the object based at least in part on light received from the reflector, the imager being positioned in front of the imager, and at least a portion of the object being positioned behind the imager, and including at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 151. A head-mounted display described in any of Examples 114 to 150, further comprising a light source that emits light in a first range of wavelengths toward at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 152. A head-mounted display as described in Example 151, wherein the first range of wavelengths includes at least one light in the infrared (IR) or near-infrared spectrum. 153. A head-mounted display described in any of Examples 114 to 152, wherein the head-mounted display is configured to track the user's gaze based on an image of at least one of the user's eye, a portion of the eye, or a portion of the tissue surrounding the eye. 154. A head-mounted display described in any of Examples 114 to 149, wherein the imager is configured to capture an image of an object in the environment in front of the eyepiece. 155. A head-mounted display as described in Example 154, wherein the imager is rear-facing and configured to image at least a portion of a target based at least in part on light received from the reflector, the imager being behind the imager, and at least a portion of the target being positioned in front of the imager in an environment in front of the user. 156. The head-mounted display of any one of Examples 114 to 155, wherein the head-mounted display includes eyewear.

[0106] Additional Considerations In the above-described embodiments, the optical arrangement is described in the context of an imaging display system, and more specifically, an augmented reality display system. However, it will be understood that the principles and advantages of the optical arrangement may be used in other head-mounted displays, optical systems, devices, or methods. In the above, it will be recognized that any feature of any one of the embodiments may be combined with and / or substituted for any other feature of any other one of the embodiments.

[0107] Unless the context clearly requires otherwise, throughout the specification and claims, words such as "comprise," "comprising," "include," "including," "have," and "having" are to be construed in an inclusive sense, rather than an exclusive or exhaustive sense. That is, they mean "including, but not limited to." The term "coupled," as generally used herein, refers to two or more elements that may be directly connected or that may be connected via one or more intermediate elements. Similarly, the word "connected," as generally used herein, refers to two or more elements that may be directly connected or that may be connected via one or more intermediate elements. Depending on the context, "coupled" or "connected" may refer to an optical coupling or optical connection, such as when light is coupled or connected from one optical element to another. Furthermore, the words "herein," "above," "below," "infra," "supra," and similar words, when used in this application, refer to this application as a whole, rather than to particular portions of this application. Where the context permits, words in the above Detailed Description using the singular or plural may also include the plural or singular, respectively. The word "or," when referring to a list of two or more items, is an inclusive (not exclusive) "or," and "or" includes all of the following interpretations of the word: any of the items in the list, all of the items in the list, and any combination of one or more of the items in the list, but not to the exclusion of other items added to the list. Furthermore, as used in this application and the appended claims, "a," "an," and "the" shall be construed to mean "one or more" or "at least one," unless expressly stated otherwise.

[0108] As used herein, a phrase referring to "at least one" of a list of items refers to any combination of those items, including single members. By way of example, "at least one of: A, B, or C" is intended to cover: A, B, C, A and B, A and C, B and C, and A, B, and C. Unless otherwise specified, conjunctive language such as the phrase "at least one of X, Y, and Z" is generally understood differently in the context in which it is used to convey that an item, term, etc. may be at least one of X, Y, or Z. Thus, such conjunctive language is generally not intended to suggest that a particular embodiment requires that at least one of X, at least one of Y, and at least one of Z, respectively, be present.

[0109] Conditional terms used herein, such as "can," "could," "might," or "may," "e.g.," "for example," "such as," and the like, unless expressly stated otherwise or understood otherwise within the context in which they are used, are generally intended to convey that certain embodiments include certain features, elements, and / or conditions, while other embodiments do not. Thus, such conditional terms are generally not intended to imply that features, elements, and / or conditions are in any way required by one or more embodiments, or that these features, elements, and / or conditions should be included or implemented in any particular embodiment.

[0110] Although specific embodiments have been described, these embodiments are presented by way of example only and are not intended to limit the scope of the present disclosure. Indeed, the novel apparatus, methods, and systems described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions, and changes in the form of the methods and systems described herein may be made without departing from the spirit of the present disclosure. For example, while blocks are presented in a given arrangement, alternative embodiments may perform similar functions with different components and / or circuit topologies, and some blocks may be deleted, moved, added, subdivided, combined, and / or modified. Each of these blocks may be implemented in a variety of different ways. Any suitable combination of elements and acts of the various embodiments described above may be made to provide further embodiments. The various features and processes described above may be implemented independently of each other or may be combined in various ways. No element or combination of elements is required or essential in all embodiments. All suitable combinations and subcombinations of features of the present disclosure are intended to be within the scope of the present disclosure.

Claims

[Claim 1] The invention described in this specification.

Images