Diffractive optical element with refractive power

Abstract

To provide a diffractive optical element having refractive power.SOLUTION: A head-mounted display system can include a camera, at least one waveguide, at least one coupling optical element configured such that light is coupled to the waveguide and guided in the waveguide, and at least one external coupling element. At least one external coupling element can be configured to couple the light guided within the waveguide out of the waveguide and direct the light to the camera. At least one coupling element may comprise a diffractive optical element having refractive power.SELECTED DRAWING: Figure 26

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of priority of U.S. Provisional Application No. 62 / 737,042, filed on September 26, 2018, entitled "DIFFRACTIVE OPTICAL ELEMENTS WITH OPTICAL POWER", which is hereby incorporated by reference in its entirety under 35 U.S.C. § 119(e).

[0002] The present disclosure relates to optical devices, including augmented reality imaging and visualization systems.

Background Art

[0003] Modern computing and display technologies have facilitated the development of systems for so - called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that appears or is perceived to be real. Virtual reality, i.e., "VR" scenarios, typically involve the presentation of digital or virtual image information without transparency to other actual real - world visual inputs, and augmented reality, i.e., "AR" scenarios, typically involve the presentation of digital or virtual image information as an augmentation to the visualization of the actual world around the user. Mixed reality, i.e., "MR" scenarios, are a type of AR scenario that typically involves virtual objects integrated into and responsive to the natural world. For example, an MR scenario may include AR image content that is perceived to be blocked by or otherwise interact with objects within the real world.

[0004] Referring to FIG. 1, an augmented reality scene 10 is depicted. To a user of AR technology, a real-world park-like setting 20 featuring people, trees, buildings in the background, and a concrete platform 30 can be seen. The user also perceives as "visible" a robot image 40 standing on the real-world platform 30 and "virtual content" such as a flying comic-like avatar character 50 that appears anthropomorphic like a bumblebee. These elements 50, 40 are "virtual" in that they do not exist within the real world. The human visual perception system is complex, and it is difficult to produce AR technology that facilitates a comfortable, natural, and rich presentation of virtual image elements among other virtual or real-world image elements.

[0005] The systems and methods disclosed herein address various challenges related to AR or VR technology.

[0006] A polarizing beam splitter may be used in a display system to direct polarized light to a light modulator and then direct the light to a viewer. Generally, there is a continuing need to reduce the size of a display system, and as a result, there is also a need to reduce the size of components of a display system that include components that utilize a polarizing beam splitter. SUMMARY OF THE INVENTION MEANS FOR SOLVING THE PROBLEM

[0007] The various implementations described herein include a display system configured to provide illumination and / or image projection to an eye. Additionally, or alternatively, the display system can image the eye and / or the environment.

[0008] In some embodiments, the head-mounted display system is configured to project light onto the user's eyes and display augmented reality image content within the user's field of view. The head-mounted display system can include a frame configured to be supported on the user's head. The display system can also include an image projector configured to project an image into the user's eyes and display image content within the user's field of view. The display system can include a camera, at least one waveguide, at least one coupling optical element configured such that light is coupled into and guided within the waveguide, and at least one external coupling element. The at least one external coupling element can be configured to couple light guided within the waveguide out of the waveguide and direct the light towards the camera. The camera can be disposed in an optical path with respect to the at least one external coupling optical element to receive at least a portion of the light that is coupled into and guided within the waveguide and coupled out of the waveguide by the external coupling element such that an image can be captured by the camera. The present invention provides, for example, the following. (Item 1) A head-mounted display system configured to project light onto a user's eyes and display augmented reality image content within the user's field of view, the head-mounted display system comprising: a frame configured to be supported on the user's head; an image projector configured to project an image into the user's eyes and display image content within the user's field of view; at least one camera; at least one waveguide; at least one coupling optical element, the at least one coupling optical element being configured such that light is coupled into and guided within the waveguide, the at least one coupling optical element comprising a diffractive optical element having refractive power; At least one external coupling element configured to couple the light guided in the waveguide out of the waveguide and direct the light towards the camera and comprising A camera is disposed in an optical path with respect to the at least one external coupling optical element so that an image can be captured by the camera, through the coupling optical element, coupled into the waveguide, guided therein, and received by the external coupling element of at least a part of the light coupled out of the waveguide, a head-mounted display system. (Item 2) The at least one coupling optical element is configured such that light reflected from the eye of a user wearing the head-mounted display system is coupled into the at least one waveguide and guided therein so that an image of the eye can be captured by the at least one camera, the system according to item 1. (Item 3) The at least one coupling optical element is configured such that light reflected from the front part of the eye of a user wearing the head-mounted display system is coupled into the waveguide and guided therein so that an image of the eye can be captured by the camera and the at least one camera captures an image of the front part of the eye, the system according to item 1. (Item 4) The at least one coupling optical element is configured such that light reflected from the corneal surface of the eye of a user wearing the head-mounted display system is coupled into the waveguide and guided therein so that an image of the eye can be captured by the camera and the at least one camera can capture an image of the corneal surface of the eye, the system according to item 1. (Item 5) The refractive power of the coupling optical element is configured to increase the collimation of the light reflected from the eye that is coupled into the waveguide and guided to the camera, the system according to item 1. (Item 6) The refractive power of the coupling optical element is configured to increase the collimation of light reflected from the anterior portion of the eye that is coupled into the waveguide and directed to the camera, for the system of claim 1. (Item 7) The refractive power of the coupling optical element is configured to increase the collimation of light reflected from the cornea of the eye that is coupled into the waveguide and directed to the camera, for the system of claim 1. (Item 8) The refractive power is a positive refractive power, for the system of claim 1. (Item 9) The system of claim 1 further comprises an eyepiece lens disposed on the frame, the eyepiece lens being configured to direct light into the user's eye and display augmented reality image content in the user's field of view, at least a portion of the eyepiece lens being transparent, the transparent portion being configured to transmit light from the user's frontal environment to the user's eye and provide a view of the user's frontal environment when the user wears the head-mounted display and is disposed at a location in front of the user's eye. (Item 10) The eyepiece lens receives light from the image projector and is configured to direct the light into the user's eye and display the augmented reality image content in the user's field of view, for the system of claim 9. (Item 11) The eyepiece lens comprises the at least one waveguide, for the system of claim 9. (Item 12) The image projector is configured to direct light into the edge of the eyepiece lens, for the system of claim 9. (Item 13) The system of claim 1 further comprises at least one internal coupling optical element configured to internally couple light from the image projector into the at least one waveguide so as to direct the light from the image projector for providing the image content to the user's eye. (Item 14) The system according to item 1, further comprising at least one external image content coupling optical element configured to couple light from the image projector guided in the waveguide out of the at least one waveguide so that the image content can be visually recognized by the eyes of the user. (Item 15) The system according to item 1, wherein the at least one coupling optical element is directed towards the eyes of the user wearing the head-mounted imaging system and receives light from the eyes. (Item 16) The system according to item 1, wherein the at least one coupling optical element is configured such that light from the environment in front of the user wearing the head-mounted imaging system is coupled into and guided in the at least one waveguide so that an image of the environment can be captured by the camera. (Item 17) The system according to item 1, wherein the at least one coupling optical element is directed towards the environment in front of the user wearing the head-mounted imaging system and receives light from the environment. (Item 18) The system according to item 14, wherein the at least one coupling optical element is configured such that light is coupled into and guided in the first waveguide towards the camera, and the at least one external image content coupling optical element is configured to couple light from the image projector guided in the second waveguide out of the second waveguide. (Item 19) The system according to item 1, wherein the at least one coupling optical element is configured such that light is coupled into and guided in the first waveguide towards the camera, and the image projector is configured to couple light into the second waveguide and provide image content to the eyes.

Brief Description of the Drawings

[0009]

Figure 1

[0010]

Figure 2

[0011]

Figure 3

[0012]

Figure 4

[0013]

Figure 5

[0014]

Figure 6

[0015]

Figure 7

[0016]

Figure 8

[0017]

Figure 9A

[0018]

Figure 9B

[0019]

Figure 9C

[0020]

Figure 10

[0021]

Figure 11A

[0022]

Figure 11B

[0023]

Figure 11C

[0024]

Figure 11D

[0025]

Figure 11E

[0026]

Figure 12A

[0027]

Figure 12B

[0028]

Figure 12C

[0029]

Figure 12D

[0030]

Figure 12E

[0031]

Figure 13A

[0032]

Figure 13B

[0033]

Figure 14A

[0034]

Figure 14B

[0035]

Figure 14C

[0036]

Figure 15A

[0037]

Figure 15B

[0038]

Figure 16

[0039]

Figure 17

[0040]

Figure 18

[0041]

Figure 19

[0042]

Figure 20A

Figure 20B

Figure 20C

Figure 20D

Figure 20E

[0043]

Figure 21

[0044]

Figure 22

[0045]

Figure 23

[0046]

Figure 24

[0047]

Figure 25

[0048]

Figure 26

[0049]

Figure 27

[0050] The drawings are provided to illustrate exemplary embodiments and are not intended to limit the scope of the present disclosure. Like reference numerals refer to like parts throughout.

DETAILED DESCRIPTION OF THE INVENTION

[0051] Reference is now made to the figures, where like reference numerals refer to like parts throughout.

[0052] FIG. 2 illustrates an embodiment of a wearable display system 60. The display system 60 includes a display 70 and various mechanical and electronic modules and systems to support the functionality of the display 70. The display 70 may be coupled to a frame 80, which is wearable by a display system user or viewer 90 and configured to position the display 70 in front of the eyes of the user 90. The display 70 may be considered eyewear in some embodiments. In some embodiments, a speaker 100 is coupled to the frame 80 and configured to be positioned adjacent to the outer ear canal of the user 90 (in some embodiments, another speaker, not shown, may optionally be positioned adjacent to the other outer ear canal of the user to provide stereo / formable sound control). The display system may also include one or more microphones 110 or other devices to detect sound. In some embodiments, the microphone is configured to enable the user to provide an input or command to the system 60 (e.g., selection of a voice menu command, natural language question, etc.) and / or enable audio communication with other persons (e.g., other users of similar display systems). The microphone may further be configured as a peripheral sensor to collect audio data (e.g., sound from the user and / or environment). In some embodiments, the display system may also include a peripheral sensor 120a, which is separate from the frame 80 and may be attached to the body of the user 90 (e.g., on the head, torso, limbs, etc. of the user 90). The peripheral sensor 120a may be configured to acquire data characterizing the physiological state of the user 90 in some embodiments. For example, the sensor 120a may be an electrode.

[0053] Continuing to refer to FIG. 2, the display 70 is operably coupled to a local data processing module 140 by a communication link 130 such as a wired conductor or wireless connectivity, which is fixedly attached to the frame 80, fixedly attached to a helmet or hat worn by the user, built into headphones, or otherwise removably attached to the user 90 (e.g., in a backpack configuration, in a belt attachment configuration), etc., and may be mounted in various configurations. Similarly, the sensor 120a may be operably coupled to the local data processing module 140 by a communication link 120b, such as a wired conductor or wireless connectivity. The local processing and data module 140 may comprise a digital memory such as a hardware processor and a non-volatile memory (e.g., flash memory or hard disk drive), both of which may be utilized to assist in the processing, caching, and storing of data. The data includes a) data captured from an image capture device (such as a camera), microphone, inertial measurement unit, accelerometer, compass, GPS unit, wireless device, gyroscope, and / or other sensors disclosed herein (e.g., operably coupled to the frame 80 or otherwise attachable to the user 90), and / or b) data that may be obtained and / or processed using a remote processing module 150 and / or a remote data repository 160 (including data related to virtual content) for passage to the display 70 after processing or retrieval. The local processing and data module 140 may be operably coupled to the remote processing module 150 and the remote data repository 160 by communication links 170, 180, such as via a wired or wireless communication link, such that these remote modules 150, 160 are operably coupled to each other and are available as resources to the local processing and data module 140. In some embodiments, the local processing and data module 140 may include one or more of an image capture device, microphone, inertial measurement unit, accelerometer, compass, GPS unit, wireless device, and / or gyroscope.In some other embodiments, one or more of these sensors may be attached to the frame 80 or may be of an independent structure that communicates with the local processing and data module 140 via a wired or wireless communication path.

[0054] Continuing to refer to FIG. 2, in some embodiments, the remote processing module 150 may include one or more processors configured to analyze and process data and / or image information. In some embodiments, the remote data repository 160 may include a digital data storage facility, which may be available through the Internet or other networking configurations in a "cloud" resource configuration. In some embodiments, the remote data repository 160 may include one or more remote servers that provide information, such as information for generating augmented reality content, to the local processing and data module 140 and / or the remote processing module 150. In some embodiments, all data is stored and all calculations are performed in the local processing and data module, enabling completely autonomous use from the remote module.

[0055] Referring now to FIG. 3, the perception of an image as "three-dimensional" or "3-D" can be achieved by providing slightly different presentations of the image to each eye of the viewer. FIG. 3 illustrates a conventional display system for simulating a three-dimensional image for a user. Two distinct images 190, 200, one for each eye 210, 220, are output to the user. Images 190, 200 are separated from eyes 210, 220 by a distance 230 along an optical axis or z-axis parallel to the viewer's line of sight. Images 190, 200 are flat, and eyes 210, 220 can be focused on the images by taking a single focused state. Such a 3-D display system relies on the human visual system to combine images 190, 200 and provide a perception of depth and / or scale of the combined images.

[0056] However, it should be understood that the human visual system is more complex and it is more difficult to provide a realistic perception of depth. For example, many viewers of conventional "3-D" display systems find such systems uncomfortable or may not perceive any sense of depth at all. Although not limited by theory, it is believed that a viewer of an object can perceive the object as "three-dimensional" due to a combination of vergence and accommodation. The vergence of two eyes with respect to each other (i.e., the rotation of the eyes where the pupils converge or diverge towards or away from each other to fixate on an object) is closely associated with the focusing (or "accommodation") of the eye's lens and pupils. Under normal conditions, changing the focus of the eye's lens, or accommodating the eye, to change the focus from one object to another at a different distance will automatically cause a coordinated change in vergence to the same distance under the relationship known as the "accommodation-vergence reflex" and pupillary dilation or constriction. Similarly, a change in vergence will, under normal conditions, induce a corresponding change in accommodation of the lens shape and pupil size. As described herein, many stereoscopic or "3-D" display systems use slightly different presentations (and thus slightly different images) to each eye to display a scene such that the three-dimensional perspective is perceived by the human visual system. However, such systems are uncomfortable for many viewers, especially because they simply provide different presentations of the scene but function against the "accommodation-vergence reflex" when the eyes view all the image information in a single accommodated state. A display system that provides a better match between accommodation and vergence can create a more realistic and comfortable simulation of a three-dimensional image.

[0057] FIG. 4 illustrates a side view of an approach for simulating a three-dimensional image using a plurality of depth planes. Referring to FIG. 4, objects at various distances from eyes 210, 220 on the z-axis are focused by eyes 210, 220 such that those objects are in focus. Eyes 210, 220 assume a particular focused state and focus the objects at different distances along the z-axis. As a result, a particular focused state can be said to be associated with a particular one of depth planes 240 that has an associated focal length such that an object or a portion of an object in that particular depth plane is in focus when the eyes are in the focused state with respect to that depth plane. In some embodiments, the three-dimensional image may be simulated by providing different presentations of the image for each of eyes 210, 220 and also by providing different presentations of the image corresponding to each of the depth planes. Although shown as separate for clarity of illustration, it should be understood that the fields of view of eyes 210, 220 may overlap, for example, as the distance along the z-axis increases. Additionally, although shown as flat for ease of illustration, it should be understood that the contours of the depth planes may be curved in physical space such that all features within the depth plane are in focus with the eyes in a particular focused state.

[0058] The distance between the object and eye 210 or 220 can also vary the amount of divergence of light from that object as viewed by that eye. FIGS. 5A-5C illustrate the relationship between distance and divergence of light rays. The distance between the object and eye 210 is represented in the order of decreasing distances R1, R2, and R3. As shown in FIGS. 5A-5C, the light rays diverge more as the distance to the object decreases. As the distance increases, the light rays become more collimated. In other words, it can be said that the light field generated by a point (object or part of an object) has a spherical wavefront curvature that is a function of the distance the point is away from the user's eye. The curvature increases as the distance between the object and eye 210 decreases. As a result, at different depth planes, the divergence of the light rays also differs, and the divergence increases as the distance between the depth plane and the viewer's eye 210 decreases. Only the monocular 210 is illustrated in FIGS. 5A-5C and various other figures herein for purposes of clarity of illustration, but it should be understood that the discussion regarding the eye 210 can apply to both eyes 210 and 220 of the viewer.

[0059] Although not limited by theory, the human eye is thought to typically be able to interpret a finite number of depth planes and provide depth perception. As a result, a highly realistic simulation of the perceived depth can be achieved by providing different presentations of images corresponding to each of these limited number of depth planes to the eye. The different presentations are focused separately by the viewer's eye, thereby based on the eye accommodation required to focus on different image features for scenes located on different depth planes and / or based on the observation of different image features on different out-of-focus depth planes, which may help provide depth cues to the user.

[0060] FIG. 6 illustrates an example of a waveguide stack for outputting image information to a user. Display system 250 includes a stack of waveguides or a stacked waveguide assembly 260 that can be utilized to provide 3D perception to the eye / brain using a plurality of waveguides 270, 280, 290, 300, 310. In some embodiments, display system 250 is the system 60 of FIG. 2, and FIG. 6 schematically shows some parts of that system 60 in more detail. For example, waveguide assembly 260 may be part of display 70 of FIG. 2. It should be understood that display system 250 may be considered a light field display in some embodiments. Additionally, waveguide assembly 260 may also be referred to as an eyepiece.

[0061] Continuing to refer to FIG. 6, waveguide assembly 260 may also include a plurality of features 320, 330, 340, 350 between the waveguides. In some embodiments, features 320, 330, 340, 350 may be one or more lenses. Waveguides 270, 280, 290, 300, 310 and / or the plurality of lenses 320, 330, 340, 350 may be configured to transmit image information to the eye using various levels of wavefront curvature or ray divergence. Each waveguide level may be associated with a particular depth plane and may be configured to output image information corresponding to that depth plane. Image input devices 360, 370, 380, 390, 400 may function as light sources for the waveguides and may be utilized to input image information into waveguides 270, 280, 290, 300, 310, and each may be configured to disperse incident light across the respective waveguide to output it towards eye 210, as described herein. Light exits from output surfaces 410, 420, 430, 440, 450 of image input devices 360, 370, 380, 390, 400 and is input into corresponding input surfaces 460, 470, 480, 490, 500 of waveguides 270, 280, 290, 300, 310. In some embodiments, input surfaces 460, 470, 480, 490, 500 may each be an edge of the corresponding waveguide or a portion of the major surface of the corresponding waveguide (i.e., one of the waveguide surfaces facing directly towards world 510 or viewer's eye 210). In some embodiments, a single beam of light (e.g., a collimated beam) may be input into each waveguide and output an entire field of cloned collimated beams directed towards eye 210 at a particular angle (and amount of divergence) corresponding to the depth plane associated with the particular waveguide. In some embodiments, a single one of image input devices 360, 370, 380, 390, 400 may be associated with and input light into a plurality (e.g., three) of waveguides 270, 280, 290, 300, 310.

[0062] In some embodiments, the image input devices 360, 370, 380, 390, 400 are discrete displays that each generate image information for input into the corresponding waveguides 270, 280, 290, 300, 310, respectively. In some other embodiments, the image input devices 360, 370, 380, 390, 400 are the output ends of a single multiplexed display that can send image information to each of the image input devices 360, 370, 380, 390, 400, for example, via one or more optical waveguides (such as optical fiber cables). It should be understood that the image information provided by the image input devices 360, 370, 380, 390, 400 may include light of different wavelengths or colors (e.g., different primary colors as discussed herein).

[0063] In some embodiments, the light input into the waveguides 270, 280, 290, 300, 310 is provided by an optical projector system 520, which includes an optical module 540 that may include a light emitter such as a light emitting diode (LED). The light from the optical module 540 may be directed and modified by a beam splitter 550 and an optical modulator 530, such as a spatial light modulator. The optical modulator 530 may be configured to vary the perceived intensity of the light input into the waveguides 270, 280, 290, 300, 310. Examples of spatial light modulators include liquid crystal displays (LCDs) and liquid crystal on silicon (LCOS) displays. The image input devices 360, 370, 380, 390, 400 are shown schematically, and in some embodiments, it should be understood that these image input devices may represent different optical paths and locations within a common projection system that are configured to output light into the associated ones of the waveguides 270, 280, 290, 300, 310.

[0064] In some embodiments, the display system 250 may be a scanning fiber display comprising one or more scanning fibers configured to project light into one or more waveguides 270, 280, 290, 300, 310 in various patterns (e.g., raster scan, spiral scan, Lissajous pattern, etc.) and ultimately into the viewer's eye 210. In some embodiments, the illustrated image input devices 360, 370, 380, 390, 400 may schematically represent a single scanning fiber or a bundle of scanning fibers configured to input light into one or more waveguides 270, 280, 290, 300, 310. In some other embodiments, the illustrated image input devices 360, 370, 380, 390, 400 may schematically represent a plurality of scanning fibers or a plurality of bundles of scanning fibers, each configured to input light into an associated one of the waveguides 270, 280, 290, 300, 310. It should be understood that one or more optical fibers may be configured to transmit light from the optical module 540 to one or more waveguides 270, 280, 290, 300, 310. It should be understood that one or more intervening optical structures may be provided between the scanning fiber or fibers and the one or more waveguides 270, 280, 290, 300, 310, for example, to redirect light exiting the scanning fiber into one or more of the waveguides 270, 280, 290, 300, 310.

[0065] The controller 560 controls the operation of one or more of the stacked waveguide assemblies 260, including the operation of the image input devices 360, 370, 380, 390, 400, the light source 540, and the optical module 530. In some embodiments, the controller 560 is part of the local data processing module 140. The controller 560 includes programming (e.g., instructions in a non-transitory medium) that adjusts the timing and provisioning of image information to waveguides 270, 280, 290, 300, 310, for example, according to any of the various schemes disclosed herein. In some embodiments, the controller may be a single monolithic device or a distributed system connected by wired or wireless communication channels. The controller 560 may, in some embodiments, also be part of the processing module 140 or 150 (FIG. 2).

[0066] Continuing to refer to FIG. 6, the waveguides 270, 280, 290, 300, 310 may be configured to propagate light within each individual waveguide by total internal reflection (TIR). The waveguides 270, 280, 290, 300, 310 may each be planar or have another shape (e.g., curved), with a major top surface and a major bottom surface and an edge extending between those major top and bottom surfaces. In the illustrated configuration, the waveguides 270, 280, 290, 300, 310 each include external coupling optical elements 570, 580, 590, 600, 610 configured to extract light from the waveguide by redirecting the light propagating within each individual waveguide out of the waveguide and outputting the image information to the eye 210. The extracted light may also be referred to as external coupled light, and the external coupling optical elements may also be referred to as light extraction optical elements. The beam of the extracted light may be output by the waveguide at the location where the light propagating within the waveguide impinges on the light extraction optical element. The external coupling optical elements 570, 580, 590, 600, 610 may be gratings, for example, including diffractive optical features as further discussed herein. For ease of explanation and clarity of the drawings, the external coupling optical elements 570, 580, 590, 600, 610 are shown disposed on the bottom major surface of the waveguides 270, 280, 290, 300, 310, but in some embodiments, the external coupling optical elements 570, 580, 590, 600, 610 may be disposed on the top major surface and / or the bottom major surface and / or directly disposed within the volume of the waveguides 270, 280, 290, 300, 310, as further discussed herein. In some embodiments, the external coupling optical elements 570, 580, 590, 600, 610 may be attached to a transparent substrate and formed within a layer of material forming the waveguides 270, 280, 290, 300, 310. In some other embodiments, the waveguides 270, 280, 290, 300, 310 may be a monolithic piece of material, and the external coupling optical elements 570, 580, 590, 600, 610 may be formed on and / or within the surface of that piece of material.

[0067] Continuing to refer to FIG. 6, as discussed herein, each of the waveguides 270, 280, 290, 300, 310 is configured to output light and form an image corresponding to a particular depth plane. For example, the waveguide 270 closest to the eye may be configured to deliver collimated light (input into such a waveguide 270) to the eye 210. The collimated light may represent an optically infinite focal plane. The next upper waveguide 280 may be configured to output collimated light that passes through a first lens 350 (e.g., a negative lens) before reaching the eye 210. Such a first lens 350 may be configured to generate some convex wavefront curvature so that the eye / brain interprets the light arising from the next upper waveguide 280 as arising from a first focal plane that is closer inwardly toward the eye 210 from optical infinity. Similarly, the third upper waveguide 290 passes its output light through both the first lens 350 and the second lens 340 before reaching the eye 210. The combined refractive power of the first lens 350 and the second lens 340 may be configured to generate another incremental amount of wavefront curvature so that the eye / brain interprets the light arising from the third waveguide 290 as arising from a second focal plane that is closer inwardly toward the person from optical infinity than the light from the next upper waveguide 280 was.

[0068] The other waveguide layers 300, 310 and lenses 330, 320 are similarly configured, and the top waveguide 310 in the stack sends its output through all of the lenses between it and the eye for the collective focusing power that represents the focal plane closest to the person. When viewing / interpreting light originating from the world 510 on the other side of the stacked waveguide assembly 260, a compensating lens layer 620 may be disposed on top of the stack to compensate for the stack of lenses 320, 330, 340, 350. Such a configuration provides the same number of perceived focal planes as there are available waveguide / lens pairs. Both the external coupling optical elements of the waveguides and the focusing sides of the lenses may be static (i.e., not dynamic or electroactive). In some alternative embodiments, one or both may be dynamic using electroactive features.

[0069] In some embodiments, two or more of the waveguides 270, 280, 290, 300, 310 may have the same associated depth plane. For example, a plurality of waveguides 270, 280, 290, 300, 310 may be configured to output images set to the same depth plane, or a plurality of subsets of waveguides 270, 280, 290, 300, 310 may be configured to output images set to the same plurality of depth planes, with one set per depth plane. This can provide the advantage of forming a tiled image to provide an extended field of view in those depth planes.

[0070] Continuing to refer to FIG. 6, the external coupling optical elements 570, 580, 590, 600, 610 may be configured to redirect light from their respective waveguides for a particular depth plane associated with the waveguide and output the light with an appropriate amount of divergence or collimation. As a result, waveguides having different associated depth planes may have external coupling optical elements 570, 580, 590, 600, 610 of different configurations that output light with different amounts of divergence depending on the associated depth plane. In some embodiments, the light extraction optical elements 570, 580, 590, 600, 610 may be three-dimensional features or surface features configured to output light at a specific angle. For example, the light extraction optical elements 570, 580, 590, 600, 610 may be volume holograms, surface holograms, and / or diffraction gratings. In some embodiments, features 320, 330, 340, 350 may not be lenses. Rather, they may simply be spacers (e.g., cladding layers and / or structures for forming voids).

[0071] In some embodiments, the external coupling optical elements 570, 580, 590, 600, 610 are diffraction features or "diffractive optical elements" (also referred to herein as "DOEs") that form a diffraction pattern. Preferably, the DOE has a relatively low diffraction efficiency such that only a portion of the light of the beam is deflected toward the eye 210 at each intersection of the DOE while the remainder continues to travel through the waveguide via TIR. The light carrying the image information is thus split into several associated output beams that exit the waveguide at multiple locations, resulting in a very uniform pattern of output emission toward the eye 210 with respect to this particular collimated beam that bounces within the waveguide.

[0072] In some embodiments, one or more DOEs may be switchable between an “on” state in which they actively diffract and an “off” state in which they do not significantly diffract. For example, a switchable DOE may comprise a layer of polymer dispersed liquid crystal in which microdroplets provide a diffraction pattern within a host medium, and the refractive index of the microdroplets may be switched to substantially match that of the host material (in which case the pattern does not significantly diffract the incident light), or the microdroplets may be switched to a refractive index that does not match that of the host medium (in which case the pattern actively diffracts the incident light).

[0073] In some embodiments, a camera assembly 630 (e.g., a digital camera including visible and infrared light cameras) may be provided to capture an image of the eye 210 and / or the tissue surrounding the eye 210, and, for example, detect user input and / or monitor the user's physiological state. As used herein, a camera may be any image capture device. In some embodiments, the camera assembly 630 may include an image capture device and a light source (e.g., infrared light) that projects light onto the eye, which light may then be reflected by the eye and detected by the image capture device. In some embodiments, the camera assembly 630 may be attached to the frame 80 (FIG. 2) and may communicate electrically with a processing module 140 and / or 150 that may process the image information from the camera assembly 630. In some embodiments, one camera assembly 630 may be utilized per eye to monitor each eye separately.

[0074] Referring now to FIG. 7, an embodiment of an output beam output by a waveguide is shown. Although one waveguide is illustrated, it should be understood that the other waveguides within the waveguide assembly 260 (FIG. 6) may function similarly, and the waveguide assembly 260 includes a plurality of waveguides. Light 640 is input into the waveguide 270 at the input surface 460 of the waveguide 270 and propagates through the waveguide 270 by TIR. At the point where the light 640 impinges on the DOE 570, a portion of the light exits the waveguide as the output beam 650. The output beam 650 is illustrated as being substantially parallel, but as discussed herein, and depending on the depth plane associated with the waveguide 270, it may be redirected to propagate at an angle (e.g., to form a diverging output beam) to the eye 210. It should be understood that a waveguide with an external coupling optical element that externally couples light to form an image that appears to be set in a depth plane at a long distance (e.g., optical infinity) from the eye 210 may be shown for the substantially parallel output beam. Other waveguides or other sets of external coupling optical elements may output a more diverging output beam pattern, which would require the eye 210 to focus at a closer distance and would be interpreted by the brain as light from a distance closer to the eye 210 than optical infinity.

[0075] In some embodiments, a full-color image may be formed in each depth plane by overlaying the image on each of the primary colors, e.g., three or more primary colors. FIG. 8 illustrates an example of a stacked waveguide assembly, and each depth plane includes an image formed using a plurality of different primary colors. The illustrated embodiment shows depth planes 240a - 240f, although more or fewer depths may also be considered. Each depth plane may have three or more primary color images associated therewith, including a first image of a first color G, a second image of a second color R, and a third image of a third color B. Different depth planes are shown in the figure by different numbers related to the diopter (dpt) following the letters G, R, and B. As a mere example, the numbers following each of these letters indicate the diopter (1 / m), i.e., the inverse distance of the depth plane from the viewer, and each box in the figure represents an individual primary color image. In some embodiments, the exact location of the depth planes for different primary colors may vary to account for differences in the focusing of light of different wavelengths by the eye. For example, the different primary color images for a given depth plane may be placed on depth planes corresponding to different distances from the user. Such an arrangement may increase visual acuity and user comfort and / or reduce chromatic aberration.

[0076] In some embodiments, the light of each primary color may be output by a single dedicated waveguide, and as a result, each depth plane may have a plurality of waveguides associated therewith. In such embodiments, each box in the figure, including those containing the letters G, R, or B, may be understood to represent an individual waveguide, and three waveguides may be provided for each depth plane, with three primary color images being provided for each depth plane. The waveguides associated with each depth plane are shown adjacent to each other in this drawing for ease of explanation, but it should be understood that in a physical device, all of the waveguides may be arranged in a stack with one waveguide per level. In some other embodiments, a plurality of primary colors may be output by the same waveguide, e.g., such that only a single waveguide is provided for each depth plane.

[0077] Continuing to refer to FIG. 8, in some embodiments, G is green, R is red, and B is blue. In some other embodiments, other colors associated with other wavelengths of light, including magenta and cyan, may also be used in addition to or in place of one or more of red, green, or blue.

[0078] It should be recognized that references throughout this disclosure to the color of a given light are understood to encompass light of one or more wavelengths within the range of wavelengths of the light as perceived by a viewer as that given color. For example, red light may include light of one or more wavelengths within the range of about 620 - 780 nm, green light may include light of one or more wavelengths within the range of about 492 - 577 nm, and blue light may include light of one or more wavelengths within the range of about 435 - 493 nm.

[0079] In some embodiments, the light source 540 (FIG. 6) may be configured to emit light of one or more wavelengths outside the visual perception range of the viewer, such as infrared and / or ultraviolet wavelength light. Additionally, the internal coupling, external coupling, and other light redirection structures of the waveguide of the display 250 may be configured to direct and emit this light from the display towards the user's eye 210, for example, for imaging and / or user stimulation applications.

[0080] Referring now to FIG. 9A, in some embodiments, light that impinges on the waveguide may need to be redirected to internally couple the light into the waveguide. An internal coupling optical element may be used to redirect and internally couple the light into its corresponding waveguide. FIG. 9A illustrates a cross-sectional side view of an example of a stack 660 of multiple or a set of waveguides, each including an internal coupling optical element. Each waveguide may be configured to output light of one or more different wavelengths or one or more different wavelength ranges. Stack 660 may correspond to stack 260 (FIG. 6), and the illustrated waveguides of stack 660 may correspond to a portion of the plurality of waveguides 270, 280, 290, 300, 310, but it should be understood that light from one or more of the image input devices 360, 370, 380, 390, 400 is input into the waveguide from a position where the light needs to be redirected for internal coupling.

[0081] The illustrated set 660 of stacked waveguides includes waveguides 670, 680, and 690. Each waveguide includes an associated internal coupling optical element (which may also be referred to as the optical input area on the waveguide). For example, internal coupling optical element 700 is disposed on a major surface (e.g., the upper major surface) of waveguide 670, internal coupling optical element 710 is disposed on a major surface (e.g., the upper major surface) of waveguide 680, and internal coupling optical element 720 is disposed on a major surface (e.g., the upper major surface) of waveguide 690. In some embodiments, one or more of the internal coupling optical elements 700, 710, 720 may be disposed on the bottom major surface of the individual waveguides 670, 680, 690 (in particular, one or more of the internal coupling optical elements are reflective deflecting optical elements). As illustrated, the internal coupling optical elements 700, 710, 720 may be disposed on the upper major surface of their respective waveguides 670, 680, 690 (or the upper part of the next lower waveguide), and in particular, those internal coupling optical elements are transmissive deflecting optical elements. In some embodiments, the internal coupling optical elements 700, 710, 720 may be disposed within the bodies of the individual waveguides 670, 680, 690. In some embodiments, as discussed herein, the internal coupling optical elements 700, 710, 720 are wavelength selective such that they selectively redirect one or more wavelengths of light while transmitting other wavelengths of light. Although illustrated on one side or corner of their respective waveguides 670, 680, 690, it should be understood that in some embodiments, the internal coupling optical elements 700, 710, 720 may be disposed within other areas of their respective waveguides 670, 680, 690.

[0082] As shown, the internally coupled optical elements 700, 710, 720 may be laterally offset from each other. In some embodiments, each internally coupled optical element may be offset such that its light is received without passing through another internally coupled optical element. For example, each internally coupled optical element 700, 710, 720 may be configured to receive light from different image input devices 360, 370, 380, 390, and 400, as shown in FIG. 6, and may be separated (e.g., laterally spaced) from other internally coupled optical elements 700, 710, 720 such that light is not substantially received from other internally coupled optical elements 700, 710, 720.

[0083] Each waveguide also includes an associated light dispersing element. For example, the light dispersing element 730 is disposed on a major surface (e.g., the upper major surface) of the waveguide 670, the light dispersing element 740 is disposed on a major surface (e.g., the upper major surface) of the waveguide 680, and the light dispersing element 750 is disposed on a major surface (e.g., the upper major surface) of the waveguide 690. In some other embodiments, the light dispersing elements 730, 740, 750 may each be disposed on the bottom major surface of the associated waveguides 670, 680, 690. In some other embodiments, the light dispersing elements 730, 740, 750 may each be disposed on both the upper and bottom major surfaces of the associated waveguides 670, 680, 690, or the light dispersing elements 730, 740, 750 may each be disposed on different ones of the upper and bottom major surfaces within different associated waveguides 670, 680, 690.

[0084] Waveguides 670, 680, 690 may be separated and isolated, for example, by gas, liquid, and / or solid layers of material. For example, as shown, layer 760a may separate waveguides 670 and 680, and layer 760b may separate waveguides 680 and 690. In some embodiments, layers 760a and 760b are formed from a low refractive index material (i.e., a material having a lower refractive index than the material forming the immediate waveguides 670, 680, 690). Preferably, the refractive index of the material forming layers 760a, 760b is 0.05 or more or 0.10 or less relative to the refractive index of the material forming waveguides 670, 680, 690. Advantageously, the lower refractive index layers 760a, 760b may function as cladding layers that promote total internal reflection (TIR) of light (e.g., TIR between the upper and lower major surfaces of each waveguide) through waveguides 670, 680, 690. In some embodiments, layers 760a, 760b are formed from air. It should be understood that although not shown, the upper and lower portions of the illustrated set 660 of waveguides may include an immediate cladding layer.

[0085] Preferably, for ease of manufacturing and other considerations, the materials forming waveguides 670, 680, 690 are similar or identical, and the materials forming layers 760a, 760b are similar or identical. In some embodiments, the materials forming waveguides 670, 680, 690 may be different between one or more waveguides and / or the materials forming layers 760a, 760b may still be different while maintaining the various refractive index relationships described above.

[0086] Continuing to refer to FIG. 9A, light rays 770, 780, 790 are incident on the set 660 of waveguides. It should be understood that light rays 770, 780, 790 may be input into waveguides 670, 680, 690 by one or more image input devices 360, 370, 380, 390, 400 (FIG. 6).

[0087] In some embodiments, the light rays 770, 780, 790 may have different properties, such as different wavelengths or different wavelength ranges, corresponding to different colors. The internal coupling optical elements 700, 710, 720 each deflect the incident light so that light propagates through an individual one of the waveguides 670, 680, 690 by TIR. In some embodiments, the internal coupling optical elements 700, 710, 720 each selectively deflect one or more specific wavelengths of light while transmitting other wavelengths to the underlying waveguide and the associated internal coupling optical element.

[0088] For example, the internal coupling optical element 700 may be configured to deflect the light ray 770 having a first wavelength or wavelength range while transmitting the light rays 780 and 790 having different second and third wavelengths or wavelength ranges, respectively. The transmitted light ray 780 impinges on the internal coupling optical element 710 configured to selectively deflect the light of the second wavelength or wavelength range, thereby being deflected. The light ray 790 is deflected by the internal coupling optical element 720 configured to selectively deflect the light of the third wavelength or wavelength range.

[0089] Continuing to refer to FIG. 9A, the deflected light rays 770, 780, 790 are deflected to propagate through the corresponding waveguides 670, 680, 690. That is, the internal coupling optical elements 700, 710, 720 of each waveguide deflect the light into its corresponding waveguide 670, 680, 690 and internally couple the light into the corresponding waveguide. The light rays 770, 780, 790 are deflected at an angle that causes the light to propagate through the individual waveguides 670, 680, 690 by TIR. The light rays 770, 780, 790 propagate through the individual waveguides 670, 680, 690 by TIR until they impinge on the corresponding light dispersion elements 730, 740, 750 of the waveguide.

[0090] Referring now to FIG. 9B, a perspective view of an embodiment of the plurality of stacked waveguides of FIG. 9A is illustrated. As described above, the internally coupled light rays 770, 780, 790 are each deflected by the internally coupled optical elements 700, 710, 720 and then each propagate by TIR within the waveguides 670, 680, 690. The light rays 770, 780, 790 then each impinge on the light dispersion elements 730, 740, 750. The light dispersion elements 730, 740, 750 each deflect the light rays 770, 780, 790 so as to propagate towards the external coupling optical elements 800, 810, 820.

[0091] In some embodiments, the optical dispersion elements 730, 740, 750 are orthogonal pupil expanders (OPEs). In some embodiments, the OPE deflects or disperses light to the external coupling optical elements 800, 810, 820, and in some embodiments, also increases the beam or spot size of the present light as it propagates to the external coupling optical elements. In some embodiments, the optical dispersion elements 730, 740, 750 may be omitted, and the internal coupling optical elements 700, 710, 720 may be configured to deflect light directly to the external coupling optical elements 800, 810, 820. For example, referring to FIG. 9A, the optical dispersion elements 730, 740, 750 may each be replaced with the external coupling optical elements 800, 810, 820. In some embodiments, the external coupling optical elements 800, 810, 820 are an exit pupil (EP) or an exit pupil expander (EPE) that directs light to the viewer's eye 210 (FIG. 7). It should be understood that the OPE may be configured to increase the size of the eyebox in at least one axis, and the EPE may increase the eyebox in an axis that intersects, for example, is orthogonal to, the axis of the OPE. For example, each OPE may be configured to redirect a portion of the light impinging on the OPE to the EPE of the same waveguide while allowing the remaining portion of the light to continue to propagate along the waveguide. In response to the collision with the OPE, again, another portion of the remaining light is redirected to the EPE, and the remaining portion of that portion continues to propagate further along the waveguide and the like. Similarly, in response to the collision with the EPE, a portion of the colliding light is directed from the waveguide towards the user, and the remaining portion of that light continues to propagate through the waveguide until it impinges on the EP again, at which point, another portion of the colliding light is directed from the waveguide, etc. As a result, a single beam of internally coupled light is "replicated" each time a portion of that light is redirected by the OPE or EPE, thereby forming a beam field of cloned light as shown in FIG. 6. In some embodiments, the OPE and / or EPE may be configured to modify the size of the beam of light.

[0092] Thus, referring to FIGS. 9A and 9B, in some embodiments, the waveguide set 660 includes, for each primary color, waveguides 670, 680, 690, internal coupling optical elements 700, 710, 720, light dispersion elements (e.g., OPEs) 730, 740, 750, and external coupling optical elements (e.g., EPs) 800, 810, 820. The waveguides 670, 680, 690 may be stacked with a gap / cladding layer between each one. The internal coupling optical elements 700, 710, 720 redirect or deflect the incident light into their respective waveguides (using different internal coupling optical elements that receive light of different wavelengths). The light then propagates at an angle that will result in TIR within the individual waveguides 670, 680, 690. In the example shown, the ray 770 (e.g., blue light) is polarized by the first internal coupling optical element 700 in the manner described above and then bounces along the waveguide, interacting with the light dispersion element (e.g., OPE) 730 and then the external coupling optical element (e.g., EP) 800. The rays 780 and 790 (e.g., green and red light, respectively) pass through the waveguide 670, and the ray 780 strikes the internal coupling optical element 710 and is thereby deflected. The ray 780 then bounces along the waveguide 680 via TIR and will proceed to its light dispersion element (e.g., OPE) 740 and then the external coupling optical element (e.g., EP) 810. Finally, the ray 790 (e.g., red light) passes through the waveguide 690 and strikes the internal coupling optical element 720 of the waveguide 690. The internal coupling optical element 720 deflects the ray 790 such that the ray propagates by TIR to the light dispersion element (e.g., OPE) 750 and then by TIR to the external coupling optical element (e.g., EP) 820. The external coupling optical element 820 then finally externally couples the ray 790 to the viewer, who also receives the externally coupled light from the other waveguides 670, 680.

[0093] FIG. 9C illustrates a top and bottom plan view of an embodiment of the plurality of stacked waveguides of FIGS. 9A and 9B. As shown, waveguides 670, 680, 690 may be vertically aligned with associated optical dispersion elements 730, 740, 750 and associated external coupling optical elements 800, 810, 820 of each waveguide. However, as discussed herein, internal coupling optical elements 700, 710, 720 are not vertically aligned. Rather, the internal coupling optical elements are preferably non-overlapping (e.g., laterally spaced as seen in the top and bottom views). As further discussed herein, this non-overlapping spatial arrangement facilitates the injection of light from different resources into different waveguides on a one-to-one basis, thereby enabling a particular light source to be uniquely coupled to a particular waveguide. In some embodiments, an array that includes non-overlapping spatially separated internal coupling optical elements may be referred to as a shifted pupil system, and the internal coupling optical elements within these arrays may correspond to sub-pupils. Eye imaging and environmental imaging

[0094] As discussed above, head-mounted displays can be used to provide users with image content that is integrated with, combined with, and / or overlaid across the wearer's view of the world in front. Such head-mounted display systems can be configured to project light into the user's eyes to form augmented reality image content and to transmit light from the user's front environment to the user. The head-mounted display system may include one or more cameras to image the environment and / or the user's eyes. An outward-facing camera can directly image the environment and may be used, for example, to determine where to place augmented reality image content relative to objects in the environment. For example, imaging the environment can provide the location of a table such that the head-mounted display can render an image of a person standing next to the table instead of on or within the table. An inward-facing camera can be directly used to image the eyes, for example, for eye tracking. Also disclosed herein are embodiments of head-mounted display systems and / or imaging systems that can be configured to image the eyes and / or the environment. In some designs, the system does not require inward- and / or outward-facing cameras to directly image the eyes and / or the environment, respectively. Such systems may employ one or more cameras configured to receive light from the eye / environment via an eyepiece, such as one or more waveguides within the eyepiece, in optical communication with the one or more cameras. When the light is collected by the waveguide, the one or more cameras can generate an image of the eyes and / or the user's front environment. Collecting light using waveguides to image the eyes and / or the environment can potentially reduce the form factor of the head-mounted display and make the head-mounted display potentially more compact and / or aesthetically desirable.

[0095] Figure 10 illustrates an exemplary imaging system 900 that is integrated with an eyepiece 950 and configured to image an eye, which can be used on a head-mounted display. The eyepiece 950, which can be disposed in front of the user's eye 210, can be used for both projecting image content into the eye and imaging the eye. Figure 10 shows one of the eyepieces 950 in front of one of the monocular eyes 210. Various head-mounted display systems such as those shown in Figure 2 may include a pair of eyepieces 950 disposed in front of the individual left and right eyes 210 and associated components. A single waveguide 940 is shown in Figure 10, but the waveguide 940 may include one, two, three, four, six, seven, eight, or more waveguides (e.g., a stack of one or more waveguides).

[0096] The imaging system 900 can include a light source or illumination source 960 that illuminates the eye and facilitates image capture, a waveguide 940 configured to propagate light therein, an eyepiece 950, and / or an imaging device 920 such as a camera for image capture. An image projector 930 for producing an image that can be projected into the eye via the eyepiece 950 is also shown. The eyepiece 950 may include one or more waveguides 940 configured to carry light from the illumination source 960 and / or the image projector 930 to the eye and light from the eye to the camera 920. The eyepiece 950 may further include one or more coupling optical elements 944 for coupling light out of the waveguide 940 to the eye for illumination and for image projection and / or for coupling light from the eye into the waveguide for image capture. The eyepiece 950 may additionally include one or more internal coupling optical elements 942 for coupling light from the illumination source 960 and / or the image projector 930 into the waveguide 940 and one or more external coupling optical elements 952 for coupling light from the waveguide to the camera 920.

[0097] The eyepiece lens 950 may be disposed on a frame that is mounted on the head. The eyepiece lens 950 may be disposed in front of the eye 210. The eyepiece lens 950 may have an inner or nasal side that is closer to the wearer's nose and an opposite outer or temple side that is closer to the wearer's temple and farther from the nose. In FIG. 10, the coupling optical element 944 is on the inner or nasal side (outer or temple side with respect to the coupling optical element 944) with respect to the internal coupling optical element 942 and the external coupling optical element 952. The illumination source 960 is also on the inner or nasal side with respect to the image projector 930 (or the image projector is on the outer or temple side of the illumination source). However, the relative positions may vary. For example, in some designs, the illumination source 960 may be on the outer or temple side of the image projector 930.

[0098] The waveguide 940 may comprise a sheet or layer having two major surfaces (front and rear surfaces) that are disposed opposite to each other and have a maximum surface area. The front surface may be farther from the user's eye 210 (closer to the environment in front of the wearer) when the user wears the head-mounted display, and the rear surface may be closer to the user's eye (and farther from the environment in front of the wearer). The waveguide 940 may include a transparent material (e.g., glass, plastic) with a refractive index greater than 1.0 such that light can be guided therein between the major surfaces by total internal reflection. Elements with the same number may have the same functionality for one or more of the embodiments described herein.

[0099] A coupling optical element 944 for coupling light from the waveguide 940 to and / or from the waveguide to the eye 210 may be disposed on or within the waveguide 940. As shown in FIG. 10, the coupling optical element 944 is arranged such that light coupled from the waveguide 940 through the coupling optical element 944 can be incident on the user's eye 210 (e.g., for illuminating the eye and / or for image projection), and may be disposed within the optical path between the user's eye 210 and the waveguide 940. The coupling optical element 944 may comprise a plurality of direction-changing features configured to redirect light incident on the coupling optical element 944 at an angle into the waveguide so as to redirect light guided within the waveguide out of the waveguide or to be guided therein by total internal reflection. The coupling optical element 944 and the direction-changing features may physically engage the waveguide 940. For example, the coupling optical element 944 may comprise a patterned (e.g., etched) holographic or diffractive optical element (e.g., a surface relief grating) within or on the waveguide 940. The coupling optical element 944 may comprise a layer disposed on the waveguide 940 or may be formed within the waveguide 940. For example, a volume holographic or other diffractive optical element may be formed by varying the refractive index of the material that constitutes the waveguide or a layer disposed thereon. Thus, the coupling optical element 944 may be disposed within the volume of the waveguide 940 or as a layer disposed thereon.

[0100] Depending on the design, the coupling optical element 944 may be transmissive or reflective and may operate transmissively or reflectively. For example, the coupling optical elements 944 may each operate transmissively or reflectively and include, for example, a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that redirects light transmitted through or reflected therefrom. The coupling optical element 944 can include a polarization optical element such as a polarization selective beam deflector (e.g., a polarizer). The polarization selective beam deflector may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements and may include a liquid crystal structure such as a liquid crystal polarization grating. The coupling optical element 944 is configured to direct light from the image projector 930 and / or the light source 960, which is guided within the waveguide 940 by total internal reflection (TIR), out of the waveguide and into the user's eye 210 at an angle less than the critical angle (e.g., a more perpendicular angle). Additionally or alternatively, the coupling optical element 944 may be configured to couple light from the eye 210 into the waveguide 940 at an angle greater than the critical angle (e.g., a less perpendicular angle) such that it is guided therein by total internal reflection to the camera 920.

[0101] As shown in FIG. 10, an internal coupling optical element 942 for coupling light from the illumination source 960 and / or the image projector 930 into the waveguide 940 may be disposed on or within the waveguide 940. The internal coupling optical element 942 may be disposed in the optical path between the light source 960 and the waveguide 940 such that light coupled from the light source 960 through the internal coupling optical element 942 is guided within the waveguide 940. The internal coupling optical element 942 may comprise a plurality of beam steering features configured to redirect light incident thereon at an angle into the waveguide such that it is guided therein, for example, by total internal reflection. The internal coupling optical element 942 may comprise a liquid crystal structure such as a liquid crystal polarization grating. Additionally or alternatively, the internal coupling optical element 942 may include a blazed grating. The internal coupling optical element 942 may comprise a layer disposed on the waveguide 940, or may be formed on or within (e.g., patterned) the waveguide 940, or otherwise fabricated therein. For example, a surface holographic or diffractive optical element (e.g., a surface relief grating) may be fabricated by patterning (e.g., etching) the surface of the waveguide or a layer thereon. A volume holographic or diffractive optical element may also be formed by varying the refractive index of the material constituting the waveguide or a layer disposed thereon. Thus, the internal coupling optical element 942 may be disposed within the volume of the waveguide 940 or in a layer disposed thereon. Depending on the design, the internal coupling optical element 942 may be transmissive or reflective and may operate transmissively or reflectively. For example, the internal coupling optical element 942 may each include a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates transmissively or reflectively and redirects, for example, light transmitted therethrough or reflected therefrom.

[0102] The internal coupling optical element 942 may include a reflective optical element (e.g., a mirror). For example, the internal coupling optical element 942 may include an off-axis reflector. Additionally, or alternatively, the internal coupling optical element 942 and / or the coupling optical element 944 may include a polarization optical element such as a polarization selective beam steering element (e.g., a polarizer). The polarization selective beam steering element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may include a liquid crystal structure such as a liquid crystal polarization grating. For example, one or both of the internal coupling optical element 942 and / or the coupling optical element 944 may include a liquid crystal polarization grating (LCPG). The LCPG can potentially provide high-efficiency diffraction over a broad wavelength. Thus, the LCPG can be useful for the internal coupling optical element 942 and / or the coupling optical element 944. The LCPG can be polarization dependent. The LCPG or other types of liquid crystal gratings, diffractive optical elements, or optical elements may include a pattern or arrangement of liquid crystal molecules configured to provide one or more functions such as redirecting light into or out of a waveguide. Thus, the internal coupling optical element 942 and / or the coupling optical element 944 may include a polarization grating. Additionally, or alternatively, the internal coupling optical element 942 and / or the coupling optical element 944 can include a liquid crystal, and thus, in some implementations, one or both may be a liquid crystal grating or a liquid crystal diffractive optical element. Additionally, or alternatively, one or both of the internal coupling optical element 942 and / or the coupling optical element 944 may include a blazed grating. In some designs, the internal coupling optical element 942 includes a liquid crystal reflector such as a cholesteric liquid crystal reflective lens (e.g., a reflective liquid crystal diffractive lens, a Bragg reflective structure, a reflective liquid crystal diffractive grating, etc.).Several non-limiting examples of liquid crystal lattices, liquid crystal polarization gratings, and other liquid crystal optical elements are hereby incorporated by reference in their entirety and for all purposes into this specification from the following published applications: U.S. Patent Publication No. 2018 / 0143438, entitled "MULTILAYER LIQUID CRYSTAL DIFFRACTIVE GRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES", filed on November 16, 2017; U.S. Patent Publication No. 2018 / 0143485, entitled "SPATIALLY VARIABLE LIQUID CRYSTAL DIFFRACTION GRATINGS", filed on November 16, 2017; U.S. Patent Publication No. 2018 / 0143509, entitled "WAVEGUIDE LIGHT MULTIPLEXER USING CROSSED GRATINGS", filed on November 16, 2017; U.S. Patent Publication No. 2018 / 0239147, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR", filed on February 22, 2018; U.S. Patent Publication No. 2018 / 0239177, entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION", filed on February 22, 2018; and U.S. Patent Publication No. 2018 / 0164627, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL", filed on December 7, 2017. However, the design of the internal coupling optical element 942 and / or the coupling optical element 944 is not limited to these and may include other types of optical elements, diffractive optical elements, liquid crystal optical elements, liquid crystal lattices, and liquid crystal polarization gratings. Further information regarding examples of cholesteric liquid crystal structures such as reflectors may also be found in the section entitled "Cholesteric Liquid Crystal Mirrors" below. As discussed above, other liquid crystal optical elements and other non-liquid crystal optical elements may be used.Thus, many types of coupled optical elements (e.g., internal coupled optical element 942 and / or coupled optical element 944), diffraction optical elements, gratings, polarization gratings, etc., which are both those described herein and other types of gratings, diffraction optical elements, liquid crystal elements, and optical elements in general, may be used. In various implementations, the internal coupled optical element 942 may be configured to couple light from the image projector 930 and / or the light source 960 into the waveguide 940 at an angle greater than the critical angle so as to be guided within the waveguide 940 to the user's eye 210 by total internal reflection to the eye.

[0103] The waveguide 940 may comprise one or more waveguides. In some implementations, the one or more waveguides 940 comprise a stack of waveguides. In some designs, for example, different waveguides of the stack of waveguides are configured to output light with different wavefront divergences as if projected from different distances from the user's eye. For example, the first waveguide or group of waveguides may be configured to output collimated or having a first divergence, light as if projected from a first depth, and the second waveguide or group of waveguides may be configured to output diverging (not collimated) or having a second divergence (greater than the first divergence), light as if projected from a second depth closer than the first depth. In some designs, different waveguides may be configured to output light having different associated colors. For example, the first waveguide may be configured to output red light, the second waveguide may be configured to output green light, and the third waveguide may be configured to output blue light. The fourth waveguide may be configured to output and / or input infrared light.

[0104] An external coupling optical element 952 for coupling light from a waveguide 940 as shown in FIG. 10 to a camera 920 may comprise a plurality of beam-steering features configured to steer light at an angle such that light incident thereon is not guided within the waveguide and is redirected out of the waveguide and towards the camera. The external coupling optical element 952 may be disposed within the interior of the waveguide 940 or may be patterned (e.g., etched) within or on the surface (e.g., major surface) of the waveguide 940. For example, surface holographic or diffractive optical elements (e.g., surface relief gratings) may be fabricated by patterning (e.g., etching) the surface of the waveguide or a layer thereon. Volume holographic or diffractive optical elements may also be formed by varying the refractive index of the material that makes up the waveguide or a layer disposed thereon. Depending on the design, the external coupling optical element 952 may be transmissive or reflective and may operate either transmissively or reflectively. For example, the external coupling optical element 952 may each include a transmissive or reflective diffractive optical element (e.g., grating) or holographic optical element that operates either transmissively or reflectively and steers light transmitted therethrough or reflected therefrom, respectively.

[0105] The external coupling optical element 942 may comprise a reflective optical element (e.g., a mirror). For example, the external coupling optical element 952 may comprise an off-axis reflector. In some designs, the external coupling optical element 952 can include a polarization optical element such as a polarization selective beam deflector (e.g., a polarizer). Thus, the polarization selective beam deflector may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may comprise a liquid crystal structure such as a liquid crystal polarization grating. In some implementations, for example, the external coupling optical element 952 can include a liquid crystal polarization grating (LCPG). The LCPG can potentially provide high efficiency diffraction at broad wavelengths. Similarly, the LCPG can be useful for the external coupling optical element 952. The LCPG can be polarization dependent. The LCPG or other types of liquid crystal gratings may include a pattern or arrangement of liquid crystal molecules configured to provide one or more functions such as deflecting light into or out of an optical waveguide. Thus, the external coupling optical element 952 may comprise a polarization grating. Additionally, or alternatively, the external coupling optical element 952 can comprise a liquid crystal, and thus, in some implementations, may be other liquid crystal optical elements such as a liquid crystal grating or a liquid crystal diffractive optical element. Additionally, or alternatively, the external coupling optical element 952 can include a blazed grating. In some designs, the external coupling optical element 952 comprises a liquid crystal reflector such as a cholesteric liquid crystal reflective lens (e.g., a reflective liquid crystal diffractive lens, a Bragg reflective structure, a reflective liquid crystal diffractive grating, etc.).Several non-limiting examples of liquid crystal lattices, liquid crystal polarization gratings, and other liquid crystal optical elements are hereby incorporated by reference in their entirety and for all purposes into this specification from the following published applications: U.S. Patent Publication No. 2018 / 0143438, entitled "MULTILAYER LIQUID CRYSTAL DIFFRACTIVE GRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES", filed on November 16, 2017; U.S. Patent Publication No. 2018 / 0143485, entitled "SPATIALLY VARIABLE LIQUID CRYSTAL DIFFRACTION GRATINGS", filed on November 16, 2017; U.S. Patent Publication No. 2018 / 0143509, entitled "WAVEGUIDE LIGHT MULTIPLEXER USING CROSSED GRATINGS", filed on November 16, 2017; U.S. Patent Publication No. 2018 / 0239147, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR", filed on February 22, 2018; U.S. Patent Publication No. 2018 / 0239177, entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION", filed on February 22, 2018; and U.S. Patent Publication No. 2018 / 0164627, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL", filed on December 7, 2017. However, the design of the external coupling optical element 952 is not limited to these and may include other types of optical elements, diffractive optical elements, liquid crystal optical elements, liquid crystal lattices, and liquid crystal polarization gratings. Further information regarding embodiments of cholesteric liquid crystal structures such as reflectors may also be found in the section entitled "Cholesteric Liquid Crystal Mirrors" below. As discussed above, other liquid crystal optical elements and other non-liquid crystal optical elements may be used.Accordingly, many types of combined optical elements (e.g., external coupling optical element 952), diffraction optical elements, gratings, polarization gratings, etc., which are both those described herein and other types of gratings, diffraction optical elements, liquid crystal elements, or optical elements in general, may be used. As referred to above, the external coupling optical element 952 may be configured to redirect light guided within the waveguide 940 at an angle less than the critical angle so that it is not guided within the waveguide by total internal reflection but is instead emitted to the camera 920.

[0106] In various designs, the coupling optical element 944 may be transmissive within the visible spectrum so that the user can see the user's frontal environment through the coupling optical element 944 and the eyepiece 950. The internal coupling optical element 942 may also redirect light within the visible spectrum, for example, when the internal coupling optical element is used to receive light from the image projector 930 and / or when the illumination source 960 is configured to output visible light and illuminate the eye 210 with visible light. In some embodiments, the internal coupling optical element 942 is configured to redirect infrared light, for example, when the illumination source 960 is configured to output infrared light and illuminate the eye 210 with infrared light. In some designs as shown in FIG. 10, the internal coupling optical element 942 may be medial or nasal to the external coupling optical element 952. However, in other designs, the internal coupling optical element 942 may be lateral or temple-side to the external coupling optical element 952. In one implementation as shown in FIG. 10, the external coupling optical element 952 may be adjacent to the internal coupling optical element 942, but non-adjacent positions are also conceivable as possibilities.

[0107] As shown in FIG. 10, the illumination source 960 may be disposed on the same side (e.g., rear or proximal side) of the eye 210 as the eyepiece lens 950 (proximal may refer to the side closest to the eye 210). Alternatively, the illumination source 960 may be disposed on the side opposite the eye 210 (e.g., front or distal side). The illumination source 960 may be configured to direct light into at least one of the major surfaces of the waveguide 940 via the internal coupling optical element 942. The light source 960 may be configured to emit invisible light (e.g., infrared). The light source 960 may include one or more LEDs. The LEDs may comprise infrared LEDs. The light source 960 may be configured to emit coherent light. In some designs, the light source 960 comprises a laser (e.g., an infrared laser). In some designs, the light source 960 emits pulsed light. For example, the camera 920 may be configured to periodically capture images. Thus, the illumination source 960 may be pulsed to coincide with the period during which the camera acquires an image. The intensity output from the illumination source 960 may be reduced when the camera is not acquiring an image. By concentrating the total energy of the illumination in a short time, an increased signal-to-noise ratio can be obtained without exposing the eye 210 to non-safe intensity levels. In one case, for example, the camera 920 captures one image every 30 milliseconds and the exposure time of the camera is a few milliseconds. The illumination source 960 may be configured to output pulses having a similar period and duration that match those of the camera 920.

[0108] In some implementations, different light sources having different wavelengths are alternatively pulsed to provide different wavelength illumination at different times, as discussed below.

[0109] The internal coupling optical element 942 may be in direct optical communication with the illumination source 960 and / or the image projector 930, for example, to direct light from the image projector 930 and / or the light source 960 into it. For example, the light emitted by the light source 960 may be incident on the internal coupling optical element 942 before optically interacting with either the coupling optical element 944 and / or the external coupling optical element 952.

[0110] As shown in FIGS. 11A-11E, the light 902 projected from the image projector 930 may form an image on the retina. The image projector 930 may include a light source, a modulator, and / or projection optics. The light source for the image projector 930 may comprise one or more LEDs, lasers, or other light sources, and may comprise one or more visible light sources. The modulator may comprise a spatial light modulator such as a liquid crystal spatial light modulator. Such a spatial light modulator may be configured to modulate the intensity of light at different spatial locations, for example. The projection optics may comprise one or more lenses. Other types of image projectors 930 capable of projecting and / or forming an image may also be employed. For example, the image projector 930 may comprise a scanning optical fiber.

[0111] The image projector 930 and the internal coupling optical element 942 may be in direct optical communication with each other. The image projector 930 may be aligned, for example, with the internal coupling optical element 942 into which light from the image projector 930 is directed. In some cases, the image projector 930 is disposed adjacent to the corresponding internal coupling optical element 942 and / or the waveguide 940. The image projector 930 may also be disposed within the optical path including the internal coupling optical element 942, the coupling optical element 944, and the eye 210.

[0112] As shown in FIGS. 10 and 11A-11E, the image projector 930 may be a separate element from the illumination source 960. However, in some cases, the image projector 930 may be used as an illumination source. For example, in addition to projecting an image into the eye 210, the image projector 930 may be used to direct visible and / or infrared light into the eye for image capture and to illuminate the eye. However, alternatively, one or more separate light sources 960 may be used to illuminate the eye 210 for image capture.

[0113] The light emitted by the illumination source 960 may comprise light in a specific wavelength range, such as, for example, invisible light. The illumination source 960 may be configured to project invisible (e.g., infrared) light onto / into the eye 210 to image one or more portions of the eye 210 (e.g., the cornea, the retina). In one exemplary implementation, the light source 960 may be configured to emit light in the range of approximately 850 nm to 940 nm. The light source 960 may be configured to emit light that extends over a wavelength range of at least approximately 20 nm. Other ranges are also possible. The emitted wavelength range may be 5 nm, 10 nm, 15 nm, 50 nm, 75 nm, 100 nm, 150 nm, 200 nm, or any range between any of these values. The light source 960 may be configured to emit light across a broad band of wavelengths, such as any range within the infrared spectrum.

[0114] The imaging device 920, which may include a camera, may include a detector array and possibly an imaging optical system. The detector array may include, for example, a CCD or CMOS detector array, and the imaging optical system may include one or more lenses. The one or more lenses may have a positive refractive power and an associated focal length. In one design, the camera 920 is focused at infinity. For example, the optical system may have a focal length f, and the detector array may be positioned at a distance from the optical system corresponding to the focal length such that an object at a large distance is imaged onto the detector array. Similarly, collimated light from an object in the eye or environment will also be focused onto the detector array and an image of the eye or object will be formed thereon.

[0115] The imaging device 920 may be positioned on the opposite side of the illumination source 960 of the waveguide 940 and / or the eye 210. In some designs, the imaging device 920 may be positioned on the same side as the light source 960 of the waveguide 940 and / or the eye 210. As shown in FIG. 10, the imaging device 920 may be positioned outside the eyepiece lens 950 or near the temple side edge, although other locations are also possible.

[0116] Figures 11A-11E illustrate the operation of the exemplary imaging system 900 of FIG. 10. FIG. 11A shows the illumination source 960 emitting light 902 towards the internal coupling optical element 942 on the waveguide 940. As shown, the light 902 can be directed to be substantially perpendicular and incident on the eyepiece lens 950, although other angles are also possible. In some designs, the light source 960 is configured to emit collimated light into the eyepiece lens 950. As shown in FIG. 11B, the illumination light 902 can be coupled into the waveguide 940 via the internal coupling optical element 942. In some designs where the internal coupling optical element 942 comprises a diffractive optical element (e.g., a grating, a holographic element), the light incident thereon is diffracted at an angle greater than the critical angle of the waveguide, and the internally coupled light 904 is guided within the eyepiece lens 950 by total internal reflection (TIR). In some designs, the internal coupling optical element 942 may be configured to direct the light towards the coupling optical element 944. The internal coupling optical element 942 may be polarization selective. For example, the internal coupling optical element 942 can include a polarization selective direction conversion element such as a polarization grating like a liquid crystal polarization grating. FIG. 11C shows the internally coupled light 904 propagating through the waveguide 940 by TIR.

[0117] FIG. 11D illustrates an exemplary imaging system 900 that couples light out from the eyepiece 950. As the internally coupled light 904 propagates through the waveguide 940, a portion of the light may be incident on the coupling optical element 944. The coupling optical element 944 can be configured to couple the internally coupled light 904 out from the eyepiece 950 towards the user's eye 210. The coupling optical element 944 may be configured to couple the light towards the eye 210 as collimated light. The coupling optical element 944 may be tuned to light within a particular wavelength range. For example, the coupling optical element 944 may be configured to couple infrared light (e.g., from about 700 nm to 15,000 nm) out from the waveguide 940. In some designs, the coupling optical element 944 can be configured to couple multiple wavelengths of light out from the eyepiece 950. For example, the coupling optical element 944 may be tuned for both infrared and visible light. The coupling optical element 944 can also be configured to couple light into the waveguide 940, as more fully described below.

[0118] The coupling optical element 944 can be configured to increase the dimensions of one or more eye boxes for the user. For example, one or more dimensions may be measured along a first axis (e.g., the x-axis). The eyepiece 950 may further include an orthogonal pupil expander (OPE). The OPE may have at least one light redirecting element disposed on or within the waveguide (e.g., on one of the major surfaces), or the OPE may be disposed within the waveguide 940. The OPE may include features similar or identical to those described above with respect to the light dispersing elements 730, 740, 750. In some implementations, the light redirecting element may comprise a diffractive optical element. The OPE may be configured to increase the dimensions of the eye box along a second axis (e.g., the y-axis) orthogonal to the first axis.

[0119] FIG. 11D shows a state where a part of light is emitted from the eyepiece lens 950 toward the user's eye 210. In some designs, the coupling optical element 944 is configured such that the internally coupled light 904 incident on the coupling optical element 944 at various portions of the coupling optical element 944 along a first axis (e.g., parallel to the x-axis) exits from the eyepiece lens 950 at each portion of the coupling optical element 944 along the first axis. This can provide light for the user to project an image or illuminate the eyes for different eye positions or locations.

[0120] As shown in FIGS. 11D - 11E, the coupling optical element 944 may be configured to couple the internally coupled light 904 as collimated light out of the eyepiece lens 950. This light can also generally be directed substantially perpendicular to the major surfaces of the eyepiece lens 950 and / or the waveguide 940. The collimated light is directed into the eye and can be focused onto the retina by the eye (e.g., the cornea and natural lens of the eye). The light 908 incident on the retina can form an image on the retina and / or provide illumination for providing image content to the eye. A part of the light 908 can, for example, be reflected or scattered from the retina, exit the eye, and provide an image of the retina to be captured. The light source 960 may be an extended light source such that the light will illuminate an area of the retina.

[0121] FIGS. 12A - 12E illustrate a state where the imaging system 900 of FIGS. 11A - 11E can be used, in addition to, or alternatively, for image collection of the eye 210. FIG. 12A shows a state where the light 910 reflected from the retina exits the eye 210. As shown, the light 910 scattered or reflected from the retina and passing through the natural lens of the eye, the pupil within the eye, and the cornea can be collimated. This light can also be incident on the eyepiece lens 950 at normal incidence (e.g., perpendicular to the major surfaces of the waveguide 940 and / or the coupling optical element 944). The coupling optical element 944 may be configured to couple the light 910 reflected from the retina into the waveguide 940.

[0122] FIG. 12B illustrates an exemplary imaging system 900 as light is coupled into the eyepiece lens 950. The coupling optical element 944 may include a direction-changing feature such as a diffractive optical element or other structure that redirects light at an angle greater than the critical angle so as to be guided within the waveguide 940. The coupling optical element 944 may be configured to direct the internally coupled light 914 generally towards the light source 960 and / or the imaging device 920. The coupling optical element 944 can be configured to couple a portion of the light traveling towards the camera 920 that is less than a certain rate back out of the waveguide 940. For example, a partially reflective element (e.g., a semi-transparent mirror) may be disposed on or within the waveguide 940 such that a portion of the internally coupled light 914 leaks out of the waveguide 940 along the portion of the waveguide 940 where the coupling optical element 944 is disposed, while a portion of the internally coupled light 914 continues to propagate within the waveguide 940 by total internal reflection. The portion of the light that does not leak may be any rate from 0 to 1. For example, the portion may be 0.90, and 90% of the light rays that propagate through the waveguide 940 along the coupling optical element 944 are maintained within the waveguide 940 at each reflection of the light rays. Other portions are also possible (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or any range between any of these values). Such a partially reflective element can similarly be used in the implementations described below.

[0123] As shown in FIG. 12C, the collimated internally coupled light 914 can continue to propagate through the waveguide 940 towards the imaging device 920. FIG. 12D shows how a portion of the internally coupled light 914 can continue to propagate until it impinges on one or more external coupling optical elements 952. To reduce the amount of leakage of the internally coupled light 914 out of the internal coupling optical element 942, the internal coupling optical element 942 can be configured to couple little of the main light propagating towards the camera 920 back out of the waveguide. For example, a partially reflective element (e.g., a semi-transparent mirror) can be disposed on or within the waveguide 940 such that a portion of the internally coupled light 914 continues to propagate within the waveguide 940 by total internal reflection while reducing leakage of the internally coupled light 914 out of the waveguide 940 along the portion of the waveguide 940 where the internal coupling optical element 942 is disposed. The portion of the light that does not leak can be any ratio from 0 to 1. For example, the portion can be 0.90, and 90% of the light rays that propagate along the coupling optical element 944 through the waveguide 940 are maintained within the waveguide 940 at each reflection of the light rays. Other portions are also possible (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or any range between any of these values). Such partially reflective elements can likewise be used in the implementations described below.

[0124] As shown in FIG. 12E, the external coupling optical element 952 can be configured to couple light induced within the waveguide 940 from the waveguide 940 to the imaging device 920 outside. As a result, the light propagating within the waveguide 940 that is incident on the external coupling element 952 can be redirected to be emitted from the waveguide 940 to the outside, for example, from the major surface of the waveguide 940 to the outside (e.g., to the front or rear side of the waveguide 940) and be directed onto the imaging device 920. The external coupling optical element 952 may be configured to direct the light 926 to exit the waveguide 940 perpendicular to (e.g., vertically with respect to) the major surface of the waveguide 940. In some designs, the external coupling optical element 952 is configured to direct the collimated light 924 onto the imaging device 920 at normal incidence with respect to the photosensitive portion of the imaging device 920. As discussed above, the camera 920 can be focused at infinity, and for example, the imaging optical system can be configured to focus the collimated light onto the detector array.

[0125] Therefore, the waveguide 940 may be configured to guide the light coupled from the user's eye 210 to be received by the imaging device 920 (e.g., a camera) so as to capture at least a partial image of the user's eye 210. The same waveguide 940 may be configured to guide the light coupled from the image projector 930 so that the light from the image projector 930 can be directed to the user's eye 210 such that the image from the image projector 930 is within the user's field of view. In some implementations, the same waveguide is configured to guide the light coupled from the illumination source 960 so that the light from the illumination source can be directed to the user's eye 210 to illuminate the eye and the image of the eye can be captured by the camera 920.

[0126] In some implementations, the same coupling optical element 944 can be configured to (i) couple light from the user's eye 210 into the waveguide 940 so as to be received by the imaging device 920, and (ii) couple light from the image projector 930 out of the waveguide 940 to the user's eye 210 to project image content into the user's field of view. In some implementations, the same coupling optical element 944 can be configured to couple light from the illumination source 960 out of the waveguide to the user's eye 210 such that light from the illumination source can illuminate the eye.

[0127] In other designs, different waveguides can be used and / or different coupling optical elements 944 can be used. In some designs, for example, the first waveguide 940 may be configured to direct light coupled from the user's eye 210 so as to be received by the camera 920 to capture at least a partial image of the user's eye 210, and the second waveguide may be configured to direct light coupled from the image projector 930 such that light from the image projector 930 is directed to the user's eye 210. The first and second waveguides may be stacked on top of each other. Another waveguide may, in addition or alternatively, be configured to direct light coupled from the illumination source 960 such that light from the illumination source is directed to the user's eye 210 and can illuminate the eye.

[0128] Also, in some implementations, the first coupling optical element 944 can be configured to (i) couple light from the user's eye 210 into the waveguide 940 so as to be received by the imaging device 920, and (ii) couple light from the image projector 930 out of the waveguide 940 to the user's eye 210 to project image content into the user's field of view. Another coupling optical element may, in addition or alternatively, be configured to couple light from the illumination source 960 out of the waveguide to the user's eye 210 such that light from the illumination source can illuminate the eye.

[0129] In some designs, the coupling optical element 944 can include a plurality of diffractive optical elements (DOEs). For example, the first DOE can be configured to couple light from the user's eye 210 into the waveguide 940 so that it is received by the imaging device 920. The second DOE can be configured to couple light from the image projector 930 out of the waveguide 940 and into the user's eye 210 to project image content into the user's field of view. Optionally, the third DOE can be configured to couple light from the light source 960 out of the waveguide 940 and into the user's eye 210 to illuminate the eye. The first and second (and possibly, the third) DOEs can be stacked, for example, in some implementations, such that light from the user's frontal environment passes through the first DOE, then impinges on the second DOE, then impinges on the third DOE, and then impinges on the user's eye. However, the order can be different.

[0130] In some designs, the first and second DOEs are integrated within a single element or volume of the waveguide 940. In some implementations, for example, both the first and second DOEs are superimposed on one another within the waveguide 2102 (e.g., occupy the same or substantially the same volume). For example, the first and second DOEs may be recorded in the same medium.

[0131] As described above, image capture of the eye, e.g., the retina, can facilitate eye tracking. FIG. 13A illustrates an imaging system 900 configured to image various portions (e.g., the retina) of an eye 210 at different times when the eye is in different positions. Stages A and B can refer to images of the eye 210 between different eye orientations. FIG. 13A shows the image of the eye 210 and the results between both the imaging of stage A and stage B.

[0132] In some implementations, light emission 928 (e.g., from illumination source 960 as described above, or from one or more illumination sources configured and / or positioned differently) can be used to acquire one or more images of retina 962, as shown by FIG. 13A. The image of retina 962 can comprise one or more regions 964, 966 that are imaged during different orientations of eye 210. FIG. 13A shows two regions 964, 966 of an image of retina 962. For example, the region 964 of the retina imaged at stage A can be imaged while eye 210 is directed at an angle perpendicular to waveguide 940. Image data regarding region 966 imaged at stage B can be acquired while eye 210 is oriented at an acute angle with waveguide 940. During one or more stages of imaging, one or more orientations of eye 210 can be used to acquire a composite image or map of retina 962. Processing electronics or a processor, such as data module 140 (see FIG. 2), can be used to find overlapping image data between two adjacent regions. Using the image data of the overlapping regions, a composite image of retina 962 can be determined. A composite image or map of a larger size (e.g., full size) of the user's retina can be stored.

[0133] As described herein, a head-mounted display can be used to map a user's eye retina based on the direction in which the user's eyes are directed. Using eye gaze to provide a realistic and intuitive interaction with objects in the user's environment and / or to identify the wearer of a head-mounted display device, a head-mounted display system can incorporate retinal mapping to account for the uniqueness of the user's eye features and other conditions that can affect eye measurements. For example, an image can be identified based on the position of blood vessels within the corresponding retinal image.

[0134] Retinal mapping can involve a process for a computing device to learn a way to associate a user's line of sight (e.g., as identified within a retinal image) with a line-of-sight point in 2D or 3D space. The line of sight may be associated with a single point in 2D or 3D space. The line of sight can also be associated with multiple points in space, which can account for the movement of virtual objects (e.g., a series of points, the location of a moving image).

[0135] A head-mounted display system can determine a user's line of sight based on a retinal image. The head-mounted display system can use a sensor (e.g., an eye camera such as imaging device 920) to obtain a retinal image. The head-mounted display system can image one or both of the user's eyes while the user is changing their line of sight (e.g., when the user is looking around to follow a moving or offset calibration target or a fixed target). To map the user's retina, the head-mounted display system can present a virtual target, e.g., a fixed target, for the user to view. The virtual target may be associated with one or more known points of the line of sight in 2D or 3D space. While the user is looking at the target, the head-mounted display system can obtain a retinal image and associate the image with the line-of-sight point. The head-mounted display system can calculate and / or generate a mapping matrix based on the association of individual retinal images with the line-of-sight points associated with the targets.

[0136] The retinal mapping results can reflect the uniqueness within each person's eye. For example, a head-mounted display system can generate a mapping matrix customized for one or both eyes of a specific individual. For example, a user may have different amounts of eye movement or eye gaze in response to a specific target. Additionally, or alternatively, a user may have different positions, sizes, shapes, and / or orientations of blood vessels within the retina. As a result, by generating calibration results specific to an individual user, the head-mounted display system can enable more accurate user interaction with eye gaze and / or enable the identification of a specific user.

[0137] Therefore, when a user wears a head-mounted display device, the system can detect whether the user is a previous user or a new user. A confusion matrix can be calculated, and a score regarding a specific eye gaze image stored in the system memory is compared with the corresponding image of the current user. The confusion matrix can include comparison scores regarding multiple eye gazes and associated retinal images. Based on the comparison scores, the system may be able to make a determination regarding the identification of the user (e.g., whether the user is the same as the individual associated with the stored retinal image or composite map) and / or the confidence level regarding the determination. The confidence level may include, for example, an identification coefficient. The stored image, e.g., a composite image or map, may be compared with a later-acquired image, referred to as an instantaneous or real-time image regarding the current user. The system may provide an alert or perform other actions if the system detects that the user is a new user.

[0138] The system may apply filtering such as digital filtering or image processing to the retinal images captured by the camera. Such filtering or imaging processing may enhance features that can be used, for example, for identification, stitching, assembly of composite images, eye tracking, etc. Such filtering or image processing may include edge enhancement. Such a filter may include, for example, a Frangi filter, although other types of filters may be used. Such a filter or process (e.g., edge enhancement or Frangi filter) can be used to improve and / or detect image features such as blood vessels or tubular structures or fibers within the retinal image.

[0139] Figure 13B illustrates a pattern of fixation targets that are sequentially presented and can be used in the retinal mapping process. These virtual targets, at which the user's eye will direct its line of sight, can redirect the eye's line of sight in various different directions, during which the retina can be imaged. The images resulting as a consequence of the different line of sight directions correspond to different parts of the retina. As discussed above, when the eye fixates on different directions and visually recognizes fixation targets positioned differently on the display, the images captured by the camera include different parts of the retina. These images can be assembled to form a larger map or composite image of the retina.

[0140] FIG. 13B shows virtual targets at 16 different locations within the user's field of view (FOV) 1200. In various implementations, the virtual targets will be presented at a given location at a given time. One or more retinal images will be acquired while the virtual target is presented to the user at that particular location. This image or these images may be associated with the target location and / or the corresponding line-of-sight direction. More or fewer target locations may be used. In the example shown in FIG. 13B, 16 target locations 1202a - 1202p are shown. More or fewer target locations may be used. The target locations may also differ. The order in which the targets are presented at different locations may vary. For example, the targets may move in a raster pattern from the left to the right side of the user's field of view, then in reverse, from the right to the left side, and again from the left to the right side, lowering the position of the targets within the field of view, with each lateral pass traversing the field of view. However, other patterns and approaches are also conceivable. Similarly, the targets can be rendered at different locations in the same or different ways. For example, the rendered targets can differ in size, shape, color, etc. The targets can be sequentially rendered to the user during the eye-tracking calibration process. For example, as discussed above, the head-mounted display system may render the targets in a serpentine pattern. For example, after target 1202a, 1202b, then 1202c, then 1202d, then 1202h, then 1202g, etc. may follow. Other patterns are also conceivable. For example, the targets can be displayed in a more random or non-sequential pattern. In some embodiments, a single target is displayed to the user and the target moves around the user's field of view (e.g., passes through or temporarily stops at positions 1202a - 1202p during the movement of the target). The head-mounted display system can obtain an image of the user's retina while the user is looking at these targets.For example, a head-mounted display system can obtain a first image when the user is looking at a target at a first location 1202a, a second image when the user is looking at a target at a second location 1202b, a third image when the user is looking at a target at a third location 1202c, and so on. A wearable system can associate the first image with the first location 1202a, the second image with the second location 1202b, the third image with the third location 1202c, and so on. The nearby images can be stitched together in a database to create a complete or partial retinal map. For example, two images can be stitched together at an appropriate alignment using a feature or a part of a feature (e.g., a blood vessel or a part thereof) common to the plurality of images. In various implementations, adjacent target positions will produce overlapping images that can be aligned and stitched together. For example, target position 1202a and target position 1202b and target position 1202b and target position 1202c can produce overlapping and adjacent retinal images that can be stitched together with each other. Thus, several different retinal images can be obtained using different eye gazes to assemble a larger image (e.g., a composite image or a map) of the retina.

[0141] As discussed above, eye tracking can be performed using a synthetic retinal image or map. For example, after a target is no longer displayed, the user can move their line of sight as the user views different real objects in front of the user and the head-mounted display or augmented reality (virtual) image content displayed by the head-mounted display. One or more retinal images can be acquired at these times. The terms "instantaneous" or "real-time" images may be used herein to describe these images acquired following calibration that can be used for eye tracking (or other purposes such as acquiring biometric data). These "instantaneous" or "real-time" images are likely to correspond to a part of the synthetic retinal image or map. The system may be configured to sufficiently match such "instantaneous" or "real-time" retinal images with a part of the synthetic retinal image or retinal map. Such matching may be based on common features or parts of features (blood vessels or parts thereof) in both the "instantaneous" or "real-time" retinal image and the part of the synthetic retinal image or map. Based on the location of the part of the synthetic retinal image or map with which this "instantaneous" or "real-time" retinal image coincides, the direction of the line of sight can be estimated. Different directions of the line of sight will result in retinal images corresponding to different parts of the retinal map. Thus, identifying the location of the "instantaneous" or "real-time" retinal image on the synthetic retinal image or map will provide information regarding the direction of the user's line of sight. Eye tracking, e.g., tracking eye movements and changes in the line of sight, may be performed using such or similar methods. As discussed above, edge enhancement, edge detection, or other digital filtering and / or processing may be used to enhance features of different images and / or correlate them with the synthetic retinal image or retinal map.

[0142] In various implementations, virtual or fixed targets are displayed (e.g., at multiple locations) and, after completion of an initial calibration process that assembles a synthetic retinal image or map, the synthetic retinal image or map can still be refined. For example, as additional retinal images are acquired, the synthetic retinal image or map can be further refined or improved using the additional images. Thus, as additional “instant” or “real-time” retinal images are acquired, e.g., for the purpose of providing eye tracking, the synthetic retinal image or map can be further refined or improved using the “instant” or “real-time” images. As the user continues to look at various locations within the display (with or without the aid of calibration targets), the synthetic retinal image or map can be further refined using additional images obtained subsequent to the initial calibration where virtual or fixed targets were displayed. The quality of the synthetic retinal image or map can thus be increased.

[0143] Eye tracking can be performed and / or a synthetic retinal image or map can be produced. An additional non-limiting example of a method in which retinal images are used is described in U.S. Patent Publication No. 2017 / 0205875, filed on January 17, 2017, entitled “EYE IMAGE COLLECTION” (which is hereby incorporated by reference in its entirety).

[0144] Thus, as discussed above, a larger portion of the retina can be recorded and mapped by obtaining a retinal image and / or other images of the eye using an imaging system as described herein, and such images can facilitate eye tracking. For example, an image of the eye 210 shown in FIG. 13A may be captured when the eye is in an arbitrary position. A processing electronic device or processor (such as the same or different one as described above for forming a composite image) can then compare a real-time captured image of the user's retina with a stored composite or larger size (e.g., full size) image of the user's retina and track eye movement. A given image of the user's retina captured in real time can indicate a specific portion of the user's retina. As described above, by comparing such a captured image with a stored image of the mapping of a larger portion of the user's retina, the system can determine the portion of the user's retina shown in the captured image, thereby determining the eye position / orientation that would produce such an image. See, for example, FIG. 13A showing two different images of a portion of the retina produced when the eye is in two different positions and / or orientations. Thus, the eye position and / or orientation can be determined by capturing different images of the retina and determining the visible portion of the retina. Such determination can be performed even when no composite image is formed; rather, a plurality of images of the retina for different eye positions / orientations are recorded and stored in a database. When a future image of the retina is acquired, that image can be compared with the images in the database of stored images to determine an image in the database that is similar to the most recently acquired eye image. Matching the most recent image to one or more of the images in the database having the associated position and / or orientation associated therewith can enable determination of the orientation and / or position of the more recent image. Other approaches to eye tracking may also be used based on images captured using the designs described herein.

[0145] As described herein, retinal images may likewise be employed for other purposes. For example, a retinal image may be used to verify that the user is the same user for whom a synthetic retinal image or map was acquired. A retinal image acquired when the user is wearing a head-mounted display system (e.g., during and / or after a calibration process) may be compared to a previously acquired (e.g., created the previous day or when the head-mounted display was previously booted up) synthetic retinal image or map that is stored. If the recently acquired retinal image does not sufficiently match a portion of the synthetic retinal image or map, it may be concluded that the current user is different from the previous user (e.g., for whom the synthetic virtual image or map was created). Such a method may be used for security purposes, e.g., to verify that the current user of a head-mounted display device is the owner or typical user of the device. Thus, biometric data acquired via retinal imaging may be used for security purposes.

[0146] Retinal imaging may likewise be used to collect biometric data for monitoring the user's health. Medical-related data may be obtained from the retinal image. Such medical data may be useful for monitoring the user's health.

[0147] Various uses of eye imaging, such as for eye tracking, health monitoring, and collection of biometric data for security, are discussed herein in the context of retinal imaging, but imaging of other parts of the user, e.g., imaging of the user's eyes, may also be employed for these and other purposes.

[0148] As described above, the eyepiece 950 can be used to facilitate imaging of the eye, but the eyepiece can also be used to image the world in front of the user. FIGS. 14A-14B illustrate, for example, an exemplary imaging system 900 that can be used to image a portion of the environment in front of the user and / or an object within a portion of the environment. The imaging system 900 used can be a similar system to that described with respect to FIGS. 11A-11E and / or FIGS. 12A-12E, except that light is collected by the eyepiece 950 from the eyepiece and the environment in front of the user. FIG. 14A illustrates, for example, light 970 from the environment that is reflected by and / or emitted by one or more physical objects 972 within the environment in front of the user and the eyepiece 950. As shown, the light 970 from the environment can be substantially collimated (e.g., to infinity) because, for example, the physical object 972 within the environment is located at a sufficient distance from the imaging system 900 such that the light rays reaching the imaging system 900 are collimated or substantially collimated. In some implementations, the imaging system 900 may be configured to image the environment and / or an object within the environment without using any optical element (e.g., lens, mirror) having refractive power within the imaging system 900.

[0149] The imaging system 900 shown in FIGS. 14A and 14B is similar to the imaging systems described above. The imaging system includes one or more waveguides 940 that include a coupling optical element 944 configured to direct light from an image projector 930 (not shown) into the eye 210 and form an image therein, and an eyepiece 950. The one or more waveguides may include a plurality of corresponding waveguides (e.g., a stack of waveguides) configured to internally / externally couple a plurality of corresponding colors / wavelengths. Each waveguide in the stack of waveguides may be configured to direct light of a particular color (e.g., red, green, blue). For example, the most distal waveguide (e.g., the stack of waveguides) may be configured for visible light (e.g., red, blue, green) such that the waveguide is configured to internally and externally couple the same wavelength of visible light. Additionally or alternatively, a waveguide configured to internally and externally couple invisible (e.g., infrared) light may be disposed proximal to the eye 210. Such a plurality of waveguides corresponding to the waveguide 940 may be used in any other implementation described herein. The imaging system 900 may also include an imaging device (e.g., a camera) 920 and an external coupling optical element 952 configured to redirect light reflected from the eye 210 and propagated to the camera within the waveguide 940. In FIGS. 14A and 14B, the illumination source 960 is excluded because the illumination source may not be required to image the user's front environment. However, an illumination source (e.g., the light source 960 described above) may be used in some designs.

[0150] The eyepiece lens 950, the waveguide 940, the coupling optical element 944, the external coupling optical element 952, and the camera 920 may be the same as or similar to those described above. For example, the coupling optical element 944 may physically engage with the waveguide 940. For example, the coupling optical element 944 and / or the external coupling optical element 952 are arranged in the optical path between the environment in front of the eyepiece lens 950 and the camera 920 such that light from the environment is coupled into the waveguide 940 through the coupling optical element 944 and incident on the camera 210 (e.g., to form an image of at least a portion of the environment), and coupled out of the waveguide through the external coupling optical element. The coupling optical element 944 may be configured with a plurality of direction-changing features that deflect light induced within the waveguide out of the waveguide or deflect light incident on the coupling optical element 944 at an angle into the waveguide such that it is induced therein by total internal reflection. The external coupling optical element 952 may be configured with a plurality of direction-changing features that deflect light (from the environment) that is induced at an angle within the waveguide such that it is not induced within the waveguide by total internal reflection but is directed outward toward the camera. The coupling optical element 944, the external coupling optical element 952, and the associated direction-changing features may physically engage with the waveguide 940. For example, the coupling optical element 944 and / or the external coupling optical element 952 may comprise one or more holographic or diffractive optical elements (e.g., surface relief gratings) patterned (e.g., etched) within or on the waveguide 940. The coupling optical element 944 and / or the external coupling optical element 952 may comprise a layer disposed on the waveguide 940 or may be formed within the waveguide 940. For example, a volume holographic or diffractive optical element may be formed by changing the refractive index of the material that constitutes the waveguide or a layer disposed thereon. Thus, the coupling optical element 944 and / or the external coupling optical element 952 may be disposed within the volume of the waveguide 940 or a layer disposed thereon. Depending on the design, the coupling optical element 944 and / or the external coupling optical element 952 may be transmissive or reflective and may operate transmissively or reflectively.For example, the coupling optical element 944 and / or the external coupling optical element 952 may each operate either transmissively or reflectively and may include, for example, a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that redirects light transmitted through or reflected from it. The coupling optical element 944 and / or the external coupling optical element 952 can include a polarization optical element such as a polarization selective beam redirecting element (e.g., a polarizer). The polarization selective beam redirecting element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements and may include a liquid crystal structure such as a liquid crystal polarization grating. In some implementations, the reflective optical element may include a reflector (e.g., a mirror). For example, other elements such as the waveguide 940 may similarly be similar to those described above.

[0151] FIG. 14B illustrates the operation of the imaging system 900 shown in FIG. 14A. Light 970 from the environment is coupled into waveguide 940 by coupling optical element 944. The coupling optical element 944 may be configured to redirect the collimated light at an angle greater than the critical angle of the waveguide 940 such that at least a portion of the collimated light is guided toward camera 920 by total internal reflection within the waveguide. The external coupling optical element 952 may be configured to receive at least a portion of the light from the user's front environment that is coupled into and guided within the waveguide 940 via the coupling optical element 944. The external coupling optical element 952 may be configured to couple the internally coupled light out of the waveguide 940 and to the camera 920 such that an image of the environment can be captured by the camera 920. The image of the environment may be passed to a processing electronic device (e.g., one or more processors) such as data module 140 (see FIG. 2). The data module 140 may be configured to reproduce a modified image of the environment within an augmented reality context. The processing electronic device may communicate with the camera 920 via a wired or wireless electronic signal. Additionally, or alternatively, the processing electronic device may communicate with the camera 920 using one or more remote receivers. The processing electronic device may be resident remotely (e.g., a cloud computing device, a remote server, etc.).

[0152] This imaging system 900 may thus be used directly to image the environment, which may be useful for various reasons. For example, imaging the environment can be used to determine where to place augmented reality image content relative to objects within the environment. For example, imaging the environment can provide the location of a table such that a head-mounted display can render an image of a person standing next to the table rather than on or within the table. The imaging system 900 described with respect to imaging the environment may also be used to image an eye 210 as described with respect to FIGS. 10, 11A-11E, and / or 12A-12E.

[0153] It may be desirable to image a wide field of view of the environment using the imaging system 900. FIG. 14C schematically illustrates an imaging system 900 for collecting light from the environment using a refractive optical element or lens such as a refractive optical element 980 (e.g., a wide-angle lens) in front of the eyepiece. The refractive optical element 980 may have a positive refractive power. The refractive optical element 980 (e.g., a positive lens) converges the collimated light 970 from the environment towards the waveguide 940. Lenses of types other than those shown in FIG. 14C may be employed. The transmitted light (not shown) may pass through a refractive optical element or lens such as a refractive optical element 990 (e.g., a negative lens) configured with an equal but opposite negative refractive power of the refractive optical element 980. The negative lens 990 has a refractive power similar or identical to that of the positive lens 980 and may offset or cancel the refractive power of the positive lens or a portion thereof. Thus, light from the environment (e.g., distal to the waveguide 940) may pass through the negative lens 990, the eyepiece 950, and the positive lens 980, and substantially not cause a net change in the refractive power introduced into the eye by these two lenses. The negative lens 990 may be configured to offset or cancel the refractive power of the positive lens 980 such that when the user views the environment in front of the eyepiece 950, the refractive power of the positive lens will not be masked. The negative lens 990 may also cancel the effect of the positive lens 980 and invert the image of an object within the environment in front of the wearer. A portion of the light 970 from the environment may be internally coupled into the waveguide 940 by the coupling optical element 944, even though a portion of the light rays converges. The internally coupled light incident on the external coupling optical element 952 may be emitted out of the waveguide 940.

[0154] The implementation (e.g., as described by FIGS. 14A - 14C) may be used outside of an extended reality context. For example, an imaging system 900 configured to image an environment is intended to be implemented within a wearable device such as, for example, glasses (including non - refractive glasses) or bifocal glasses. Such an imaging system 900 may not require an image projector 930 and / or a light source 960. Additionally, or alternatively, such an imaging system 900 may not require internally coupled optical elements configured for a corresponding image projector 930 and / or light source 960.

[0155] It may be advantageous to implement such an imaging system 900 to image an environment onto a viewing screen (e.g., a TV screen, a computer screen) of a hand - held device (e.g., a mobile phone, a tablet). The imaging system 900 may improve video chat capabilities. For example, when a viewer looking at the screen sees the chat partner, the chat partner may appear to be looking directly at the viewer. This may be possible because the light rays captured by the imaging system 900 will be captured within the same area that the user is looking at (e.g., in contrast to viewing a screen having light rays captured by separate outward - facing cameras located in different locations).

[0156] In an implementation where the imaging system 900 of FIG. 14C is also used to image the eye 210, the light source 960 and / or the image projector 930 may be configured to direct light into the waveguide 940. Since the light reflected from the eye, which is internally coupled within the waveguide, will pass through the refractive optical element 990 (e.g., a negative lens), a positive refractive power type refractive optical element may be disposed between the light source 960 and / or the image projector 930 and the waveguide 940. The positive lens can be configured to offset or cancel any refractive power provided by the refractive optical element 990 before the internally coupled light from the light source and / or the light projector is incident on the eye 210. Lenses of types other than those shown in FIG. 14C may also be used as the optical element 990. Alternatively, or in addition, processing electronics that communicate with the light source and / or the image projector can be configured to modify the image enough so that a non-distorted image is presented to the user after the light has passed through the refractive optical element 990. Corresponding internal coupling optical elements, external coupling optical elements, and / or coupling optical elements may, in some designs, be configured to act on non-collimated light (e.g., divergent, convergent light).

[0157] In various implementations, the same waveguide 940 may be used to (i) propagate light from the eyepiece 950 and the user's frontal environment to the camera 940, and (ii) propagate light from the image projector 930 to the eye 210 for use in forming image content therein. Using the same waveguide 940 can simplify the system and / or the eyepiece, make the system and / or the eyepiece more compact, and potentially provide a reduced form factor. Reducing the thickness of the eyepiece 950 by reducing the number of waveguides 940 can also be advantageous for other reasons. Lower cost and a more simplified manufacturing process are some such advantages.

[0158] Also, in various designs, the same or different imaging systems are used within the same head-mounted display. For example, as described above, the light from the eye may be propagated to the camera 940 through the waveguide within the eyepiece 950 to image the eye. Such a system may also use the eyepiece to transfer light from the illumination source to the eye 210 to illuminate the eye. In some designs, the eyepiece may additionally be used to propagate light from the image projector 930 to the eye 210 to form image content therein. Assisting in imaging the environment and the eye (and potentially illuminating the eye) using the eyepiece can simplify the system and / or make the system more compact, and potentially provide a reduced form factor.

[0159] Furthermore, in some implementations, the same waveguide 940 may be used to (i) propagate light from the environment in front of the eyepiece 950 to the camera 940, (ii) propagate light from the eye 210 to the camera to capture an image of the eye. The same waveguide may be used to propagate light from the image projector 930 to the eye 210 to form image content therein and / or to propagate light from the illumination source 960 to the eye 210 to illuminate the eye for image capture. Using the same waveguide 940 can simplify the system and / or the eyepiece, make the system and / or the eyepiece more compact, and potentially provide a reduced form factor. Reducing the thickness of the eyepiece 950 by reducing the number of waveguides 940 can also be advantageous for other reasons. Lower cost and a more simplified manufacturing process are some such advantages.

[0160] Similarly, in addition to coupling light from the environment into waveguide 940, the same coupling optical element 944 may be configured to direct light from image projector 930 into eye 210, form image content therein, and / or direct light from the eye into waveguide 940 so as to be directed into camera 920 therein. Additionally or alternatively, the same coupling optical element 944 may be configured to couple light from illumination source 960 that is guided within waveguide 940 out of the waveguide to the user's eye 210.

[0161] As discussed above, one or more of coupling optical element 944, internal coupling optical element 942, or external coupling optical element 952 may comprise a polarization selective coupling element. Thus, in various designs, the light input into eyepiece 950 or waveguide 940 is polarized such that it is appropriately acted upon by a polarization selective direction conversion element.

[0162] Thus, in some embodiments, illumination source 960 comprises a polarization source of a suitable polarization such that it is appropriately acted upon by a polarization selective coupling / direction conversion element.

[0163] One or more polarization-specific optical filters and polarization modification elements may be included within various imaging systems 900 such as those where the image projector 930 and / or the light source 960 are arranged to face each other through the waveguide 940. The polarization sensing elements may be useful in reducing the directed light emission into the imaging device 920 and / or, for example, in reducing the saturation of the imaging device 920 in configurations where these elements are aligned on the opposite side of the waveguide 940 at the same lateral position. FIGS. 15A-15B illustrate such a configuration. The light source 960, as shown in FIG. 15A, may be configured to direct light through a polarization-specific optical filter 982, such as a polarizer (e.g., a linear polarizer), and / or through a polarization modification element 986 configured to modify the polarization state of the incident light, such as a polarization rotator. A retarder, such as a half-wave retarder, may, for example, be able to rotate linear polarization. Thus, a properly oriented half-wave retarder or half-wave plate may be able to rotate s-polarization to p-polarization or vice versa. Thus, in various implementations, the polarization-specific optical filter 982 and / or the polarization modification element 986 are arranged within the optical path between the light source 960 and the internal coupling optical element 942 so as to provide properly oriented polarization to the internal coupling optical element. In some implementations, the imaging system 900 does not include a polarization modification element but includes a properly oriented polarization optical filter, such as a polarizer.

[0164] The light emitted by the light source 960 may pass through an array of optical elements in a particular order. For example, as shown in FIG. 15A, the light may first pass from the light source 960 through the polarization-specific optical filter 982 (e.g., a polarizer) and then through the polarization modification element 986 (e.g., a rotator). After the light passes through the polarization modification element 986, the light may be incident on the internal coupling optical element 942, which may direct the light into the waveguide 940 as induced therein.

[0165] For example, the light source 960 may be configured to emit light of mixed polarization (e.g., s-polarization and p-polarization). The polarization-specific optical filter 982 may be configured to transmit only light of a first polarization state (e.g., p-polarization). As the light continues, the polarization modification element 986 may be configured to change the polarization state of the light (e.g., from p-polarization to s-polarization). The internal coupling optical element may be configured to redirect the s-polarization at an angle greater than the critical angle of the waveguide so that the s-polarization is induced within the waveguide. The internally coupled light 904 may be substantially polarized within the second polarization (s-polarization) as it propagates through the waveguide 940. The coupling optical element 944 may be configured to redirect only light of the second polarization state (s-polarization). The coupling optical element 944 may be configured to couple the internally coupled light 904 from the waveguide 940 out to the eye 210 to provide illumination for image capture.

[0166] To prevent direct illumination (e.g., saturation) of the imaging device 920, a polarization correction element 958 and / or a polarization-specific optical filter 984 may be disposed within or on the waveguide 940 such that only light of a certain polarization state (e.g., p-polarization) can pass through the polarization-specific optical filter 984 to the imaging device 920. The polarization correction element 958 (e.g., a half-wave plate) may be configured to change the state of polarization (e.g., from s-polarization to p-polarization). The polarization-specific optical filter 984 may be configured to transmit only light of a certain polarization (e.g., p-polarization) therethrough. Thus, light passing through the polarization-specific optical filter 982 will not be configured to pass directly through the polarization-specific optical filter 984. In any of the above implementations (e.g., the image projector 930 and / or the light source 960 are on the same optical axis as shown in FIG. 15A), the configuration of the polarization-specific optical filter 982, the polarization correction element 986, the internal coupling optical element 942, the polarization correction element 958, and / or the polarization-specific optical filter 984 may be implemented according to the design of FIG. 15A. The polarization-specific optical filter 984 may be a transmissive-reflective polarizer (e.g., a polarizer beam splitter) configured to transmit light of a first polarization and redirect or reflect light of a second polarization different from the first.

[0167] A partially reflective element (e.g., a semi-transparent mirror) may be included to redirect internally coupled light 904 to imaging device 920. The partially reflective element may be disposed between internal coupling optical element 942 and polarization correction element 986 such that a portion of the internally coupled light 914 is reflected toward imaging device 920 while reducing leakage of the internally coupled light 914 out of waveguide 940. The portion of light that does not leak may be any ratio from 0 to 1. For example, the portion may be 0.90, and 90% of the light rays propagating through waveguide 940 along coupling optical element 944 are maintained within waveguide 940 at each reflection of the light rays. Other portions are also possible (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or any value within the range between these values).

[0168] FIG. 15B illustrates the propagation of light reflected or scattered from the retina. A portion of the light 910 reflected from the retina, having a second polarization (s-polarization), incident on coupling optical element 944 is redirected by coupling optical element 944 at an angle greater than the critical angle of waveguide 940 and thus may be induced therein. A portion of the light may not be coupled into waveguide 940 and will transmit therethrough as uninternally coupled light 912. The internally coupled light 904 may propagate through waveguide 940 toward the camera.

[0169] Other implementations may also benefit from the use of polarization-selective elements proximal to the light source and camera. For example, various systems can be configured to provide illumination having a first polarization and capture an image with a camera using light having a different polarization. For example, such a configuration may be used to reduce unwanted reflections from the cornea, etc. when imaging the retina. The reflection from the cornea will be specular. Thus, when light having a first polarization is incident on the cornea, the light reflected from the cornea will retain that first polarization. In contrast, the retina is diffusive. When light having a first polarization is incident on the retina, the light reflected from the retina will not retain only the first polarization. Diffuse reflection is more likely to result in unpolarized light. Thus, a second polarization different from the first polarization will be present in the reflected light. Similarly, by illuminating with a first polarization and imaging with a second different polarization, the retina can be imaged with reduced glare from the cornea.

[0170] Thus, in various implementations, both polarization-specific optical filters 982, 984 may be used to reduce unwanted light reflected from the eye 210 (e.g., from the cornea). For example, unwanted light, glare, or flash that may saturate the image captured by the imaging device 920 can be reflected from the cornea. The light reflected from the cornea is specular and can maintain its polarization. In contrast, the light reflected from the retina can be more diffusely reflected and not be as homogeneously polarized. Similarly, a combination of polarizers may be used to remove some or most of the unwanted reflected light. First, polarization can be used to illuminate the eye 210. In some designs, a polarized illumination source (e.g., light source 960) may be used. Additionally, or alternatively, a first polarizer (e.g., polarization-specific optical filter 982) may be positioned at the start of the optical path of the illumination source to provide an initial polarization of the light. A second polarizer (e.g., polarization-specific optical filter 984) may be positioned in the optical path before the light is incident on the imaging device 920. The second polarizer is 90 from the first polarizer oIt may be rotated (e.g., polarizers 982, 984 may "cross"). As a result, the eye is illuminated with a first polarization, and some of the light of the first polarization will be reflected from the cornea. This light will not pass through the proximal polarizer 984 of the camera. However, the light reflected from the retina will include a second polarization. Similarly, the light diffusely reflected from the retina will pass through the proximal polarizer 984 of the camera, enabling an image of the retina to be captured by the camera. Thus, in such a configuration, unwanted light received from the eye (e.g., from the cornea) that may be incident on the imaging device 920 can be reduced or eliminated. Other configurations are also conceivable. For example, a polarization-selective internal coupling optical element 942 for coupling light from the light source 960 into the waveguide 940 and a polarization-selective external coupling optical element for coupling the light out of the waveguide to the camera 920 may be employed to have different polarization selectivity properties. For example, the polarization-selective internal coupling optical element may selectively redirect light from an illumination source having a first polarization into the waveguide, while the external coupling optical element may selectively redirect light of a second different polarization out of the waveguide to the camera. The effect can again be to reduce or remove unwanted light received from the eye (e.g., from the cornea) before it is incident on the imaging device 920.

[0171] Using the eyepiece lens 950, various imaging systems 900 capable of collecting light and forming an image of the retina are discussed herein. However, the imaging system 900 can also be configured to image other parts of the eye, such as the anterior part of the eye. FIG. 16 illustrates how the imaging system 900 can be used to image the anterior part (e.g., the cornea) of the eye 210. The imaging system 900 may include one or more elements of the exemplary imaging system 900 described above. In addition, the exemplary imaging system 900 may include one or more refractive power type optical elements or lenses, such as refractive power type refractive optical elements 980, 990 having refractive power. For example, a positive refractive power lens or a positive lens 980 may be disposed proximal (e.g., closer to the eye 210) to the eyepiece lens 950 between the eye 210 and the eyepiece lens. A negative refractive power lens or a negative lens 990 may be disposed distal to the eyepiece lens 950 between the eyepiece lens and the user's forward environment. One or both of the lenses 980, 990 may be a variable focus element (e.g., a variable focus lens) and / or may include a liquid crystal element. In some designs, one or both of the lenses 980, 990 include a Fresnel lens. The lenses 980, 990 may incorporate liquid crystal and produce Fresnel lens functionality. Such functionality may enable variable focusing of one or both of the lenses 980, 990. In some designs, one or more of the lenses 980, 990 may be integrated with and / or manufactured (e.g., formed) on or in the eyepiece lens 950.

[0172] In various embodiments, the coupling optical element 944 is configured to redirect collimated light reflected from the eye 210 into the optical waveguide as induced therein. Thus, the positive lens 980 may be configured to collimate light reflected from the eye 210, such as the anterior part (e.g., the cornea) of the eye. The positive lens 980 may thus have a focal length equal to or substantially equal to the distance of the lens to the part of the eye 210 to be imaged, such as the cornea.

[0173] The negative lens 990 may have a refractive power similar to or the same as that of the positive lens 980 and may offset or cancel out the refractive power of the positive lens. Thus, light from the environment (e.g., the distal end of the waveguide 940) may pass through the negative lens 990, the eyepiece lens 950, and the positive lens 980, and not substantially affect the net change in refractive power introduced by these two lenses. Accordingly, the negative lens 990 may be configured to offset or cancel the refractive power of the positive lens 980 such that when the user views the environment in front of the eyepiece lens 950, the refractive power of the positive lens will not be masked. The negative lens 990 may also cancel out the effect of the positive lens 980 and invert the image of an object in the environment in front of the wearer.

[0174] FIG. 16 illustrates light 928 that is incident on and scattered from the cornea. The imaging system 900 may be configured to capture the direct light 988 reflected from the cornea. For example, the positive lens 980 may collect a portion of the light 988 scattered from the cornea and collimate the direct light 988. The direct light 988 collimated by the positive lens 980 is incident on the coupling optical element 944, which is configured to redirect the collimated light into the waveguide 940 at an angle greater than the critical angle of the waveguide such that the light is guided therein by TIR. The coupling optical element 944, the external coupling optical element 952, and / or the waveguide 940 may be as described above. The resulting externally coupled light 906 may be directed by the external coupling optical element 952 from the waveguide 940 out to a camera (not shown).

[0175] FIG. 16 shows light 928, such as collimated light, that can result from the eyepiece 950 as described above. The illumination source 960 can couple light into the waveguide 940, and the coupling element 944 can couple the main light from the illumination source 960 out of the waveguide. The coupling element 944 may be configured to couple the light out of the waveguide 940 as collimated light. The main light illuminates the front portion of the eye (e.g., the cornea) and scatters therefrom. As discussed above, this scattered light 988 can be collected by the positive lens 980 and the imaging system 900 to form an image of the front portion of the eye 210. Also, as discussed above, this illumination 928 directed onto the eye 210 may be invisible (e.g., infrared) light.

[0176] FIG. 16 also shows an alternative arrangement for illuminating the eye 210. In some designs, one or more light sources 934, such as LEDs or emitters, are disposed relative to the eye 210 and induced through the waveguide 940 by TIR and directed to shine light thereon without being directed onto the eye 210. In some implementations, the eyepiece 950 or the waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210. In some designs, a plurality of such light sources 934 may be arranged in a pattern (e.g., a circular or ring-shaped pattern) in the vicinity and / or around the eye. In some designs, the pattern of the light sources 934 may define an illumination axis parallel (e.g., coaxial) to the optical axes of the one or more lenses 980, 990. The one or more light sources 934 may be similar to the one or more light sources 960 described above and may, for example, be pulsed. Similarly, the one or more light sources 934 may comprise an infrared light source, such as an infrared LED, or another type of invisible light. Alternatively, the one or more light sources may comprise a visible light source that emits visible light. Or, the one or more light sources may emit both visible and invisible (e.g., infrared) light.

[0177] FIG. 17 illustrates another exemplary imaging system 900 configured to image a portion of an eye 210, such as the anterior portion of the eye (e.g., the cornea). The imaging system 900 shown in FIG. 17 employs a reflective optical element 996 configured to collimate light from the eye, as opposed to the transmissive optical element (lens) 980 shown in FIG. 16. Since chromatic aberration is generally not applicable to reflective optical elements such as the reflector 996 shown in FIG. 17, the reflective optical element will have less aberration than the transmissive optical element. Thus, by using a reflective surface when collecting light from the eye 210, less (e.g., chromatic) aberration is introduced into the captured image of the eye.

[0178] FIG. 17 illustrates an imaging system 900 that includes a curved transmissive optical element 996 having, for example, a wavelength-dependent reflective coating 998. The curved transmissive optical element 996 may be disposed distally (on the environmental side of the eyepiece 950) of the waveguide 940. Thus, the curved transmissive optical element 996 may be disposed between the wearer's forward environment and the waveguide 940 and / or the coupling optical element 944. Similarly, the waveguide 940 and / or the coupling optical element 944 may be disposed between the curved transmissive optical element 996 and the eye 210.

[0179] The wavelength-dependent reflective coating 998 may be configured to reflect light of a certain wavelength or range of wavelengths. In some implementations, for example, the wavelength-dependent reflective coating 998 may be configured to reflect non-visible light (e.g., infrared light) within a certain range of wavelengths, while the wavelength-dependent reflective coating 998 may be configured to transmit visible light. The wavelength-dependent reflective coating 998 may, in some cases, be disposed on the surface of the curved transmissive optical element 996.

[0180] As discussed above, in various designs, the coupling optical element 944 is configured to redirect the collimated light reflected from the eye 210 into the waveguide 940 as it is induced therein. Thus, the reflective optical element 996 may be configured to collimate the light reflected from the eye 210, such as from the anterior portion of the eye (e.g., the cornea). The curved reflective optical element 996 may thus have a positive refractive power with respect to the light incident on its proximal side that is reflected from the wavelength-dependent reflective coating 998. In particular, in various designs, the reflective optical element 994 may have a focal length equal to or substantially equal to the distance to the portion of the eye 210 to be imaged from the reflective optical element 996, such as the cornea, iris, etc. Exemplary values for the focal length may be, for example, from 2 cm to 8 cm. In some implementations, the focal length is from 4 cm to 6 cm. In some designs, the focal length is about 5 cm. The focal length may be within any range formed by any of these values, or in different designs, outside such a range.

[0181] In various implementations, the reflective optical element 996 is disposed on the distal side of the eyepiece lens 950 in front of the eyepiece lens. Thus, the reflective optical element 996 is disposed between the eyepiece lens 950 and the user's forward environment. Similarly, the eyepiece lens 950 is disposed between the reflective optical element 996 and the eye 210.

[0182] The curved transmissive optical element 996 may have a curved reflective surface with a curvature of any shape. In some implementations, the surface is rotationally symmetric. In some implementations, the surface may be spherical or aspherical (e.g., parabolic). Non-rotationally symmetric shapes are also conceivable as possibilities. However, in various designs, the reflective surface has a positive refractive power. The reflective optical element 996 may include, for example, a concave mirror for at least a certain wavelength and / or polarization.

[0183] The curved transmissive optical element 996 may be configured to have a refractive power that is negligible in transmission. Similarly, the curved transmissive optical element 996 may be configured to transmit light without introducing convergence or divergence. In one embodiment, the curved transmissive optical element 996 may have a curvature of the inner radius that is substantially the same as the curvature of the outer radius. The thin optical element 996 may, for example, reduce optical aberrations with respect to the light transmitted therethrough, may be lighter in weight, and / or may be more compact.

[0184] In various designs, the reflective optical element 996 includes a material that is transmissive to visible light such that the user can see the wearer's frontal environment. In some cases, to improve transmission, the curved transmissive optical element 996 may be coated with an anti-reflective coating on the outer surface (e.g., the distal surface). The anti-reflective coating may be configured to reduce the reflection of visible light, such as red, green, and / or blue light, for example. However, the reflective optical element 996 may be configured to reflect a portion of the light scattered from the eye 210 and form an image of the eye. Thus, the reflective optical element 996 may act differently for different lights. For example, the reflective optical element 996 may act differently for different wavelengths. The reflective optical element 996 may be configured to reflect infrared light and transmit visible light.

[0185] As discussed above, one or more light sources 934 may be configured to illuminate the eye 210 with infrared light. The light 988 resulting from reflection from the eye 210 (e.g., the cornea) may diverge as schematically illustrated in FIG. 17. The curved transmissive optical element 996 may be arranged to receive the light 988 reflected from the eye (e.g., the cornea, iris). The wavelength-dependent reflective coating 998 may be configured to reflect the light 988 reflected from the eye because the wavelength illumination used to illuminate the eye is the same wavelength (e.g., 850 nm) reflected by the reflective coating on the curved transmissive optical element 996. For example, the eye may be illuminated with infrared light (e.g., 850 nm), and the curved transmissive optical element 996 may be configured to reflect infrared light (e.g., 850 nm) and pass visible light. The shape of the curved transmissive optical element 996 may also be configured to collimate the light 988 reflected from the eye and redirect the collimated light into the waveguide 940 by being induced therein by TIR and reflect the light to the coupling optical element 944.

[0186] In FIG. 17, as in some other designs, one or more light sources 934, such as LEDs or emitters, may be arranged with respect to the eye 210 and induced through the waveguide 940 by TIR and direct the light thereon without being directed onto the eye 210. In some implementations, the eyepiece 950 or the waveguide 940 is not within the optical path between the one or more light sources 934 and the eye 210. In some designs, a plurality of such light sources 934 may be arranged in a pattern (e.g., a circular or ring-shaped pattern) near and / or around the eye. In some designs, the pattern of the light sources 934 may define an illumination axis parallel (e.g., coaxial) to the optical axes of one or more lenses 980, 990. The one or more light sources 934 may be similar to the one or more light sources 960 described above and may, for example, be pulsed. Similarly, the one or more light sources 934 may comprise an infrared light source such as an infrared LED or another type of invisible light. However, other types of light sources may also be used.

[0187] FIG. 18 illustrates another exemplary imaging system 900 configured to image a portion of an eye 210, such as the anterior portion of the eye (e.g., the cornea). In the implementation shown in FIG. 18, polarization selectivity is employed to assist in controlling the path of light reflected from the eye. In particular, in various designs, the coupling optical element 944 is polarization selective. For example, light having a first polarization is transmitted through the coupling optical element 944, while light having a second, different polarization is redirected into the waveguide 940 by the coupling optical element 944 so as to be coupled therein by TIR. Thus, in various implementations, a polarizer (not shown) is disposed between the eye and the waveguide 940 such that the eye 210 is illuminated with polarized light or the light from the eye incident on the waveguide is polarized. For example, the emitter 934 may emit polarized light, or a polarizer may be disposed in front of the emitter 934 such that the eye 210 is illuminated with polarized light. Thus, in various designs, the polarization of the polarized light incident on and / or reflected from the eye 210 received by the optical coupling element 944 may be a first polarization such that the light is directed towards the reflector 996.

[0188] Similarly, in various implementations, the coupling optical element 944 (and / or the external coupling optical element 952) is each configured to transmit light in a first polarization state, such as a first linear, circular, or elliptical polarization state (e.g., p-polarization, left-handed circular, or elliptical polarization, etc.), and redirect light in a second polarization state, such as a second linear, circular, or elliptical (e.g., s-polarization, right-handed circular, or elliptical polarization, etc.) polarization state, into and / or out of the waveguide. In some implementations, the eye illuminator 934 may further include a polarization modification element (e.g., a polarizer) configured to emit only or primarily a first polarization (e.g., p-polarization) or to transmit only light in a first polarization state (e.g., p-polarization). Additionally, the coupling optical element 944 and / or the external coupling optical element 952 may each be configured to redirect light in a second polarization (e.g., s-polarization) into and / or out of the waveguide.

[0189] Similar to the imaging system 900 shown in FIG. 17, the curved reflector 998 shown in FIG. 17 comprises a curved transmissive optical element 996 having a wavelength-dependent reflective coating 998. The wavelength-dependent reflective coating 998 may be configured to reflect light of a certain wavelength or range of wavelengths. In some implementations, for example, the wavelength-dependent reflective coating 998 may be configured to reflect non-visible light (e.g., infrared light) within a certain range of wavelengths, while the wavelength-dependent reflective coating 998 may be configured to transmit visible light. The wavelength-dependent reflective coating 998 may, in some cases, be disposed on the surface of the curved transmissive optical element 996.

[0190] In various implementations, the curved transmissive optical element 996 is disposed on the distal side of the eyepiece 950 in front of the eyepiece. Thus, the reflective optical element 996 is disposed between the eyepiece 950 and the user's forward environment. Similarly, the eyepiece 950 is disposed between the reflective optical element 996 and the eye 210.

[0191] Therefore, light having a first polarization (e.g., p-polarization) from the eye 210 is incident on the coupling optical element 944 and passes through it to the curved transmissive optical element 996. The imaging system 900 further includes a polarization correction optical element 978 such as a retarder (e.g., a quarter-wave retarder). This retarder 978 is transmissive and imparts a quarter-wave phase difference on the light transmitted through it. This light is incident on the curved transmissive optical element 996 and is reflected therefrom. The wavelength-dependent reflective coating 998 may be configured to reflect light of a certain wavelength reflected from the eye. As a result, this light is reflected from the curved surface of the curved transmissive optical element 996 and collimated. This collimated light passes through the retarder 978 again, thereby imparting another quarter-wave phase difference on the light transmitted through it. The phase difference (e.g., a full-wave phase difference) introduced in these two passes through the retarder rotates the polarization. Therefore, the first polarization (e.g., p-polarization) transmitted through the polarization-selective coupling optical element 944 in the first pass is converted to a second polarization (s-polarization) and redirected into the waveguide 940 so as to be guided to the camera 920 by TIR. As discussed above, in various designs, the coupling optical element 944 is configured to redirect the collimated light reflected from the eye 210 into the waveguide 940 as induced therein. Therefore, the reflective optical element 996 may be configured to collimate the light reflected from the eye 210, such as the anterior portion of the eye (e.g., the cornea). The curved reflective optical element 996 may thus have a positive refractive power. In particular, in various designs, the reflective optical element 994 may have a focal length equal to or substantially equal to the distance from the reflective optical element 996 to the portion of the eye 210 to be imaged, such as the cornea, iris, etc. Exemplary values of the focal length may be, for example, 2 cm to 8 cm. In some implementations, the focal length is 4 cm to 6 cm. In some designs, the focal length is about 5 cm.

[0192] In various designs, the reflective optical element 996 may comprise a curved surface configured to reflect light. The curved surface may, in some cases, be spherical or rotationally symmetric. The reflective optical element 996 may comprise, for example, a concave mirror, at least for certain wavelengths and / or polarizations.

[0193] In various designs, the reflective optical element 996 includes a material that is transmissive to visible light such that the user can see the wearer's frontal environment. The wavelength-dependent reflective coating 998 disposed on the surface of the curved transmissive optical element 996 may thus be transmissive to visible light or at least certain wavelengths of visible light. The curved transmissive optical element 996 may also be coated with an anti-reflective coating on its outer surface (e.g., the distal surface). The anti-reflective coating may be configured to reduce the reflection of red, green, and / or blue light. However, the reflective optical element 994 may be configured to reflect a portion of the light scattered from the eye 210 and form an image of the eye. Thus, the reflective optical element 996 may act differently on different light. For example, the reflective optical element 996 may act differently on light of different polarization states (and / or wavelengths). The reflective optical element 996 may be configured to transmit visible light and reflect infrared light.

[0194] As shown in FIG. 17, one or more light sources 934, such as LEDs or emitters in FIG. 18, are arranged with respect to the eye 210, guided through the waveguide 940 by TIR, and may direct light onto it without being directed onto the eye 210. Thus, in some implementations, the eyepiece lens 950 or the waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210. In some designs, a plurality of such light sources 934 may be arranged in a pattern (e.g., a circular or ring pattern) in the vicinity and / or around the eye. The one or more light sources 934 may be similar to the one or more light sources 960 described above and may, for example, be pulsed. Similarly, the one or more light sources 934 may comprise an infrared light source, such as an infrared LED, or another type of invisible light. In particular, in various implementations, the light source 934 may emit light reflected by the wavelength-dependent reflective coating 998 and / or the curved transmissive optical element 996. However, other types of light sources may also be used.

[0195] The polarization-selective coupling optical element 944 is configured to be polarization-selective depending on the type of linearly polarized light incident thereon, although other polarization-selective coupling optical elements may be polarization-selective for other types of polarization states such as different types of circular or elliptical polarization. The polarization-selective coupling optical element 944 may be configured such that, for example, a first polarization such as a first circular or elliptical polarization (e.g., left-handed polarization or LHP-polarization) is transmitted through the polarization-selective coupling optical element 944, and a second polarization such as a second circular or elliptical polarization (e.g., right-handed polarization or RHP) is redirected into the optical waveguide, or vice versa. Such a polarization-selective coupling optical element 944 may comprise a liquid crystal such as a cholesteric liquid crystal. Some examples of liquid crystal optical elements are discussed in the section entitled "Cholesteric Liquid Crystal Mirrors" below, U.S. Patent Publication No. 2018 / 0164627, filed Dec. 7, 2017, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL", U.S. Patent Publication No. 2018 / 0239147, filed Feb. 22, 2018, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR", and U.S. Patent Publication No. 2018 / 0239177, filed Feb. 22, 2018, entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION", each of which is hereby incorporated by reference in its entirety and for all purposes.

[0196] A polarization correction element or retarder such as a circular polarizer may be disposed between the eye and the polarization-selective coupling optical element 944 to convert the light reflected from the eye into a first polarization (e.g., LHP). The LHP light will pass through the polarization-selective coupling optical element 944, be reflected from the reflector 998, have its polarization changed to RHP, and be redirected by the polarization-selective coupling optical element 944 into the waveguide towards the camera.

[0197] In some implementations, the reflector 996 may be polarization selective in its reflectivity such that only light in a certain polarization state is reflected and / or light in a different polarization state is transmitted. Such an optical element may comprise a liquid crystal such as a cholesteric liquid crystal. Examples of such optical elements are discussed in the section entitled "Cholesteric Liquid Crystal Mirrors" below, U.S. Patent Publication No. 2018 / 0164627, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL", filed on Dec. 7, 2017, U.S. Patent Publication No. 2018 / 0239147, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR", filed on Feb. 22, 2018, and U.S. Patent Publication No. 2018 / 0239177, entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION", filed on Feb. 22, 2018 (each of which is incorporated herein by reference in its entirety and for all purposes). Such an optical element may reflect light in a first polarization state, such as a first circular or elliptical polarization state (left-handed circular or elliptical polarization), and transmit light in a second polarization state, such as a second circular or elliptical polarization state (e.g., right-handed circular or elliptical polarization), or vice versa. In some embodiments, the liquid crystal is disposed on the curved surface of the reflector 996 such that, upon reflection, the reflector has a refractive power, such as a positive refractive power. In various other implementations, the liquid crystal optical element may be flat or planar. For example, the liquid crystal may be disposed on a flat or planar substrate or layer. Despite being flat, a refractive power may be included within the liquid crystal optical element. Such an element may be referred to as a cholesteric liquid crystal reflective lens. Thus, light from the eye may be collimated and reflected to the coupling optical element 998. The reflector may, for example, reflect light in a first polarization state (e.g., left-handed circular or elliptical polarization) and transmit light in a second polarization (e.g., right-handed circular or elliptical polarization).Therefore, the eye 210 is illuminated with left-handed circularly polarized light, or light reflected from the eye is transmitted through a polarizer (e.g., a circular or elliptical polarizer) that transmits light having a first polarization (e.g., left-handed circular or elliptical polarization). The coupling optical element 944 may also be polarization selective and may transmit LHP light and redirect RHP light into the waveguide. The LHP light from the eye passes through the coupling optical element 944. The transmitted LHP light also impinges on the wavelength selective liquid crystal reflector 996 and is reflected therefrom. In one design, the wavelength selective liquid crystal reflector 996 converts a first polarization state (e.g., LHP) to a second polarization state (e.g., RHP) in response to reflection. The light in this second polarization state (e.g., RHP light) is directed to the coupling optical element 944, which redirects the light in the second polarization state (RHP) into the waveguide 940 to the camera 920.

[0198] In some designs, the coupling optical element 944 does not include a liquid crystal grating, but instead includes, for example, a surface relief diffraction grating or a holographic grating. As discussed above, these coupling optical elements 944 that do not include cholesteric liquid crystal may also include volume diffraction or holographic optical elements or gratings.

[0199] Therefore, light scattered from the eye is reflected by the reflective optical element 996 back into the waveguide 940 so as to be coupled into the waveguide by the coupling element 944. However, in contrast, a portion of the unpolarized light from the wearer's frontal environment corresponding to the second polarization state (e.g., RHP) will be transmitted through the reflective optical element 996. Thus, an object may be visible to the wearer through the reflective optical element 996.

[0200] However, in various designs, the reflective optical element 996 will have a negligible refractive power in transmission. For example, the reflective optical element 996 may have curved surfaces with the same curvature on both sides of the optical element such that the total refractive power of the optical element for light transmitted therethrough will be negligible.

[0201] As discussed above, in various implementations, the reflective optical element 996 comprises a cholesteric liquid crystal reflective lens that is a cholesteric liquid crystal reflective element as discussed in the section entitled "Cholesteric Liquid Crystal Mirrors" below, in U.S. Patent Publication No. 2018 / 0164627, filed Dec. 7, 2017, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL", in U.S. Patent Publication No. 2018 / 0239147, filed Feb. 22, 2018, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR", and in U.S. Patent Publication No. 2018 / 0239177, filed Feb. 22, 2018, entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION" (each incorporated herein by reference in its entirety and for all purposes). Such an optical element can act on a particular wavelength or wavelength range. Thus, light such as infrared light reflected from the eye can be acted on by the cholesteric liquid crystal reflective element. However, light not within that wavelength range, such as visible light from the environment, can pass through the cholesteric liquid crystal reflective element without being acted on by the cholesteric liquid crystal reflective element. Thus, the cholesteric liquid crystal reflective element can have a negligible refractive power for the ambient visible light passing through it.

[0202] As discussed above, in one implementation, the illumination source 960 couples light into the waveguide 940 that is redirected out of the waveguide and illuminates the eye 210. In such an embodiment, the coupling optical element 944 may be polarization selective. For example, the coupling optical element 944 may transmit a first polarization (p-polarization) and may transmit a second polarization (s-polarization).

[0203] Thus, when light from the illumination source 906 propagates through the waveguide 940 and is redirected by the coupling optical element 944, this illumination will be s-polarized. A polarization correction optical element (e.g., a quarter-wave retarder) may be disposed between the waveguide 940 and the eye 210 to cause rotation of the polarization reflected from the eye. Light from the light source 960 reflected from the eye 210 passes twice through the quarter-wave retarder, and as a result, the s-polarized light emitted from the waveguide by the coupling element 944 to illuminate the eye will be converted to p-polarized light.

[0204] This p-polarized light will pass through the coupling optical element 944 and the waveguide and will be incident on the reflective optical element 996.

[0205] The imaging system 900 may further include a second polarization correction element 978, which may include, for example, a retarder or a waveplate as discussed above. This retarder may include, for example, a quarter-wave retarder. The second polarization correction element 978 may be disposed distal to the waveguide 940, i.e., between the waveguide and the reflector 996. The second polarization correction element 978 may also be disposed between the coupling element light 944 and the reflector 996. The light (p-polarized) from the eye 210 transmitted through the coupling element 944 passes through the second polarization correction element 978 and is converted to circularly polarized light. When the reflector 996 reflects the circularly polarized light, this light will be reflected back through the polarization correction element 978 and then back into the waveguide 940. Two passes through this polarization correction element (e.g., a quarter-wave retarder) 978 convert the light to s-polarized light and will be redirected into the waveguide by the coupling element 944 so as to be directed into a camera (not shown).

[0206] As shown in FIG. 18, the light 988 reflected from the eye 210 diverges. This light is incident on a reflector 996 that is curved or otherwise has a positive refractive power, whereby it can be collimated. A coupling optical element 944 configured to redirect the collimated light into the waveguide 940 will thus direct the collimated light from the curved reflective optical element 996 towards an imaging device 920 (not shown). Thus, the light reflected from the eye 210 and collimated by the curved reflective optical element 996 is coupled into the waveguide 940 and guided therein towards an external coupling optical element 952. The external coupling optical element 952 may be configured to direct the light from the eyepiece lens 950 outwards towards a camera (not shown).

[0207] Various modifications are possible in the configuration of the imaging system. Different types of reflectors 996 and coupling elements 944 may be employed. The reflector 996 and the coupling element 944 may be configured, for example, to act on linear polarization or circular or elliptical polarization. As discussed, the reflector 996 has a refractive power. The reflector 996 and the coupling element 944 may comprise a cholesteric liquid crystal lattice reflector and / or a lens, with or without refractive power. A polarization correction element 978 such as a retarder may be included between the coupling element 944 and the reflector and / or between the coupling element 944 and the eye. In some embodiments, a polarizer such as a circular polarizer or a linear polarizer may be disposed between the eye and the coupling element 944. For example, if unpolarized light is reflected from the eye, a polarizer (e.g., a circular polarizer or a linear polarizer) may be disposed between the eye and the coupling element 944. In some such cases, the coupling element 944 is polarization selective.

[0208] In configurations such as those shown in FIGS. 17 and 18, where light reflected from the eye passes through waveguide 940 to curved reflective optical element 996, is collimated, and is redirected back into the waveguide, background noise is introduced. This background noise originates from the light that first passes through coupling optical element 944 from the eye. As discussed above, coupling optical element 944 may be configured to redirect collimated light into waveguide 940 such that an image is induced therein onto camera 920. However, coupling optical element 944 will redirect some of the uncollimated light incident thereon. Thus, in the initial passage through coupling optical element 944 and waveguide 940 to curved reflective optical element 996, a portion of the uncollimated (diverging) light reflected from the eye is coupled into the waveguide by coupling optical element 944 and will contribute to the background noise for the image of the eye formed by camera 920. This noise will be superimposed on the image formed by the collimated light retroreflected by curved reflective optical element 996 that is coupled into the waveguide by coupling optical element 944 such that an image is induced therein onto camera 920.

[0209] In one design, this noise can be removed from the image. The process for removing the noise may involve: (a) measuring the amount of light (referred to as N) coupled by the coupling optical element 944 in an initial pass through the curved reflective optical element 996 through the coupling optical element 944 that is redirected and reaches the camera 920; and (b) measuring the total signal at the camera 920 when the light passes through the coupling optical element 944 and the waveguide 940 to the curved reflective optical element 996, is collimated, reflected back to the coupling optical element, and redirected to the camera. This total signal may also include a certain noise N because the uncollimated light reflected from the eye passes through the coupling optical element 944 and reaches the curved reflective optical element 996, and thus a portion of the uncollimated light will be redirected by the coupling optical element 944 to the camera 920. If the noise N can be measured separately from the total signal T, which includes the noise superimposed across the image of the eye, the noise N can be removed from the total signal T as represented by the following equation. I = T - N Wherein, I represents the image with the noise component N removed.

[0210] The above two measurements (a) and (b) can be obtained in various ways. For example, as shown in FIG. 19, the shutter 936 can be disposed between the curved reflective optical element 996, the waveguide 940, and the coupling optical element 944. The shutter 936 can be configured to block light when the shutter is in the first state and transmit light when the shutter is in the second state. The shutter 936 may include, for example, a liquid crystal shutter.

[0211] Therefore, the noise component N can be measured when the shutter 936 is in the first state where the light reflected from the eye 210 is incident on the coupling optical element 944 and passes through it towards the curved reflective optical element 996. However, the closed shutter prevents it from reaching the curved reflective optical element. As discussed above, a portion of the light reflected from the eye 210, although mostly not collimated, couples into the coupling optical element 944, is redirected in the waveguide, and is guided therein to the camera 920. As referred to above, this light does not contribute to image formation but will be background noise. The camera 920 may record this noise N when the shutter 936 is closed.

[0212] The total signal T, which includes both the noise N and the image, can be measured when the shutter 936 is in the second state where the shutter is open. The light reflected from the eye 210 is again incident on the coupling optical element 944. A portion of this light reflected from the eye 210, although mostly not collimated, couples into the coupling optical element 944, is redirected in the waveguide, and is guided therein to the camera 920. However, most of the light reflected from the eye 210 passes through the coupling optical element 944, through the open shutter 936, and to the curved reflective optical element 996. The curved reflective optical element 996 collimates the light and reflects at least a portion of it back to the coupling optical element 944, which redirects this collimated light into the waveguide 920 so as to be guided to the camera 920 to form an image of the eye 210. The camera 920 can capture the actual image of the eye 210.

[0213] A processing electronic device (such as processing electronic device 140) that communicates with camera 920 can receive a noise component N measured when shutter 936 is in a first closed state and a total signal T measured when the shutter is in a second open state, and can subtract the two (T - N). In this way, in the initial pass through it, the noise N contributed by the uncollimated light reflected from eye 210 and coupled into coupling optical element 944 can be subtracted from the total image signal T. The processing electronic device may communicate with camera 920 via a wired electronic signal. Additionally, or alternatively, the processing electronic device may communicate with camera 920 using one or more remote receivers. The processing electronic device may reside remotely (e.g., a cloud computing device, a remote server, etc.).

[0214] Other methods may be employed to perform the measurements of (a) and (b), obtain N and T, and subtract N from T. For example, if curved reflective optical element 996 is wavelength selective as shown in FIG. 18, the eye can be illuminated with light of different wavelengths at different times. For example, to perform measurement (a) and quantify noise N, the eye can be illuminated with a wavelength not reflected by curved reflective optical element 996. However, to perform measurement (b) and quantify total signal T, the eye can be illuminated with a wavelength reflected by curved reflective optical element 996. The noise N can then be subtracted from total T as discussed above (e.g., T - N).

[0215] FIG. 20-20E illustrates an exemplary imaging system 900 configured to perform measurements using wavelength modulation and remove noise component N as discussed above. The imaging system 900 in FIGS. 20A-20E includes a curved transmissive optical element 996 that is wavelength selective (such as those described with reference to FIGS. 17 and 18 above). For example, the curved transmissive optical element 996 has a wavelength-dependent reflective coating 998 on its curved surface. The imaging system 900 may also include one or more light sources or illumination sources (not shown) configured to illuminate the eye 210. The one or more light sources may be configured to emit infrared light. However, the one or more light sources can be configured to emit different colors or wavelengths of light at different times. Such wavelength modulation can enable separate measurement of N to be removed from the total signal T.

[0216] In various implementations, for example, one or more illumination sources 960, 934 are configured to emit one or more wavelengths λ that are reflected by the curved reflective optical element in a first state Reflect and one or more wavelengths λ that are not reflected in a second state Not Reflect In the second state, a negligible amount or less of the wavelength λ that is reflected by the curved reflective optical element is emitted. Similarly, in the first state, a negligible amount or less of the wavelength λ that is not reflected is emitted. Reflect is emitted. Not Reflect is emitted.

[0217] In some embodiments, the wavelength λ that is reflected may be about 800 nm to 950 nm. The wavelength λ that is reflected may be about 835 nm to 915 nm. The wavelength λ that is reflected may be about 840 nm to 870 nm. In some designs, the wavelength λ that is reflected is about 850 nm. The light emission 928 from the one or more light sources 960 can illuminate the eye. Reflect Reflect Reflect Reflect Reflect Reflect Reflect Reflect

[0218] As shown in FIG. 20B, the wavelength λ that is not reflected by the curved reflective optical element 944Not Reflect Light 988 having (and an amount of light λ below an ignorable amount reflected by the curved reflective optical element 944) Reflect ) is reflected from a part of the eye 210 (e.g., the cornea). This light is a wavelength λ not reflected by the curved reflective optical element 944 Not Reflect Since it comprises, the light ray 916 is shown to propagate through the curved reflective optical element 996 into the environment in front of the user.

[0219] The light 988 incident on the coupling optical element 944 is not collimated, but the coupling optical element nonetheless couples at least a portion of the light 914 into the waveguide 940 so as to be directed to the camera 920. Thus, the camera 920 can capture an image (Image #1) corresponding to a noise component N resulting from the uncollimated light redirected by the coupling optical element 944 in the initial passage to the curved reflective optical element 996. This image (Image #1) is background noise and not a recognizable image of the eye. The processing electronics 140 is shown to receive this first image (Image #1).

[0220] In FIGS. 20C - 20E, an illumination source (not shown) emits one or more wavelengths λ Reflect and wavelengths λ below an ignorable amount that are not reflected by the curved reflective optical element Not Reflect This wavelength λ Reflect can be, for example, 850 nm.

[0221] As shown in FIG. 20C, in the first passage through the coupling optical element 944, a portion of the light 988 reflected from the eye 210 incident on the coupling optical element 944 is coupled by the coupling optical element 944 into the waveguide 940 (as in FIG. 20B) and directed towards the camera 920. In addition, the wavelength λ ReflectThe curved transmissive optical element 996, which selectively reflects light, reflects and collimates the internally uncoupled light 918 reflected from the eye 210 that is incident on the curved transmissive optical element. As shown in FIG. 20E, the coupling optical element 944 redirects the collimated reflected light and couples it into the waveguide 940 towards the camera 920. FIG. 20E shows both components reaching the camera 920, namely, the light 988 that is incident on the coupling optical element 944 and reflected from the eye 210 in the first pass through the coupling optical element 944, which is coupled into the waveguide 940 by the coupling optical element, and the light reflected and collimated by the curved transmissive optical element 996, which is coupled into the waveguide by the coupling optical element. The camera 920 may capture an image (Image #2) corresponding to the total image component T. The processing electronics 140 is shown to receive the second image (Image #2).

[0222] As discussed above, the processing electronics may subtract the noise from the image T - N. In this example, Image #1 can be subtracted from Image #2. Thus, the processing electronics 140 may be configured to modify the second image based on the first image. However, other approaches are also possible. For example, the processing electronics 140 may be configured to create a new image representing a version of the second image with reduced optical noise. Implementations for subtracting noise from the image may be used in the implementations described above. For example, the implementations shown in FIGS. 10, 11A - 11E, and / or FIGS. 12A - 12E may include a wavelength - dependent reflective coating 998, a shutter 936, and / or a curved transmissive optical element 996 configured to selectively reflect the internally uncoupled light 912 and direct the light towards the imaging device 920.

[0223] As discussed above, Image #1 is at one or more wavelengths λ at which light is not reflected by the curved reflective optical element Not Reflect and at wavelengths λ at which the amount reflected is below an ignorable amount Reflectacquired for the case of being illuminated. Image #2 is for one or more wavelengths λ at which light is reflected by the curved reflective optical element Reflect and wavelengths λ below a negligible amount that are not reflected Not Reflect acquired for the case of being illuminated. Thus, one or more illumination sources 960, 934 may be configured to modulate the wavelength. For example, in one design, one or more illumination sources 960, 934 are for one or more wavelengths λ that are not reflected by the curved reflective optical element Not Reflect and wavelengths λ below a negligible amount that are reflected Reflect and may include a first illumination source configured to output. The one or more illumination sources may further include a second illumination source configured to output one or more wavelengths λ that are reflected by the curved reflective optical element Reflect and wavelengths λ below a negligible amount that are not reflected Not Reflect The intensities of the first and second illumination sources may alternatively be increased and decreased, turned on and off, attenuated and not attenuated, passed and blocked, to provide modulation of the wavelength of the light and illuminate the eye. For example, during a first time interval, the first illumination source may be blocked while the second illumination source is not blocked. During a subsequent second time interval, the second illumination source may be blocked while the first illumination source is not blocked. This process can be repeated to provide modulation of the wavelength of the light and illuminate the eye. In other designs, the wavelength of the light source may be adjusted and de - adjusted to reciprocate and shift the wavelength between λ Reflect and λ Not Reflect Other arrangements are also possible.

[0224] As described above, the imaging system 900 may also be included within a head-mounted display such as an augmented reality display that provides the ability to image the eye by collecting light with the eyepiece 950. Such an imaging system 900 may be used for eye tracking. Multiple images of the retina or anterior portion of the eye may be acquired. Eye movement and / or repositioning can be confirmed from these images, and the eye position and / or orientation can be tracked. These imaging systems may also be used for biometric imaging and / or for identifying the user. For example, an image of the user's eye such as the retina or iris may be acquired and recorded. Subsequent images of the wearer's eye (e.g., retina or iris) may be acquired at a later time. The two images may be compared to determine whether the wearer at a subsequent instance is the same as the wearer at the first instance. However, other uses for the imaging system are also conceivable.

[0225] The illumination system may be waveguide-based and may be described above as comprising one or more waveguides, but other types of light redirecting optical elements may be employed instead of waveguides. Such a light redirecting optical element includes a redirecting feature and may emit light from the light redirecting optical element, for example, onto a spatial light modulator. Thus, in any of the embodiments described herein and in any of the embodiments below, any reference to a waveguide may be replaced with a light redirecting optical element instead of a waveguide. Such a light redirecting optical element may comprise, for example, a polarizing beam splitter such as a polarizing beam splitting prism.

[0226] As discussed above, the systems described herein can enable the collection and / or biometric identification of biometric data. For example, an eye or a part thereof (e.g., the retina) can be imaged to provide such biometric data and / or biometric identification. Images of the eye, such as of the retina, may be acquired at various times when a head-mounted display system is worn by a user, presumably the same user. The collection of such images can be recorded, for example, in a database. These images may be analyzed to collect biometric data. Such biometric data can be useful for monitoring a user's health or medical status. Different medical parameters can be monitored by imaging a patient's eye, e.g., the retina. The medical parameters can be recorded and compared with subsequent measurements taken when the user is wearing the head-mounted display system.

[0227] In addition, when a person starts wearing a head-mounted display system and an image of the user's eye that does not match the image stored in the database is captured, a conclusion can be drawn that the person currently wearing the head-mounted display system is different from the previous user. This can be useful in determining whether the intended user is wearing the headset or whether it is being worn by a new user. Such a feature can enable certain medical, security, and / or usability applications or functionality. For example, a head-mounted display may be configured to identify the wearer based on the characteristics of the wearer's eyes. For example, the system can be configured to determine an individual based on the characteristics of the wearer's retina (e.g., blood vessels), corneal features, or other eye features. In some implementations, for example, a set of markers may be determined for a particular wearer. Based on the set of markers, the system may be able to determine that the previous user was wearing the headset or, alternatively, that a different user was wearing the headset. The markers may include the shape or center of the user's cornea, the configuration of blood vessels within the user's retina, the intensity and / or position of the reflection of light from the cornea, the shape of the sides of the eye, and / or any other biometric marker. In one implementation, a confusion matrix can be determined. As discussed above, for example, in the discussion of developing a retinal map using virtual / fixed targets in various locations (see, e.g., FIG. 13B), the system may cause the user to look at a set of predetermined directions or eye postures and develop a matrix of characteristics of the eye or part of the eye (e.g., cornea, retina, etc.) associated with each direction or eye posture. Using such a matrix, the system can determine the identification of an individual. Other methods are also conceivable.

[0228] Similarly, as discussed above, various configurations of the system are also conceivable. For example, FIG. 21 shows an exemplary eyepiece 900 that can be used to project light into a user's eye while simultaneously imaging the user's eye. The eyepiece 900 shown includes an internal coupling optical element 2104, a light dispersing element 2108, a light collecting element 2116, and an external coupling optical element 2120 on the opposite side of the coupling optical element 2112. These optical elements may each be disposed within or on the waveguide 2102. The waveguide 2102 may correspond to, for example, one of the waveguides 670, 680, 690 described herein (see, e.g., FIGS. 9A-9C). The internal coupling optical element 2104 may correspond to one of the internal coupling optical elements 700, 710, 720 described herein and / or the internal coupling optical element 942 (see, e.g., FIG. 10) and may be configured to input image content from a projector into the waveguide and / or input illumination from a light source 960. The light dispersing element 2108 may correspond to one of the light dispersing elements 730, 740, 750 described herein (see, e.g., FIGS. 9A-9C) and may be used to diffuse light in a given direction and redirect light from the internal coupling optical element 2104 to the coupling optical element 2112. The coupling optical element 2112 may correspond to the coupling optical element 944 described herein (see, e.g., FIG. 10). In some designs, the coupling optical element 2112 includes the functionality described herein for the external coupling optical elements 800, 810, 820 (see FIGS. 9A-9C). The light collecting element 2116 may be configured to reduce the lateral spatial extent of the light received from the coupling optical element 2112 and redirect the light toward the external coupling optical element 2120. The external coupling optical element 2120 may correspond to the external coupling optical element 952 described herein (see, e.g., FIG. 10).

[0229] The internal coupling optical element 2104 may be disposed within or on the waveguide 2102 so as to receive light from, for example, a projector (e.g., image projector 930) and / or a illuminator (e.g., light source 960). The light may be passed through the waveguide 2102 to an associated light dispersing optical element 2108. Any one of the internal coupling optical element 2104, the light dispersing optical element 2108, or the coupling optical element 2112 may be disposed on a major surface of the waveguide (e.g., on the top or bottom surface) or within the waveguide. Similarly, any one or combination of the condensing element 2116 and / or the external coupling optical element 2120 may be disposed on a major surface of the waveguide 2102 (e.g., on the top or both major surfaces) or within the waveguide.

[0230] The coupling optical element 2112 may receive light from the light dispersing element 2108 (e.g., via TIR), expand the light, and direct it onto the user's eye. Thus, the coupling optical element 2112 may be disposed in front of the user's eye and project image content therein. Additionally or alternatively, the coupling optical element 2112 may be configured to provide illumination light onto and / or into the user's eye.

[0231] Light reflected from the eye (e.g., illumination light from an illumination source) may be reflected and captured by the coupling optical element 2112. Thus, in some embodiments, the coupling optical element 2112 may serve both to externally couple light received from the light dispersing element 2108 and to internally couple light received from the eye into the waveguide 2102.

[0232] In some embodiments, the coupling optical element 2112 may include one or more diffractive optical elements (DOEs) such that the coupling optical element 2112 has dual functionality. The first DOE (e.g., a grating, a holographic region) may similarly be configured to externally couple light, and the second DOE may be configured to internally couple light reflected from the eye into the waveguide 2102. In some embodiments, both the first and second DOEs are superimposed within the waveguide 2102 (e.g., occupy the same or substantially the same volume).

[0233] Alternatively, in some embodiments, the combined optical element 2112 includes at least two DOEs stacked over or in front of each other. For example, referring to FIG. 21, a first DOE of the combined optical element 2112 is disposed over a second diffractive element, while the second diffractive element may be disposed under the first DOE. The order of each DOE may be reversed in other implementations. Cholesteric liquid crystal mirror

[0234] Some liquid crystals are in a phase referred to as the chiral or cholesteric phase. In the cholesteric phase, the liquid crystal may exhibit a twist of molecules along an axis perpendicular to the director, and the molecular axis is parallel to the director. As described herein, a cholesteric liquid crystal (CLC) layer extends in the cholesteric phase in a direction, for example, a direction perpendicular to the director such as the layer depth direction, and includes a plurality of liquid crystal molecules that are continuously rotated or twisted in a certain rotational direction, for example, clockwise or counterclockwise. The director of the liquid crystal molecules in the chiral structure can be characterized as a spiral having a helical pitch (p), which corresponds to the length in the layer depth direction corresponding to the net rotation angle of the liquid crystal molecules of the chiral structure by one rotation in the first rotational direction. In other words, the helical pitch refers to the distance over which the liquid crystal molecules undergo a complete 360° twist. The liquid crystal exhibiting chirality may also have a twist angle or rotation angle (φ), for example, referring to the relative azimuthal rotation between adjacent liquid crystal molecules in the layer normal direction, and may be described as having a net twist angle or net rotation angle, for example, referring to the relative azimuthal rotation between the uppermost and lowermost liquid crystal molecules across a defined length, for example, the length of the chiral structure or the thickness of the liquid crystal layer. As described herein, the chiral structure refers to a plurality of liquid crystal molecules in the cholesteric phase that extend in a direction, for example, a direction perpendicular to the director such as the layer depth direction, and are continuously rotated or twisted in a certain rotational direction, for example, clockwise or counterclockwise. On one side, the director of the liquid crystal molecules in the chiral structure can be characterized as a spiral having a helical pitch.

[0235] FIG. 22 illustrates a cross-sectional side view of a cholesteric liquid crystal (CLC) layer 1004 having a plurality of uniform chiral structures according to an embodiment. In the CLC layer 1004, chiral structures adjacent in the lateral direction, e.g., the x-direction, have liquid crystal molecules arranged in the same manner. In the illustrated embodiment, the chiral structures 1012-1, 1012-2, … 1012-i are similarly configured such that liquid crystal molecules of different chiral structures at approximately the same depth, e.g., liquid crystal molecules closest to the light incident surface 1004S, have the same rotation angle, and a continuous rotation angle of continuous liquid crystal molecules at approximately the same depth, and a net rotation angle of the liquid crystal molecules of each chiral structure.

[0236] CLC1004 includes a CLC layer 1008 comprising liquid crystal molecules arranged as a plurality of chiral structures 1012-1, 1012-2, … 1012-i, each chiral structure comprising a plurality of liquid crystal molecules, and i is any suitable integer greater than 2. In operation, when incident light having a combination of a light beam with left-handed circular polarization and a light beam with right-handed circular polarization is incident on the surface 1004S of the CLC layer 1008 by Bragg reflection, light with one of the circular polarization chiralities is reflected by the CLC layer 1004, while light with the opposite polarization chirality is transmitted through the CLC layer 1008 without substantial interference. As described throughout this specification and the present disclosure, chirality is defined as being visible in the propagation direction. According to an embodiment, when the polarization direction or polarization chirality of the light beams 1016-L, 1016-R is matched to have the same rotation direction as the liquid crystal molecules of the chiral structures 1012-1, 1012-2, … 1012-i, the incident light is reflected. As shown, the incidence on the surface 1004S results in a light beam 1016-L with left-handed circular polarization and a light beam 1016-R with right-handed circular polarization. In the illustrated embodiment, the liquid crystal molecules of the chiral structures 1012-1, 1012-2, … 1012-i are continuously rotated in the clockwise direction, in which direction the incident light beams 1016-L, 1016-R are traveling, i.e., the positive x-direction, which is the same rotation direction as the light beam 1016-R with right-handed circular polarization. As a result, the light beam 1016-R with right-handed circular polarization is substantially reflected, while the light beam 1016-L with left-handed circular polarization is substantially transmitted through the CLC layer 1004.

[0237] As described above, by matching the chirality of the polarization of incident elliptical or circularly polarized light with the direction of rotation of the liquid crystal molecules in the chiral structure of the CLC layer, the CLC layer can be configured as a Bragg reflector. Further, one or more CLC layers having different helical pitches can be configured as a wavelength-selective Bragg reflector with a high bandwidth. Based on the concepts described herein with respect to various embodiments, the CLC layer can be configured as an off-axis or on-axis mirror that selectively reflects a first range of wavelengths, for example, infrared wavelengths (e.g., near infrared), while transmitting another range of wavelengths, for example, visible wavelengths.

[0238] FIG. 23 illustrates an example of an eye tracking system 2300 that employs a cholesteric liquid crystal reflector (CLCR), e.g., a wavelength-selective CLCR 1150, configured to image the viewer's eye 302 according to various embodiments. Unlike the CLC layer 1004 described above with respect to FIG. 22, the chiral structures within the wavelength-selective CLCR 1150 adjacent in a lateral direction, e.g., the x-direction, have liquid crystal molecules arranged differently. That is, the chiral structure is configured such that liquid crystal molecules of different chiral structures at approximately the same depth, e.g., the liquid crystal molecules closest to the light incident surface 1004S, have different angles of rotation. As a result, light incident on the CLCR 1150 is reflected at an angle (θ R ) with respect to the layer depth direction as further described below in the context of the eye tracking system 2300.

[0239] Eye tracking can be a useful feature in a bidirectional vision or control system, including, among other uses, the wearable display systems described anywhere in this specification, particularly for virtual / augmented / mixed reality display applications. To achieve effective eye tracking, it may be desirable to acquire an image of the eye 302 at a low eye line angle, and in this regard, it may then be desirable to position the eye tracking camera 702b near the center position of the viewer's eye. However, such a position of the camera 702b can interfere with the user's view. Alternatively, the eye tracking camera 702b may be positioned at a lower position or side. However, such a position of the camera can increase the difficulty of obtaining robust and accurate eye tracking because the eye image is captured at a steeper angle. By configuring the CLCR 1150 to selectively reflect infrared (IR) light 2308 (e.g., having a wavelength of 850 nm) from the eye 302 while transmitting visible light 2304 from the world, the camera 702b can be installed away from the user's view while capturing the eye image at a vertical or low eye line angle. Such a configuration does not interfere with the user's view because visible light is not reflected. The same CLCR 1150 can also be configured as an IR illumination source 2320 by reflecting IR light from an IR source, e.g., an IR LED, into the eye 302 as shown. The low eye line angle of the IR illuminator can result in, for example, less occlusion from eyelashes, which is a configuration that allows for more robust detection of specular reflections and can be a useful feature in modern eye tracking systems.

[0240] Still referring to FIG. 23, according to various embodiments, the CLCR1150 includes one or more cholesteric liquid crystal (CLC) layers each having a plurality of chiral structures, and each chiral structure extends in the layer depth direction (e.g., the z-direction) and includes a plurality of liquid crystal molecules that are continuously rotated in a first rotation direction, as described above. The arrangement of the liquid crystal molecules in the chiral structure varies periodically in the lateral direction perpendicular to the layer depth direction such that one or more CLC layers are configured to substantially Bragg reflect a first incident light having a first wavelength (λ1) while substantially transmitting a second incident light having a second wavelength (λ2). As described above, each of the one or more CLC layers is configured to substantially Bragg reflect the first and second incident lights of elliptical or circular polarization having a chiral property of polarization that matches the first rotation direction when viewed in the layer depth direction, while being configured to substantially transmit the first and second incident lights of elliptical or circular polarization having a chiral property of polarization opposite to the first rotation direction when viewed in the layer depth direction. According to an embodiment, the arrangement of the liquid crystal molecules that varies periodically in the lateral direction is arranged to have a period in the lateral direction such that the ratio between the first wavelength and the period is from about 0.5 to about 2.0. According to an embodiment, the first wavelength is within the near-infrared range of about 600 nm to about 1.4 μm, e.g., about 850 nm, and the second wavelength is within the visible range having one or more colors as described elsewhere in this specification. According to various embodiments, the liquid crystal molecules of the chiral structure are pre-tilted with respect to a direction perpendicular to the layer depth direction. Configured such that one or more CLC layers are such that the first incident light is reflected at an angle (θ o about 50 o about 60 o about 70 o or about 80 R degrees or more with respect to the layer depth direction (z-direction).

[0241] Accordingly, the configured wavelength-selective CLCR1150 comprises one or more cholesteric liquid crystal (CLC) layers each comprising a plurality of liquid crystal molecules that extend in the layer depth direction and are continuously rotated in a first rotational direction, and the arrangement of the liquid crystal molecules with a chiral structure is such that the one or more CLC layers are configured to substantially Bragg reflect the first incident light having a first wavelength, e.g., an IR wavelength, while substantially transmitting the second incident light having a second wavelength, e.g., a visible wavelength, and vary periodically in a lateral direction perpendicular to the layer depth direction.

[0242] Similar liquid crystal layers and structures may be used for the reflectors 996 and coatings 998 described above in connection with FIGS. 17 - 20E. The coating 998 may comprise, for example, a liquid crystal coating and, in certain implementations, may be wavelength and / or polarization selective. However, other types of coatings 998 and reflectors 996 may be employed.

[0243] For example, as discussed above in connection with FIG. 16, the lens 980 may be used to modify (e.g., collimate) the propagation of light directed to the coupling optical element 944. This light may be, for example, light reflected from the user's eye, such as the anterior surface of the user's eye (e.g., the corneal surface). The distance from the eye, e.g., the anterior surface (e.g., the corneal surface) to the coupling optical element 944 may be, for example, about 20 mm. A positive lens 980, such as a lens having a focal length of about 20 mm, may be configured to collimate light reflected from the anterior portion of the eye (e.g., the cornea) of the eye 210. The light reflected from the anterior surface of the eye may be coupled into the waveguide 940 and guided therein to the camera. By using a focal length set to the distance to the anterior surface of the eye, the camera can image such a surface. Thus, in various implementations, the positive lens 980 may accordingly have a focal length equal to or substantially equal to the distance to the portion of the eye 210 to be imaged from the lens, e.g., the cornea.

[0244] Although refractive optical elements are shown, other types of lenses or optical elements with a refractive power such as a positive refractive power may be used. For example, the lens may include diffractive optical elements such as diffractive lenses or holograms. Such lenses may be disposed between the eye and the coupling optical element 944 in some implementations.

[0245] In various implementations, the coupling optical element 944 may include a refractive power. The coupling optical element 944 may comprise, for example, a diffractive optical element having a refractive power. The diffractive optical element may comprise, for example, a diffraction grating. The diffractive optical element may comprise a holographic optical element or a hologram. The diffractive optical element may have diffractive features, such as surface features, configured to perform both, for example, redirecting light into the waveguide and providing a refractive power. Other types of diffractive optical elements are also conceivable. In various implementations, the diffractive optical element may comprise a liquid crystal and may comprise a liquid crystal grating. The diffractive optical element may also comprise a polarization grating. Additionally, the diffractive optical element may comprise a liquid crystal polarization grating. Some non-limiting examples of liquid crystal gratings, liquid crystal polarization gratings, and other liquid crystal optical elements are hereby incorporated by reference in their entirety and for all purposes into this specification from the following published applications, namely, U.S. Patent Publication No. 2018 / 0143438, filed Nov. 16, 2017, entitled "MULTILAYER LIQUID CRYSTAL DIFFRACTIVE GRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES", U.S. Patent Publication No. 2018 / 0143485, filed Nov. 16, 2017, entitled "SPATIALLY VARIABLE LIQUID CRYSTAL DIFFRACTION GRATINGS", U.S. Patent Publication No. 2018 / 0143509, filed Nov. 16, 2017, entitled "WAVEGUIDE LIGHT MULTIPLEXER USING CROSSED GRATINGS", U.S. Patent Publication No. 2018 / 0239177, filed Feb. 22, 2018, entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION", and U.S. Patent Publication No. 2018 / 0164627, filed Dec. 7, 2017, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL", which are discussed therein.

[0246] The present diffractive optical element may have a refractive power that modifies the propagation of light incident thereon. The diffractive optical element may, for example, collimate the reflected light as a surface having a distance from the diffractive optical element corresponding to the focal length of the diffractive optical element. Such a distance may be, for example, about 15 to 20 mm (e.g., 20 mm or an approximation thereof). Such a focal length may provide collimation of light reflected from the anterior surface of the eye, such as the corneal surface (e.g., the cornea).

[0247] FIG. 24 shows an embodiment of a coupling optical element (e.g., a grating) 2111 disposed on waveguide 2102 of the eyepiece 950. The coupling optical element 2111 comprises a diffractive optical element configured to couple light incident thereon into waveguide 2102. The diffractive optical element also includes refractive power. For example, the diffractive optical element includes diffractive features such as surface diffractive features configured to provide refractive power, e.g., to collimate light incident thereon from the anterior surface of the eye (e.g., the corneal surface). A focusing element 2116 and an external coupling optical element 2120 are also shown disposed on waveguide 2102. The focusing element 2116 is arranged to receive the light coupled into waveguide 2102 by the coupling element 2111. The focusing element 2116 is configured to redirect the light incident thereon from the coupling optical element 2111 to the external coupling optical element 2120. The focusing element 2116 is configured to reduce the lateral spatial extent of the light (e.g., the light beam) from the at least one coupling element prior to reaching the at least one external coupling optical element 2120. In certain configurations, fewer optical elements may be used, possibly to reduce, for example, cost and / or optical losses, or for other reasons. For example, the focusing element 2116 may be omitted in some embodiments. In such embodiments, the light may be internally coupled from the input coupling element 2111 (e.g., after being reflected from the eye 210) and directly coupled to the external coupling optical element 2120. The light may propagate through the eyepiece 950 between the input coupling element 2111 and the external coupling optical element 2120. Other configurations are also possible. A camera is disposed relative to the external coupling optical element 2120 and receives light therefrom. The external coupling optical element 2120 is configured to direct the light received from the focusing element 2116 to the camera for image capture.

[0248] FIG. 24 also shows an internal coupling optical element 2104, a light dispersion element 2108, and an image content external coupling optical element 2110 disposed on the waveguide 2102. The internal coupling optical element 2104 may be configured to couple light received from an image projector into the waveguide 2102. The light distribution element 2108 may be configured to redirect light received from the internal coupling optical element 2104 to the external coupling optical element 2110 and, in addition, increase the spatial extent of the light, as discussed above. The external coupling optical element 2110 may be configured to couple light guided within the waveguide 2102 out of the waveguide and direct such light towards the eye for viewing the image content from the projector.

[0249] In various implementations, one or more of these optical elements 2111, 2116, 2120, 2110, 2108, 2104 may be disposed within or on the waveguide 2102. Similarly, as discussed above, one or more of these optical elements 2111, 2116, 2120, 2110, 2108, 2104 may comprise diffractive optical elements.

[0250] In the implementation illustrated in FIG. 24, a coupling optical element (e.g., a coupling grating) 2111 may be laterally displaced from the external coupling optical element 2110 on the waveguide 2102. In the implementation shown, a space laterally separates the coupling optical element 2111 from the external coupling optical element 2110.

[0251] FIG. 25 shows another similar implementation in which a coupling optical element (e.g., a coupling grating) 2111 is laterally displaced from the external coupling optical element 2110 on the waveguide 2102. However, in the implementation shown in FIG. 25, a space does not laterally separate the coupling optical element 2111 from the external coupling optical element 2110.

[0252] Displacing the coupling optical element (e.g., coupling grating) 2111 laterally from the external coupling optical element 2110 on the waveguide 2102 can enable the coupling optical element to include a refractive power, e.g., such that the refractive power does not affect the propagation of light from and / or through the external coupling optical element 2110 to the eye, collimating the light received from the anterior surface of the eye (e.g., corneal surface). The image presented to the eye from the image projector and the view of the environment in front of the user and the head-mounted display are thus not necessarily affected (e.g., distorted or defocused, etc.) by the refractive power of the coupling optical element 2111.

[0253] FIG. 26 illustrates an implementation of an imaging system 900 configured to image a plurality of portions of an eye. For example, an imaging system 900 such as that shown in FIG. 26 may be configured to image both the anterior surface of the eye (e.g., the corneal surface) and the retina. The imaging system 900 includes a pair of internal coupling optical elements, namely, a first internal coupling optical element 2111a and a second internal coupling optical element, disposed on a waveguide 2102 of an eyepiece lens. The imaging system 900 further includes a pair of condenser elements, namely, a first condenser optical element 2116a and a second condenser element 2116b, disposed on the waveguide 2102. In one configuration, fewer optical elements may be used, for example, to reduce cost and / or optical losses, or for other reasons. For example, the condenser elements 2116a, 2116b may be omitted in some embodiments. In such embodiments, light may be internally coupled from the first and second input coupling elements 2111a, 2111b (e.g., after being reflected from a part of the eye 210 such as the retina and / or the cornea) and directly coupled to the corresponding first and second external coupling optical elements 2120a, 2120b. The light may propagate through an eyepiece lens 950 (e.g., via a waveguide such as the waveguide 2102) between the first and second input coupling elements 2111a, 2111b and the corresponding external coupling optical elements 2120a, 2120b. Other configurations are also conceivable. Additionally, the imaging system 900 further includes a pair of optical external coupling optical elements, namely, a first external coupling optical element 2120a and a second external coupling optical element 2120b, disposed on the waveguide 2102.

[0254] The first coupling optical element 2111a is configured to couple the light incident thereon into the waveguide 2102. The first condensing element 2116a is arranged to receive the light coupled into the waveguide 2102 by the first coupling element 2111a. The first condensing element 2116a is configured to redirect the light incident thereon from the first coupling optical element 2111a to the first external coupling optical element 2120a. The first condensing element 2116a is also configured to reduce the lateral spatial extent of the light (e.g., light beam) from the first coupling element 2111a prior to reaching the first external coupling optical element 2120a. A camera is disposed relative to the first external coupling optical element 2120a and receives light therefrom. Although not shown in FIG. 26, an area corresponding to the detection area 2130 of the camera is shown. The first external coupling optical element 2120a is configured to direct the light received from the first condensing element 2116a to the camera, specifically, to the detection area 2130 shown in FIG. 26, for capturing an image.

[0255] Similarly, the second coupling optical element 2111b is configured to couple the light incident thereon into the waveguide 2102. The second condensing element 2116b is arranged to receive the light coupled into the waveguide 2102 by the second coupling element 2111b. The second condensing element 2116b is configured to redirect the light incident thereon from the second coupling optical element 2111b to the second external coupling optical element 2120b. The second condensing element 2116b is also configured to reduce the lateral spatial extent of the light (e.g., light beam) from the second coupling element 2111a prior to reaching the second external coupling optical element 2120a. A camera is disposed relative to the second external coupling optical element 2120b and receives light therefrom. Although not shown in FIG. 26, an area corresponding to the detection area 2130 of the camera is shown. The second external coupling optical element 2120b is configured to direct the light received from the second condensing element 2116b to the camera, specifically, to the detection area 2130 shown in FIG. 26, for capturing an image.

[0256] FIG. 26 also shows an internal coupling optical element 2104, a light dispersing element 2108, and an image content external coupling optical element 2110 disposed on a waveguide 2102. The internal coupling optical element 2104 may be configured to couple light received from an image projector into the waveguide 2102. The light distribution element 2108 may be configured to redirect light received from the internal coupling optical element 2104 to the external coupling optical element 2110 and, in addition, increase the spatial extent of the light, as discussed above. The external coupling optical element 2110 may be configured to couple light guided within the waveguide 2102 out of the waveguide and direct such light to the eye for viewing image content from the projector.

[0257] In the implementation illustrated in FIG. 26, first and second coupling optical elements (e.g., coupling gratings) 2111a, 2111b are laterally displaced on the waveguide 2102 from the external coupling optical element 2110. In the implementation shown, space separates the first and second coupling optical elements 2111a, 2111b laterally from the external coupling optical element 2110. Further, in the embodiment of FIG. 26, the first and second coupling optical elements 2111a, 2111b are laterally displaced from each other on the waveguide 2102. In other implementations, two or more of the first coupling optical element (e.g., coupling grating) 2111a, the second coupling optical element 2111b, and the external coupling optical element 2110 need not be laterally displaced from each other on the waveguide 2102.

[0258] In various implementations, one or more of these optical elements 2111a, 2111b, 2116a, 2116b, 2120a, 2120b, 2110, 2108, 2104 may be disposed within or on the waveguide 2102. Similarly, as discussed above, one or more of these optical elements 2111a, 2111b, 2116a, 2116b, 2120a, 2120b, 2110, 2108, 2104 may comprise diffractive optical elements.

[0259] The imaging system 900 of FIG. 26 can be configured to image a plurality of parts of the eye. For example, the imaging system 900 may be configured to image both the anterior surface of the eye (e.g., the corneal surface) and the retina. The first internal coupling optical element 2111a may have, for example, a lens disposed in front of it that has a refractive power or is accompanied by a refractive power. The first coupling optical element 2111a is configured, for example, to couple light into the waveguide 2102, but may also include a diffractive optical element configured to impart a refractive power thereto. Additionally, or alternatively, a lens having a refractive power may be disposed in front of the first coupling optical element 2111a. The refractive power may be configured to modify the propagation of the light received by the first internal coupling optical element 2111a such that a particular part of the eye can be imaged. In some embodiments, the refractive power may be a positive refractive power. Further, the refractive power may be such that the anterior surface of the eye can be imaged. The refractive power may correspond to a focal length of, for example, about 15 - 25 mm (e.g., about 20 mm). As a result, light reflected from the anterior surface of the eye, which may be at about 15 - 25 mm (e.g., about 20 mm) from the first coupling optical element 2111a, can be collimated and coupled into the waveguide.

[0260] In contrast, in various implementations, the second coupling optical element 2111b may not have refractive power and may not include a lens disposed in front of it. The lack of refractive power associated with the second coupling optical element 2111b will result in light from the anterior portion of the eye (e.g., the corneal surface) not being collimated and not being imaged by the camera. However, light reflected from the retina can be collimated as it passes through the natural lens of the user's eye. The collimated light by the natural lens of the eye may be coupled into the waveguide 2102 by the second coupling optical element 2111b and imaged by the camera. Thus, the light collected by the first coupling optical element 2111a may image the anterior surface of the eye (e.g., the corneal surface of the eye), and the second coupling optical element 2111b may image the retina of the user's eye. In some implementations, such a configuration enables the light collected by the first coupling optical element 2111a to form an image of a flash on the eye, e.g., on the anterior surface of the eye (e.g., the corneal surface of the eye). The second coupling optical element 2111b may image the retina of the user's eye, as discussed above. In some embodiments, the second coupling optical element 2111b and / or the lens disposed in front of it may have a certain amount of net refractive power (e.g., a non-zero amount of net refractive power) that is weaker than that of the first coupling optical element 2111a and / or the lens disposed in front of the first coupling optical element 2111a.

[0261] In some implementations, the image formed by the light collected by the first coupling optical element 2111a is adjacent to (e.g., does not overlap across) the image formed by the light collected by the second coupling optical element 2111b. For example, an image of the anterior surface of the user's eye (e.g., the cornea) may be formed adjacent to (e.g., without overlapping across) the image of the retina.

[0262] The polarization technique can be used to attenuate or remove light from the anterior surface (e.g., the corneal surface) so as not to affect the image formed by the light collected by the second coupling optical element 2111b. For example, the eye can be illuminated with polarized light having a first polarization, and the camera can form an image using light from a second external coupling optical element 2120b that uses light of a second different polarization. For example, the second external coupling optical element 2111b may be a polarization selective coupling element that selectively externally couples light of a second polarization different from the first polarization. Additionally, or alternatively, a polarizer 2140 that filters out (e.g., selectively transmits the second polarization) the first polarization may be included between the second external coupling optical element 2120b and the camera 920, as shown in FIG. 27, which is a cross-section through the waveguide 950, 2102, the condenser optical elements 2116a, 2116b, and the external coupling optical elements 2120a, 2120b shown in FIG. 26.

[0263] Such a configuration may be used to reduce unwanted reflections (e.g., glare) from the cornea, etc. when imaging the retina. The reflection from the cornea will be a specular reflection. Thus, when light of the first polarization is incident on the cornea, the light reflected from the cornea will retain the first polarization. In contrast, the retina is diffusive. When light of the first polarization is incident on the retina, the light reflected from the retina will not simply retain the first polarization. Diffuse reflection is more likely to result in non-polarized light. Thus, a second polarization different from the first polarization will be present in the light reflected from the retina. As a result, an image of the retina will be acquired by forming an image using the light externally coupled from the second external coupling optical element 2120b that uses light of the second polarization, while an image of the cornea or glare will be suppressed. Similarly, by illuminating with the first polarization and imaging with a second different polarization, the retina can be imaged with reduced glare from the cornea.

[0264] Thus, in various implementations, a polarization-specific optical filter or a polarization-selective optical element (e.g., a grating) may be used to reduce unwanted light reflected from the eye 210 (e.g., the cornea). For example, unwanted light, glare, or flash may be reflected from the cornea such that it saturates an image captured by the camera. As discussed above, light reflected from the cornea is a specular reflection and may maintain its polarization. In contrast, light reflected from the retina may be more diffusely reflected and may not be polarized as homogeneously. Similarly, a combination of polarizers may be used to remove some or most of the unwanted light reflected from the cornea. First, polarization can be used to illuminate the user's eye. In some designs, a polarized illumination source (e.g., a light source) may be used. Additionally, or alternatively, a first polarizer (e.g., a polarization-specific optical filter or a polarization-selective optical coupling element that couples illumination light into an illumination waveguide) may be positioned at the start of the optical path of the illumination source to provide an initial polarization of light to the eye. A second polarizer (e.g., a polarization-specific optical filter or a polarization-selective coupling element) may be positioned in the optical path before the light enters the camera. The second polarizer is 90 oIt may be rotated (e.g., the polarizer may be "crossed"). As a result, the eye will be illuminated with a first polarization of light with the first polarized light reflected from the cornea. This light will not pass through a crossed polarizer (which preferentially passes light of a second polarization) located proximal to the camera. However, the light reflected from the retina will include a second polarization. Similarly, the light diffusely reflected from the retina will pass through the polarizer 2140 proximal to the camera and will enable an image of the retina to be captured by the camera. Thus, in such a configuration, unwanted light received from the eye (e.g., the cornea) and incident on the camera can be reduced or eliminated from image capture using the light from the second coupling optical element 2111b. Other configurations are also conceivable. For example, the polarization selective coupling element 2111b and / or the polarization selective external coupling optical element 2120b may be used in addition to or instead of a polarizer such as the polarizer 2140 proximal to the camera. The effect can again be to reduce or remove unwanted light received from the eye (e.g., the cornea) before it is incident on the imaging device 920.

[0265] As shown in FIG. 27, in various implementations, such a polarizer is not used within the optical path from the first coupling optical element 2111a and the camera or between the first external coupling optical element 2120a and the camera. As a result, images of the cornea and the flash can be obtained from the light coupled into the waveguide by the first coupling optical element 2111a and / or the light coupled out of the waveguide from the first external coupling optical element 2120a. As discussed above, this first coupling optical element 2111a may have a refractive power or a lens associated therewith that is specifically used to image the cornea and / or the flash. Similarly, a polarization selective coupling optical element 2111a or a polarization selective external coupling optical element 2120a that filters out the first polarized light will not be used as the first coupling optical element 2111a and the external coupling optical element 2120a, respectively. In addition, a polarizer 2140 between the coupling optical element 2111a and the camera 920 or between the external coupling optical element 2111a and the camera 920 that filters out the first polarized light will not be used.

[0266] Various modifications are possible. For example, the first and second external coupling optical elements 2120a, 2120b are described above as coupling light and forming an image on a single camera (e.g., a single detection area 2130), but in other implementations, the first and second external coupling optical elements 2120a, 2120b can couple light and form an image on separate first and second cameras. Other modifications are also possible.

[0267] In some implementations, one or more of the eyepieces described above with reference to FIGS. 24-27 (e.g., the eyepiece 950) may be a dedicated imaging eyepiece layer (e.g., omitting the internal coupling optical element 2104, the light dispersing element 2108, and / or the external coupling element 2110). In such an implementation, the imaging eyepiece layer may be included as a layer within the stack of waveguides. One or more other layers within the stack of waveguides may include the internal coupling optical element 2104, the light dispersing element 2108, and / or the external coupling element 2110.

[0268] In some embodiments, the dedicated imaging eyepiece layer can be configured to capture an image of the environment. In some such embodiments, the imaging eyepiece layer may be disposed closest to the environment when disposed within the head-mounted display. In some such configurations, the dedicated imaging eyepiece layer may be the outermost layer within the waveguide stack such that it is positioned between the other layer (e.g., waveguide) and the environment. In some embodiments, the dedicated imaging eyepiece layer can be configured to capture an image of the user's eye 210. In some such embodiments, the imaging eyepiece layer may be disposed closest to the user when disposed within the head-mounted display. In some implementations, the dedicated imaging eyepiece layer may be the innermost layer within the waveguide stack such that it is positioned between the other layer (e.g., waveguide) and the user. Other configurations are also possible.

[0269] In one embodiment, the first and second coupling elements 2111a, 2111b may be laterally aligned but displaced in depth (e.g., along the z-axis, i.e., towards the other side of the page of FIG. 26). For example, in such an embodiment, the first and second coupling elements 2111a, 2111b may be positioned on opposite sides of the same waveguide. In one embodiment, the first input coupling element 2111a may be disposed on or within the first waveguide, and the second input coupling element 2111b. Additionally, or alternatively, the first and second condenser elements 2116a, 2116b and / or the first and second external coupling optical elements 2120a, 2120b may be disposed on opposite sides of the same waveguide and / or on or within corresponding separate waveguides.

[0270] Referring to FIGS. 24 - 27, one or more of the coupling optical elements 2111, 2111a, 2111b described above may be wavelength - selective such that the optical element is configured to interact only with a certain wavelength or band of wavelengths. The wavelength or band of wavelengths may include non - visible light (e.g., infrared light or a specific band thereof). In some implementations, the first input coupling element 2111a may be configured to act at a lower wavelength than that for the second input coupling element 2111b (or vice versa). For example, the first coupling optical element 2111a may be configured to interact with light of about 800 nm, and / or the second coupling optical element 2111b may be configured to interact with light of about 950 nm (or vice versa). Optical filters may also be used to provide similar wavelength selection. In various implementations, thus, the light coupled into the waveguide by the first coupling optical element that reaches at least one may be of a different wavelength than the light coupled into the waveguide by the second optical element that reaches at least one camera. Other configurations are also possible.

[0271] In the foregoing specification, the invention has been described with reference to specific embodiments. However, it will be apparent that various modifications and changes can be made therein without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a limiting sense.

[0272] Indeed, it is to be understood that the systems and methods of the present disclosure each have several innovative aspects, none of which alone is involved in or required for the desirable attributes disclosed herein. The various features and processes described above can be used independently of each other or combined in various ways. All possible combinations and sub - combinations are intended to fall within the scope of the present disclosure.

[0273] Certain features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. Further, features may be described above as acting in a certain combination and may further be exemplified as such, but one or more features from the exemplified combination may in some cases be deleted from the combination, and the exemplified combination may be directed to a sub-combination or a variation of a sub-combination. No single feature or group of features is necessary or essential to all embodiments.

[0274] In particular, conditional statements used herein such as "can", "could", "might", "may", "e.g.", and equivalents, unless specifically stated otherwise or understood otherwise within the context in which they are used, are generally intended to convey that while one embodiment includes certain features, elements, and / or steps, other embodiments do not include them. Thus, such conditional statements are generally not intended to imply that features, elements, and / or steps are required in any way for one or more embodiments, or that one or more embodiments necessarily include logic for determining whether these features, elements, and / or steps are to be included or implemented in any particular embodiment, regardless of the author's input or prompting. The terms "comprising", "including", "having", and equivalents are synonyms and are used inclusively in a non-limiting manner, and do not exclude additional elements, features, acts, operations, etc. Also, the term "or" is used in its inclusive sense (and not in its exclusive sense), and thus, for example, when used to connect a list of elements, the term "or" means one, some, or all of the elements in the list. Additionally, the articles "a", "an", and "the" as used in this application and the appended examples should be construed to mean "one or more" or "at least one" unless otherwise defined. Similarly, operations may be depicted in the drawings in a particular order, but it should be recognized that this is not necessary for achieving the desired result, and that such operations may be performed in the particular order shown or in a sequential order, or that all of the illustrated operations need not be performed. Further, the drawings may schematically depict one or more exemplary processes in the form of flowcharts. However, other operations not depicted may also be incorporated into the exemplary methods and processes schematically illustrated. For example, one or more additional operations may be performed before, after, simultaneously with, or during any of the illustrated operations.In addition, the operations may be rearranged or reordered in other embodiments. In certain situations, multitasking and parallel processing may be advantageous. Further, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described may generally be integrated together in a single software product or packaged into multiple software products. In addition, other implementations are within the scope of the following examples. In some cases, the actions recited in the examples may be performed in a different order and still achieve desirable results.

[0275] Accordingly, the present disclosure is not intended to be limited to the implementations shown herein and should be accorded the widest scope consistent with the present disclosure, principles, and novel features disclosed herein. Various exemplary systems and methods are provided below. (Example)

Example

[0276] (Example 1) A head-mounted display system configured to project light into a user's eye and display augmented reality image content within the user's field of view, a frame configured to be supported on the user's head, an image projector configured to project an image into the user's eye and display image content within the user's field of view, a camera, at least one waveguide, at least one coupling optical element configured such that light is coupled into and guided within the waveguide, at least one external coupling element configured to couple light guided within the waveguide out of the waveguide and direct the light toward the camera. A head-mounted display system comprising a camera disposed in an optical path with respect to at least one external coupling optical element for receiving at least a portion of light, wherein the image is coupled into the waveguide via a coupling element, guided therein, and coupled out of the waveguide by the external coupling element so that the image can be captured by the camera. (Example 2) The system according to Example 1, wherein the at least one coupling optical element is configured such that light from the front environment of a user wearing the head-mounted display system is coupled into and guided within the at least one waveguide so that an image of the environment can be captured by the camera. (Example 3) The system according to any of the above examples, wherein the at least one coupling optical element is configured such that light reflected from the eyes of a user wearing the head-mounted display system is coupled into and guided within the at least one waveguide so that an image of the eyes can be captured by the camera. (Example 4) The system according to any of the above examples, wherein the at least one coupling optical element is configured such that light reflected from the eyes of a user wearing the head-mounted display system is coupled into and guided within the waveguide so that an image of the eyes can be captured by the camera, and the system is configured to form an image of the anterior portion of the eyes. (Example 5) The system according to any of the above examples, wherein the at least one coupling optical element is configured such that light reflected from the eyes of a user wearing the head-mounted display system is coupled into and guided within the waveguide so that an image of the eyes can be captured by the camera, and the system is configured to form an image of the corneal surface of the eyes. (Example 6) The system according to any of the above examples, wherein the at least one coupling optical element is configured such that light reflected from the eyes of a user wearing the head-mounted display system is coupled into and guided within the waveguide so that an image of the eyes can be captured by the camera, and the system is configured to form an image of the retina of the user's eyes. (Example 7) The system according to any of the above embodiments, further comprising an eyepiece lens disposed on the frame, the eyepiece lens being configured to direct light into the user's eye and display augmented reality image content in the user's field of view, at least a part of the eyepiece lens being transparent, the transparent part allowing light from the user's front environment to pass through to the user's eye and providing a view of the user's front environment, and being disposed at a location in front of the user's eye when the user wears the head-mounted display. (Example 8) The system according to Example 7, wherein the eyepiece lens is configured to receive light from the image projector and direct the light into the user's eye to display augmented reality image content in the user's field of view. (Example 9) The system according to any of Examples 7 - 8, wherein the eyepiece lens comprises the at least one waveguide. (Example 10) The system according to any of Examples 7 - 9, wherein the image projector is configured to direct light into the edge of the eyepiece lens. (Example 11) The system according to Example 9 or 10, wherein the image projector is configured to direct light into the edge of the at least one waveguide. (Example 12) The system according to any of the above embodiments, further comprising at least one internal coupling optical element configured to internally couple light from the image projector into the at least one waveguide so as to guide the light from the image projector to provide the image content to the user's eye. (Example 13) The system according to any of the above embodiments, wherein the at least one coupling optical element is also configured to couple the light from the image projector guided within the waveguide out of the at least one waveguide so that the image content can be visually recognized by the user's eye. (Example 14) The same coupling optical element is configured to couple the light from the image projector guided in the waveguide out of the waveguide so that the image content can be visually recognized by the user's eyes, and to couple the light into the at least one waveguide so that it is guided into the camera. The system according to any of the above embodiments. (Example 15) The system according to any of Examples 1-12, further comprising at least one external image content coupling optical element configured to couple the light from the image projector guided in the waveguide out of the at least one waveguide so that the image content can be visually recognized by the user's eyes. (Example 16) The system according to any of the above embodiments, wherein the at least one coupling optical element faces the eyes of a user wearing the head-mounted imaging system and receives light from the eyes. (Example 17) The system according to any of the above embodiments, wherein the at least one coupling optical element is configured such that light from the environment in front of the user wearing the head-mounted imaging system is coupled into the at least one waveguide and guided therein so that an image of the environment can be captured by the camera. (Example 18) The system according to any of the above embodiments, wherein the at least one coupling optical element faces the environment in front of the user wearing the head-mounted imaging system and receives light from the environment. (Example 19) In the system according to Example 15, the at least one external image content coupling optical element configured to couple the light from the image projector guided in the waveguide out of the at least one waveguide and the at least one coupling optical element configured such that light is coupled into the waveguide and guided therein into the camera are superimposed on each other. (Example 20) The at least one image content external coupling optical element configured to couple the light from the image projector induced within the waveguide out of the at least one waveguide, and the at least one coupling optical element configured such that light is coupled into the waveguide and guided therein to the camera are stacked over each other, the system according to Example 15. (Example 21) The at least one image content external coupling optical element configured to couple the light from the image projector induced within the waveguide out of the at least one waveguide, and the at least one coupling optical element configured such that light is coupled into the waveguide and guided therein to the camera are integrated within the same diffractive optical element, the system according to Example 15. (Example 22) The at least one coupling optical element is configured such that light is coupled into the first waveguide and guided therein to the camera, and the at least one image content external coupling optical element is configured to couple the light from the image projector induced within the second waveguide out of the second waveguide, the system according to Example 15. (Example 23) The at least one coupling optical element is configured such that light is coupled into the first waveguide and guided therein to the camera, and the image projector is configured to couple light into the second waveguide and provide image content to the eye, the system according to any of the above examples. (Example 24) The image projector includes a light source, a modulator, and projection optics, the system according to any of the above examples. (Example 25) The image projector includes a scanning optical fiber, the system according to any of the above examples. (Example 26) The modulator includes an optical modulator, the system according to either Example 24 or 25. (Example 27) The optical modulator includes a spatial light modulator, the system according to Example 26. (Example 28) The camera includes a detector array and imaging optics, the system according to any of the above examples. (Example 29) The imaging optical system is the system described in Example 28 configured to focus collimated light onto the detector array. (Example 30) The at least one waveguide includes a material that is transparent to visible light and has a refractive index sufficient to guide light within the waveguide by total internal reflection, in any of the systems described in the above examples. (Example 31) The at least one waveguide includes a stack of waveguides, in any of the systems described in the above examples. (Example 32) Different waveguides in the stack of waveguides are configured to output light with different wavefront divergences as if projected from different distances from the user's eye, in the system described in Example 31. (Example 33) Different waveguides in the stack of waveguides are configured to output light with different colors, in the system described in Example 31 or 32. (Example 34) The different waveguides include first, second, and third waveguides, and the system is configured such that the first waveguide is for red light, the second waveguide is for green light, and the third waveguide is for blue light, in any of the systems described in Example 31, 32, or 33. (Example 35) The internal coupling optical element includes a diffractive optical element or a reflector, in any of the systems described in Examples 12 - 34. (Example 36) The internal coupling optical element includes a diffractive optical element, in any of the systems described in Examples 12 - 34. (Example 37) The coupling optical element includes a diffractive optical element, in any of the systems described in the above examples. (Example 38) The coupling optical element includes a liquid crystal, in any of the systems described in the above examples. (Example 39) The coupling optical element includes a liquid crystal polarization grating, in any of the systems described in the above examples. (Example 40) The external coupling optical element includes a diffractive optical element, in any of the systems described in the above examples. (Example 41) The coupling optical element includes a liquid crystal, in any of the systems described in the above examples. (Example 42) The coupling optical element is the system according to any of the above embodiments, comprising a liquid crystal polarization grating. (Example 43) The coupling element is the system according to any of the above embodiments, configured to increase the size of the eye box along at least one axis. (Example 44) The system according to Example 43, further comprising an orthogonal pupil expander, comprising at least one optical re-directing element configured to increase the size of the eye box along an axis orthogonal to at least one axis, within or on the at least one waveguide. (Example 45) The system according to Example 44, wherein the at least one optical re-directing element comprises a diffractive optical element. (Example 46) The coupling element is the system according to any of the above embodiments, configured to (a) couple light into the at least one waveguide for reception by the camera, and (b) couple light from the image projector out of the at least one waveguide to the user's eye. (Example 47) The coupling element is the system according to any of the above embodiments, configured to (a) couple light from the environment into the at least one waveguide for reception by the camera, and (b) couple light from the image projector out of the at least one waveguide to the user's eye. (Example 48) The coupling element is the system according to any of the above embodiments, configured to (a) couple light from the eye into the at least one waveguide for reception by the camera, and (b) couple light from the image projector out of the at least one waveguide to the user's eye. (Example 49) The system according to any of the above embodiments, further comprising a reflective surface having a refractive power, arranged to receive light reflected from the user's eye passing through the eyepiece and direct the light back to the eyepiece. (Example 50) The system according to Example 49, wherein the at least one coupling element is configured such that light from the user's eye, which passes through at least one waveguide and is reflected back from a reflective surface into at least one waveguide, is coupled into and guided within the at least one waveguide. (Example 51) The system according to any one of Examples 49 - 50, wherein the camera is disposed in an optical path with respect to at least one external coupling optical element to receive at least a portion of the light from the user's eye that is reflected from a reflective surface, coupled into and guided within the waveguide via a coupling element, and coupled out of the waveguide by the external coupling element. (Example 52) The system according to any one of Examples 49 - 51, wherein the reflective surface reflects infrared light and transmits visible light. (Example 53) The system according to any one of Examples 49 - 52, wherein the reflective surface is curved. (Example 54) The system according to any one of Examples 49 - 53, wherein the reflective surface is disposed on a curved optical element. (Example 55) The system according to any one of Examples 49 - 54, wherein the reflective surface is disposed on a concave mirror. (Example 56) The system according to any one of Examples 49 - 55, wherein the reflective surface has a positive refractive power upon reflection and a negligible refractive power upon transmission. (Example 57) The system according to any one of Examples 49 - 56, wherein the reflective surface is configured to collimate the light from the user's eye. (Example 58) The system according to any one of Examples 49 - 57, wherein the reflective surface is configured to collimate the light from the retina of the user's eye. (Example 59) The system according to any one of Examples 49 - 58, wherein the reflective surface is configured to collimate the light from the anterior region of the user's eye. (Example 60) The system according to any one of Examples 49 - 59, wherein the reflective surface is configured to collimate the light from the cornea of the user's eye. (Example 61) The system according to any one of Examples 49 - 60, wherein the reflective surface is formed on a curved optical element and comprises an infrared reflective coating. (Example 62) The system according to Example 61, wherein the curved optical element has a negligible refractive power with respect to the light transmitted therethrough. (Example 63) The system according to Example 61 or 62, wherein the curved optical element has first and second curved surfaces on opposite sides of the curved optical element, and the first and second curved surfaces have the same curvature. (Example 64) The system according to any one of Examples 49 - 63, further comprising a retarder disposed with respect to the reflective surface and the coupling optical element so as to rotate the polarization of light that passes through at least one waveguide and is reflected from the reflective surface back to at least one waveguide and the coupling optical element. (Example 65) The system according to any one of the above examples, wherein at least one coupling element comprises a polarization - selective direction - converting element. (Example 66) The system according to any one of the above examples, wherein at least one coupling element comprises a polarization grating. (Example 67) The system according to any one of the above examples, wherein at least one coupling element is configured to deflect light guided in at least one waveguide as collimated light directed towards the user's eye out of the waveguide and towards the eye. (Example 68) The system according to any one of the above examples, wherein at least one coupling element is configured to deflect collimated light from the reflective surface into at least one waveguide. (Example 69) The system according to any one of the above examples, wherein at least one external coupling element comprises an off - axis reflector. (Example 70) The system according to any one of the above examples, wherein at least one external coupling element comprises a polarization - selective direction - converting element. (Example 71) The system according to any one of the above examples, wherein at least one external coupling element comprises a polarization grating. (Example 72) The system according to any of the above embodiments, wherein at least one external coupling element comprises a liquid crystal. (Example 73) The system according to any of the above embodiments, wherein at least one external coupling element comprises a liquid crystal polarization grating. (Example 74) The system according to any of the above embodiments, further comprising a circular polarizer. (Example 75) The system according to any of the above embodiments, wherein the internal coupling element comprises a polarization selective direction conversion element. (Example 76) The system according to any of the above embodiments, wherein the internal coupling element comprises a polarization grating. (Example 77) The system according to any of the above embodiments, wherein at least one internal coupling element comprises a diffractive optical element. (Example 78) The system according to any of the above embodiments, wherein at least one internal coupling element comprises a diffraction grating. (Example 79) The system according to any of the above embodiments, wherein the internal coupling element comprises an off-axis reflector. (Example 80) The system according to any of Examples 49-79, wherein the reflective surface comprises a liquid crystal reflector. (Example 81) The system according to any of Examples 49-80, wherein the reflective surface comprises a cholesteric liquid crystal reflective lens. (Example 82) The waveguide (a) guides light coupled from the user's eye into the at least one waveguide for reception by the camera so as to capture at least a partial image of the user's eye, and (b) guides light coupled from the image projector so that light from the image projector can be directed towards the user's eye and the image from the image projector is within the user's field of view. The system according to any of the above embodiments. (Example 83) The coupling element (a) couples light from the user's eye into the at least one waveguide for reception by the camera, and (b) couples light from the at least one waveguide out to the user's eye. The system according to any of the above embodiments. (Example 84) The system according to any one of Examples 49 - 83, further comprising an electronic device configured to cause the camera to capture a first image when light reflected from a reflective surface is blocked. (Example 85) The system according to Example 84, wherein the electronic device is configured to cause the camera to capture a second image when light reflected from a reflective surface is not blocked. (Example 86) The system according to Example 85, wherein the electronic device is configured to modify the second image using the first image. (Example 87) The system according to Example 85 or 86, wherein the electronic device is configured to subtract from the second image based on the first image. (Example 88) The system according to any of the above examples, configured to perform eye tracking based on an image of the eye. (Example 89) The system according to Example 88, wherein the step of performing eye tracking based on the image of the eye includes the step of storing an image of the retina of the eye. (Example 90) The system is configured to obtain an image of a part of the retina of the eye using the camera, compare the image of the part of the retina with one or more stored images of the retina, and determine the user's line of sight based on a comparison between one or more stored images and the image of the part of the retina obtained from the camera, as described in any of the above examples. (Example 91) The system according to Example 90, wherein the step of determining the user's line of sight includes the step of determining the part of the retina corresponding to the image of the part of the retina. (Example 92) The system according to any of Examples 90 - 91, wherein the step of determining the user's line of sight includes the step of determining the orientation of the eye. (Example 93) The system according to any of the above examples, configured to obtain biometric data based on one or more images of the user's eye obtained using the camera. (Example 94) The system is the system according to any of the above embodiments, configured to identify a user via biometric sensing based on one or more images of the eye obtained using the camera. (Example 95) The system is the system according to any of the above embodiments, configured to preferentially capture an image using the camera with light of a second polarization different from the first polarization, and providing illumination of the first polarization. (Example 96) The system is the system according to any of the above embodiments, configured to preferentially capture an image of the user's eye using the camera with light of a second polarization different from the first polarization, and illuminating the user's eye with light of the first polarization. (Example 97) The system according to Example 95 or 96, wherein the first and second polarizations are orthogonal. (Example 98) The system according to any of the above embodiments, further comprising a light source arranged to provide illumination so as to capture an image using the camera. (Example 99) The system according to any of the above embodiments, further comprising a light source arranged to illuminate the user's eye. (Example 100) The system according to Example 98 or 99, wherein the light source comprises one or more infrared light sources. (Example 101) The system according to any of Examples 98 - 100, wherein the light source comprises one or more infrared light emitting diodes (LEDs). (Example 102) The system according to any of Examples 98 - 101, wherein the light source is pulse - oscillated. (Example 103) The system according to any of Examples 98 - 102, further comprising an off - axis reflector arranged to receive light from the light source and arranged to illuminate the user's eye with the light. (Example 104) The system according to any of Examples 98 - 103, wherein the light source is configured to input light into an optical waveguide and provide the illumination. (Example 105) The system according to any of Examples 98 - 104, wherein the light source is configured to input light into an optical waveguide arranged with respect to the eye and provide illumination to the eye. (Example 106) The system according to Example 104 or 105, further comprising an illumination internal coupling optical element configured to couple light from the light source into the waveguide. (Example 107) The system according to any one of Examples 98 - 103, wherein the light source is configured to input light into at least one of the waveguides to provide illumination. (Example 108) The system according to Example 107, further comprising an illumination internal coupling optical element configured to couple light from the light source into at least one of the waveguides to provide illumination. (Example 109) The system according to any one of Examples 98 - 103, wherein the light source is configured to input light into the same waveguide as that used to project image content onto the user's eye. (Example 110) The system according to any one of Examples 98 - 104, wherein the light source is configured to input light into the same waveguide as that used to guide light to the camera in order to provide illumination to the user's eye. (Example 111) The system according to any one of Examples 98 - 105, wherein the light source is configured to input light into the same waveguide as that used to guide light from the user's eye to the camera. (Example 112) The system according to any one of Examples 109 - 111, further comprising an illumination internal coupling optical element configured to couple light from the light source into the waveguide. (Example 113) The system according to any one of Examples 106, 108, or 112, wherein the illumination internal coupling optical element is polarization selective and internally couples light of a first polarization. (Example 114) The system according to any one of Examples 98 - 113, wherein the light source is a polarized light source configured to output polarized light having a first polarization. (Example 115) The system according to any one of Examples 98 - 114, wherein the light source is configured to direct polarized light having a first polarization onto the eye. (Example 116) The system according to any one of Examples 98 - 115, further comprising an illumination polarizer having a first polarization, which is disposed in the optical path between the light source and the eye and polarizes the light directed to the eye. (Example 117) The system according to Example 116, wherein the illumination polarizer is disposed in the optical path between the light source and the waveguide and is configured to provide illumination. (Example 118) The system according to any one of Examples 98 - 117, further comprising an image acquisition polarizer in the optical path between the eye and the camera. (Example 119) The system according to Example 118, wherein the image acquisition polarizer is proximal to the camera. (Example 120) The system according to Example 118 or 119, wherein the image acquisition polarizer is disposed in the optical path between (a) the at least one waveguide configured to direct light to the camera and (b) the camera. (Example 121) The system according to any one of Examples 118 - 120, wherein the image acquisition polarizer reduces the amount of light of the first polarization reaching the camera. (Example 122) The system according to any one of Examples 118 - 121, comprising a polarizer configured to selectively couple light of a second polarization different from the first polarization to the camera. (Example 123) The system according to any of the above examples, further comprising at least one condenser element disposed in the optical path between the at least one coupling element and the at least one external coupling optical element, and reducing the lateral spatial extent of the light from the at least one coupling element prior to reaching the at least one external coupling optical element. (Example 124) The system according to any of the above examples, wherein the at least one condenser element comprises a diffractive optical element. (Example 125) The system according to any of the above examples, wherein the at least one condenser element comprises a hologram or a diffraction grating. (Example 126) The at least one waveguide includes a material that is transparent to infrared light and has a refractive index sufficient to guide infrared light within the waveguide by total internal reflection, the system according to any of the above embodiments. (Example 127) The at least one coupling optical element includes an exit pupil expander, the system according to any of the above embodiments. (Example 128) The system includes a refractive power for increasing the collimation of light reflected from an eye and coupled into a waveguide and directed to a camera, the system according to any of the above embodiments. (Example 129) The system includes a refractive power for increasing the collimation of light reflected from a front portion of an eye and coupled into a waveguide and directed to a camera, the system according to any of the above embodiments. (Example 130) The system includes a refractive power for increasing the collimation of light reflected from a cornea of an eye and coupled into a waveguide and directed to a camera, the system according to any of the above embodiments. (Example 131) The refractive power includes a positive refractive power, the system according to any of Examples 128 - 130. (Example 132) The refractive power is provided by a lens, the system according to any of Examples 128 - 131. (Example 133) One or more stored images of a retina of an eye include a synthetic image of the retina of the eye generated using a plurality of images of different portions of the retina of the eye, the system according to any of Examples 88 - 132. (Example 134) The synthetic image of the retina includes a plurality of images of the retina stitched together, the system according to any of Examples 88 - 133. (Example 135) The plurality of images of the retina stitched together each include an image obtained when a fixed target is displayed within the user's field of view at various locations, the system according to any of Examples 88 - 134. (Example 136) The system according to any of Examples 88-135, wherein one or more stored images of the retina each comprise an image obtained when a fixed target was displayed within the user's field of view at various locations. (Example 137) The system according to any of Examples 88-136, further configured to update a synthetic image using an acquired image of a portion of the eye's retina. (Example 138) The system according to any of Examples 88-137, wherein the step of updating a synthetic image of the retina using an acquired image of a portion of the retina includes stitching the acquired image into a section of the synthetic image corresponding to the portion of the retina shown in the acquired image. (Example 139) The system according to any of Examples 88-138, further configured to apply a digital filter to an acquired image of a portion of the eye's retina to obtain a filtered image of the portion of the retina. (Example 140) The system according to Example 139, further configured to compare one or more stored images of the retina with the filtered image of the portion of the retina. (Example 141) The system according to any of Examples 139-140, wherein the digital filter comprises a Frangi filter. (Example 142) The system according to any of Examples 88-139, configured to apply an edge to enhance an acquired image of a portion of the retina. (Example 143) The system according to any of the above examples, configured to perform user identification verification using an image of the retina. (Example 144) The system is configured to use the camera to acquire an image of a portion of the retina of the eye, and compare one or more stored images of the retina with the image of the portion of the retina, as configured in any of the above examples. (Example 145) The system according to Example 144, wherein one or more stored images of the retina of an eye comprise a composite image of the retina of the eye generated using a plurality of images of different portions of the retina of the eye. (Example 146) The system according to any one of Examples 144-145, wherein the composite image of the retina comprises a plurality of images of the retina stitched together. (Example 147) The system according to any one of Examples 144-146, wherein the plurality of images of the retina stitched together each comprise an image obtained when a fixation target was displayed within the user's field of view at various locations. (Example 148) The system according to any one of Examples 144-146, wherein one or more stored images of the retina each comprise an image obtained when a fixation target was displayed within the user's field of view at various locations. (Example 149) The system according to any one of Examples 144-148, further configured to update the composite image using an acquired image of a portion of the retina of the eye. (Example 150) The system according to any one of Examples 144-149, wherein the step of updating the composite image of the retina using an acquired image of a portion of the retina comprises stitching the acquired image into a section of the composite image corresponding to the portion of the retina shown in the acquired image. (Example 151) The system according to any one of Examples 144-150, further configured to apply a digital filter to an acquired image of a portion of the retina of the eye to obtain a filtered image of the portion of the retina. (Example 152) The system according to Example 151, further configured to compare one or more stored images of the retina with the filtered image of the portion of the retina. (Example 153) The system according to any one of Examples 144-152, wherein the digital filter comprises a Frangi filter. (Example 154) The system according to any one of Examples 144-153, configured to apply an edge to enhance an acquired image of a portion of the retina.

Example

[0277] (Example 1) A head-mounted display system configured to project light onto a user's eyes, display augmented reality image content within the user's field of view, and image at least a part of the environment in front of a user wearing the head-mounted display system, A frame configured to be supported on the user's head, An image projector configured to project an image, A camera, An eyepiece lens disposed on the frame, the eyepiece lens being configured to direct light into the user's eyes and display augmented reality image content in the user's field of view, at least a part of the eyepiece lens being transparent, the transparent portion transmitting light from the environment in front of the user to the user's eyes and providing a view of the environment in front of the user, and when the user wears the head-mounted display, being disposed at a location in front of the user's eyes, (a) At least one waveguide, (b) At least one internal coupling optical element configured to internally couple light from the image projector into the at least one waveguide so as to guide the light from the image projector therein, (c) At least one coupling optical element configured to couple the light from the image projector guided within the waveguide out of the waveguide and direct the light to the user's eyes, (d) At least one external coupling element configured to couple the light within the waveguide out of the waveguide and direct the light to the camera, An eyepiece lens comprising, Comprising, The image projector is disposed in the optical path with respect to the at least one internal coupling optical element so that the light is coupled out of the waveguide to the user's eyes by the at least one coupling element and the image from the projector is guided therein so as to be within the user's field of view, and couples the light from the image projector into the waveguide, The coupling element is configured such that light from the environment in front of the user wearing the head-mounted display is coupled into and guided within the waveguide. The camera is disposed in the optical path with respect to the at least one external coupling optical element such that an image of the environment can be captured by the camera, and at least a part of the light from the environment in front of the user is received, coupled into and guided within the waveguide via the coupling element, and coupled out of the waveguide by the external coupling element. The waveguide (a) guides the light coupled from the environment into the waveguide for reception by the camera so as to capture at least a partial image of the environment in front of the user, and (b) guides the light coupled from the projector such that the light from the projector can be directed towards the user's eyes and the image from the projector is within the user's field of view. A head-mounted display system. (Example 2) The image projector includes a light source, a modulator, and projection optics, and the system according to Example 1. (Example 3) The image projector includes a scanning optical fiber, and the system according to Example 1 or 2. (Example 4) The modulator includes an optical modulator, and the system according to any one of Examples 2 or 3. (Example 5) The optical modulator includes a spatial light modulator, and the system according to Example 4. (Example 6) The camera includes a detector array and imaging optics, and the system according to any one of the above examples. (Example 7) The imaging optics is configured to focus collimated light onto the detector array, and the system according to Example 6. (Example 8) The at least one waveguide includes a material that is transparent to visible light and has a refractive index sufficient to guide light within the waveguide by total internal reflection, and the system according to any one of the above examples. (Example 9) The at least one waveguide includes a stack of waveguides, and the system according to any one of the above examples. (Example 10) The system according to Example 9, wherein different waveguides in the stack of waveguides are configured to output light with different wavefront divergences as if projected from different distances from the user's eye. (Example 11) The system according to Example 9 or 10, wherein different waveguides in the stack of waveguides are configured to output light with different colors. (Example 12) The system according to any one of Examples 9, 10, or 11, wherein the different waveguides include first, second, and third waveguides, and the system is configured such that the first waveguide is for red light, the second waveguide is for green light, and the third waveguide is for blue light. (Example 13) The system according to any of the above examples, wherein the internal coupling optical element comprises a diffractive optical element or a reflector. (Example 14) The system according to any of the above examples, wherein the coupling optical element comprises a diffractive optical element. (Example 15) The system according to any of the above examples, wherein the external coupling optical element comprises a diffractive optical element. (Example 16) T...

Claims

【Claim 1】 The invention described in this specification.

Images