Augmented reality display having a waveguide configured to capture images of the eye and / or environment

Abstract

A head mounted display system may include a camera, at least one waveguide, at least one coupling optical element configured such that light is coupled into and guided in the waveguide, and at least one outcoupling element. The at least one outcoupling element may be configured to couple light guided within the waveguide out of the waveguide and guide the light to the camera. The camera may be disposed in an optical path relative to the at least one outcoupling optical element to receive at least a portion of the light that is coupled into and guided in the waveguide via the coupling element and that is coupled out of the waveguide by the outcoupling coupling element so that an image may be captured by the camera.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority under 35 U.S.C. §119(e) to U.S. Provisional Application No. 62 / 561,645, filed on September 21, 2017, entitled “AUGMENTEDREALITY DISPLAY WITH EYEPIECE CONFIGURED TO CAPTURE IMAGES OF EYE AND ENVIRONMENT,” the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates to optical devices including augmented reality imaging and visualization systems. Background Art

[0004] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images, or portions thereof, are presented to a user in a manner that appears to be real or can be perceived as real. Virtual reality (or "VR") scenes generally involve the presentation of digital or virtual image information that is opaque to other actual real-world visual input; augmented reality (or "AR") scenes generally involve the presentation of digital or virtual image information as an enhancement to the visualization of the actual world around the user. A mixed reality (or "MR") scene is an AR scene and generally involves virtual objects that are integrated into and responsive to the natural world. For example, a MR scene may include AR image content that appears to be blocked by an object in the real world or is otherwise perceived to interact with the object.

[0005] refer to Figure 1 , an augmented reality scene 10 is shown. A user of AR technology sees a real-world park-like setting 20 featuring people, trees, buildings in the background, and a concrete platform 30. The user also perceives that he / she "sees" "virtual content" such as a robot statue 40 standing on the real-world platform 30, and a cartoon-like avatar character 50 flying by, which appears to be an avatar of a bumblebee. These elements 50, 40 are "virtual" because they do not exist in the real world. Because the human visual perception system is complex, it is challenging to produce AR technology that facilitates comfortable, natural, rich presentation of virtual image elements along with other virtual or real-world image elements.

[0006] The systems and methods disclosed herein address various challenges associated with AR and VR technologies.

[0007] Polarizing beam splitters may be used in display systems to direct polarized light to a light modulator, which then directs the light to a viewer. Generally, there is a continuing desire to reduce the size of display systems, and thus, there is a continuing desire to reduce the size of components of display systems, including components that utilize polarizing beam splitters. Summary of the invention

[0008] Various implementations described herein include a display system configured to provide illumination and / or image projection to the eye. Additionally or alternatively, the display system can image the eye and / or the environment.

[0009] In some embodiments, a head-mounted display system is configured to project light into the user's eyes to display augmented reality image content in the user's field of view. The head-mounted display system may include a frame configured to be supported on the user's head. The display system may also include an image projector configured to project an image into the user's eyes to display image content in the user's field of view. The display system may include a camera, at least one waveguide, at least one coupling optical element configured so that light is coupled into the waveguide and guided therein, and at least one out-coupling element. The at least one out-coupling element may be configured to couple the light guided in the waveguide out of the waveguide and guide the light to the camera. The camera may be disposed in an optical path relative to the at least one out-coupling optical element to receive at least a portion of the light, which is coupled into the waveguide via a coupling element and guided therein and coupled out of the waveguide by the out-coupling element, so that an image can be captured by the camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A user view of augmented reality (AR) through an AR device is shown.

[0011] Figure 2 An example of a wearable display system is shown.

[0012] Figure 3 A conventional display system for simulating a three-dimensional image for a user is shown.

[0013] Figure 4 Aspects of a method of simulating a three-dimensional image using multiple depth planes are shown.

[0014] Figures 5A-5C The relationship between the radius of curvature and the focal radius is shown.

[0015] Figure 6 An example of a waveguide stack for outputting image information to a user is shown.

[0016] Figure 7 An example of an outgoing light beam output by a waveguide is shown.

[0017] Figure 8 An example of a stacked waveguide assembly is shown where each depth plane includes an image formed using multiple different component colors.

[0018] Fig. 9A A cross-sectional side view of an example of a stacked waveguide group is shown, each stacked waveguide including an incoupling optical element. As discussed herein, a waveguide stack can include an eyepiece.

[0019] Fig. 9B Shows Fig. 9A A perspective view of an example of multiple stacked waveguides.

[0020] Fig. 9C Shows Fig. 9A and 9B A top plan view of an example of multiple stacked waveguides.

[0021] Fig.10 Schematically illustrated is a cross-sectional side view of an example imaging system including an eyepiece, an image projector, a light source for illuminating the eye, and a camera for capturing an image of the eye.

[0022] Fig.11A A light source for illuminating the eye and an image projector for injecting an image into the eye are shown schematically, both emitting light towards the coupling optical elements on the waveguide of the eyepiece.

[0023] Fig. 11B The projection light from the light source and from the image projector coupled into the waveguide is schematically shown.

[0024] Fig. 11C Schematic showing how the incoupled light propagates through the waveguide by total internal reflection.

[0025] Fig.11D Light from the light source and from the image projector coupled out of the eyepiece is shown schematically.

[0026] Fig.11E Waveguides and coupling optics are schematically shown, configured to propagate incoupled light at least along the full dimension of the coupling optics (e.g., along the x-direction). Light entering the eye is shown as coming from an extended light source (e.g., the region of the retina that the imaging light will capture).

[0027] Fig. 12A is a cross-sectional view schematically illustrating light exiting the eye, reflecting from the retina and incident on the eyepiece.

[0028] Fig. 12BExample light coupled into a waveguide of an eyepiece is schematically shown.

[0029] Fig. 12C The collimated in-coupled light from the eye is schematically shown, which propagates through the waveguide towards the imaging device.

[0030] Fig.12D The incoupled light from the eye is schematically shown propagating to one or more outcoupling optical elements.

[0031] Fig.12E Light from the eye is schematically shown being coupled out of the waveguide by outcoupling optics and directed to a camera so that an image of the eye (e.g., the retina) can be captured by the camera.

[0032] Fig.13A Schematically shown is how an imaging system can image various parts of the eye (e.g., the retina), which can enable the orientation of the eye to be determined and the eye position to be tracked.

[0033] Fig. 13B A pattern of sequentially displayed fixation targets is shown, which are used to orient the eye in various different directions to image the retina. The resulting images correspond to different parts of the retina. For example, when the eye is oriented in various directions to view fixation targets positioned differently on the display, the images captured by the camera include different parts of the retina. These images can be combined to form a larger map or composite image of the retina.

[0034] Fig.14A A cross-sectional view of an imaging system is schematically shown, the imaging system including an eyepiece and a camera for collecting light from the environment in front of the eyepiece. The light from the environment is shown as being reflected or emitted from one or more physical objects in the environment. The collection of light from objects in the environment in front of the eyepiece can enable an image of the environment to be captured.

[0035] Fig. 14B Schematically shown is light from the environment being coupled into the waveguide of the eyepiece by coupling optics.

[0036] Fig. 14C An imaging system is schematically shown that uses a powered optical element such as a refractive optical element (e.g., a lens such as a wide field lens) in front of the eyepiece to collect light from the environment.

[0037] Fig.15AAn exemplary imaging system is schematically shown, comprising a polarization selective in-coupling optical element for receiving light from an illumination source and coupling the light into a waveguide of an eyepiece. The eyepiece further comprises a polarization selective light coupling element for coupling the light out of the waveguide. A polarizer may be used to polarize the light from the illumination source, and a half-wave retarder may be used to rotate the orientation of the linearly polarized light so that the light is transferred into the waveguide by the polarization selective in-coupling optical element.

[0038] Fig. 15B Light from the eye (e.g., light from the retina illuminated by infrared light from an illumination source) is schematically shown being coupled back into the waveguide and directed to the camera for image capture.

[0039] Fig.16 An imaging system configured to image the front of an eye (e.g., cornea) is schematically shown. The imaging system includes an eyepiece as described above. The imaging system also includes a positive lens for collimating light collected from the front of the eye to be coupled into a waveguide via an optical coupling element and propagated to a camera for image capture. The system also includes a negative lens to offset the positive power introduced by the positive lens and prevent inversion of an image of the environment in front of the eyepiece that would otherwise be caused by the positive lens.

[0040] Fig.17 Another example imaging system is schematically shown that is configured to image the front of the eye (e.g., the cornea). The imaging system includes a curved wavelength selective reflector for collimating light from the front of the eye to be coupled into a waveguide via an optical coupling element and propagated to a camera for image capture. The wavelength selective reflector can operate in a reflective mode for infrared light reflected from the eye and in a transmissive mode for visible light from the environment in front of the user.

[0041] Fig.18 An example imaging system is schematically shown that also includes a curved wavelength selective reflector for collimating light from the front of the eye for coupling into a waveguide via an optical coupling element and propagating to a camera for image capture. Polarization selectivity may be employed to assist in controlling the path of light reflected from the eye. Illumination of the eye is provided via the waveguide rather than via multiple light sources located between the waveguide and the eye, as in Fig.18 shown.

[0042] Fig.19 An imaging system is schematically shown including a shutter to aid in the process of subtracting noise.

[0043] Figures 20A-20E An alternative process for noise subtraction using wavelength modulation in combination with a curved wavelength selective reflector is schematically shown.

[0044] Fig.21 An example eyepiece is shown that can be used to simultaneously project light into a user's eyes to provide image content thereto and simultaneously receive image data of the user's eyes or the environment in front of the user.

[0045] Fig. 22 A cross-sectional side view of an example of a cholesteric liquid crystal diffraction grating (CLCG) having multiple uniform chiral structures is shown.

[0046] Fig.23 An example of an imaging system is shown that includes a forward-facing camera configured to image the wearer's eye using a cholesteric liquid crystal (CLC) off-axis mirror.

[0047] The accompanying drawings are provided to illustrate example embodiments and are not intended to limit the scope of the present disclosure.Throughout the text, like reference numerals refer to like parts. DETAILED DESCRIPTION

[0048] Reference will now be made to the drawings, wherein like reference numerals refer to like parts throughout.

[0049] Figure 2 An example of a wearable display system 60 is shown. The display system 60 includes a display 70, and various mechanical and electronic modules and systems that support the functions of the display 70. The display 70 can be coupled to a frame 80, which can be worn by a display system user or viewer 90 and is configured to position the display 70 in front of the eyes of the user 90. In some embodiments, the display 70 can be considered as glasses. In some embodiments, a speaker 100 is coupled to the frame 80 and is configured to be located near the ear canal of the user 90 (in some embodiments, another speaker (not shown) can optionally be located near the other ear canal of the user to provide stereo / shaped sound control). The display system may also include one or more microphones 110 or other devices to detect sound. In some embodiments, the microphone is configured to allow the user to provide input or commands to the system 60 (e.g., selection of voice menu commands, natural language questions, etc.) and / or may allow audio communication with other people (e.g., users of other similar display systems). The microphone may also be configured as a peripheral sensor to collect audio data (e.g., sound from the user and / or the environment). In some embodiments, the display system may further include a peripheral sensor 120a, which may be separated from the frame 80 and attached to the body of the user 90 (e.g., the head, torso, limbs, etc. of the user 90). In some embodiments, the peripheral sensor 120a may be configured to acquire data representing the physiological state of the user 90. For example, the sensor 120a may be an electrode.

[0050] Continue to refer Figure 2, the display 70 is operably coupled to the local data processing and module 140 via a communication link 130 (such as via a wired lead or a wireless connection), and the local data processing and module 140 can be installed in various configurations, such as being fixedly attached to the frame 80, being fixedly attached to a helmet or hat worn by the user, being embedded in a headset, or otherwise removably attached to the user 90 (e.g., in a backpack configuration, in a belt-coupled configuration). Similarly, the sensor 120a can be operably coupled to the local processing and data module 140 via a communication link 120b (e.g., via a wired lead or a wireless connection). The local processing and data module 140 can include a hardware processor and digital storage such as non-volatile memory (e.g., flash memory or a hard drive), both of which can be used to assist in processing, caching, and storing data. The data includes: a) data captured from sensors (which may, for example, be operably coupled to the frame 80 or otherwise operably attached to the user 90), such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, radios, gyroscopes, and / or other sensors disclosed herein; and / or b) data acquired and / or processed using the remote processing module 150 and / or remote data repository 160 (which includes data related to virtual content), which data may be transmitted to the display 70 after such processing or retrieval. The local processing and data module 140 may be operably coupled to the remote processing module 150 and the remote data repository 160 through communication links 170, 180, such as via a wired or wireless communication link, so that these remote modules 150, 160 are operably coupled to each other and can be used as resources for the local processing and data module 140. In some embodiments, the local processing and data module 140 may include one or more of an image capture device, a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a radio, and / or a gyroscope. In some other embodiments, one or more of these sensors may be attached to the frame 80 or may be a separate structure that communicates with the local processing and data module 140 via a wired or wireless communication path.

[0051] Continue to refer Figure 2In some embodiments, the remote processing module 150 may include one or more processors configured to analyze and process data and / or map image information. In some embodiments, the remote data repository 160 may include a digital data storage facility that may be available via the Internet or other network configuration in a "cloud" resource configuration. In some embodiments, the remote data repository 160 may include one or more remote servers that provide information to the local processing and data module 140 and / or the remote processing module 150, such as information for generating augmented reality content. In some embodiments, all data is stored and all calculations are performed in the local processing and data module, allowing for fully autonomous use from the remote module.

[0052] Reference now Figure 3 , the perception of an image as "three-dimensional" or "3-D" can be achieved by providing a slightly different image presentation to each eye of the viewer. Figure 3 A conventional display system for simulating a three-dimensional image for a user is shown. Two different images 190, 200 - one for each eye 210, 220 - are output to the user. The images 190, 200 are spaced a distance 230 from the eyes 210, 220 along an optical axis or z-axis parallel to the viewer's eye gaze. The images 190, 200 are flat, and the eyes 210, 220 can focus on the images by presenting a single accommodation state. Such a 3D display system relies on the human visual system to combine the images 190, 200 to provide depth perception and / or scale of the combined image.

[0053] However, it should be understood that the human visual system is more complex and providing a realistic perception of depth is more challenging. For example, many viewers of conventional "3-D" display systems find such systems uncomfortable or simply unable to perceive a sense of depth. Without being limited by theory, it is believed that a viewer of an object may perceive the object as "three-dimensional" due to a combination of vergence and accommodation. The vergence movement of the two eyes relative to each other (i.e., the rotation of the eyes so that the pupils move toward or away from each other to converge the eye gaze of the eyes to look at an object) is closely related to the focusing (or "accommodation") of the lens and pupil of the eye. Under normal circumstances, changing the focus of the lens of the eye or adapting the eye to change the focus when switching from one object to another object located at a different distance will automatically result in a matching change in vergence to the same distance under a relationship known as the "accommodation-vergence reflex" and pupil dilation or contraction. Similarly, under normal circumstances, a change in vergence will trigger a matching change in the adaptation of the lens shape and pupil size. As described herein, many stereoscopic or "3-D" display systems display a scene to each eye using a slightly different presentation (and therefore slightly different images) so that the human visual system perceives a three-dimensional perspective. However, such systems are uncomfortable for many viewers because, among other things, such systems simply provide different presentations of the scene, and the eyes view all image information in a single state of accommodation, and violate the "accommodation-vergence reflex." Display systems that provide a better match between accommodation and vergence can form a more realistic and comfortable three-dimensional image simulation.

[0054] Figure 4 Aspects of a method for simulating a three-dimensional image using multiple depth planes are shown. Figure 4, objects at different distances from the eyes 210, 220 on the z-axis are accommodated by the eyes 210, 220 so that those objects are in focus. The eyes 210, 220 present a particular state of accommodation to bring objects at different distances along the z-axis into focus. Thus, it can be said that a particular state of accommodation is associated with a particular one of the depth planes 240, the particular depth plane having an associated focal length so that when the eye is in the accommodation state of that depth plane, objects or portions of objects in the particular depth plane are in focus. In some embodiments, a three-dimensional image can be simulated by providing a different presentation of an image to each of the eyes 210, 220, and a three-dimensional image can also be simulated by providing a different presentation of an image corresponding to each of the depth planes. Although shown as separate for clarity of illustration, it should be understood that, for example, the fields of view of the eyes 210, 220 can overlap as the distance along the z-axis increases. Additionally, while shown as flat for ease of illustration, it will be appreciated that the profile of the depth plane may be curved in physical space, for example to bring all features in the depth plane into focus with the eye in a particular state of accommodation.

[0055] The distance between an object and eye 210 or 220 may also change the amount of divergence of light from the object, as seen by the eye. Figures 5A-5C The relationship between distance and light divergence is shown. The distance between the object and the eye 210 is represented by the order of decreasing distance R1, R2 and R3. Figures 5A-5C As shown, as the distance to the object decreases, the light rays become more divergent. As the distance increases, the light rays become more collimated. In other words, it can be said that the light field generated by a point (object or part of an object) has a spherical wavefront curvature that is a function of how far the point is from the user's eye. As the distance between the object and the eye 210 decreases, the curvature increases. Therefore, at different depth planes, the divergence of the light rays is also different, and the divergence increases as the distance between the depth plane and the viewer's eye 210 decreases. Although for the sake of clarity in the illustration, Figures 5A-5C Only a single eye 210 is shown in and other figures herein, but it should be understood that the discussion regarding eye 210 may apply to both eyes 210 and 220 of a viewer.

[0056] Without being limited by theory, it is believed that the human eye can generally interpret a limited number of depth planes to provide depth perception. Therefore, by providing the eye with different representations of images corresponding to each of these limited number of depth planes, a highly convincing simulation of perceived depth can be achieved. The different representations can be focused separately by the viewer's eyes, thereby facilitating providing depth cues to the user based on adaptation of the eye required to focus on different image features of a scene located at different depth planes and / or based on observing different image features located at different depth planes that are out of focus.

[0057] Figure 6 An example of a waveguide stack for outputting image information to a user is shown. The display system 250 includes a waveguide stack or stacked waveguide assembly 260 that can be used to provide a three-dimensional perception to the eye / brain using multiple waveguides 270, 280, 290, 300, 310. In some embodiments, the display system 250 is Figure 2 A system 60, wherein Figure 6 Some parts of the system 60 are schematically shown in more detail. For example, the waveguide assembly 260 may be Figure 2 It should be understood that in some embodiments, the display system 250 can be considered a light field display. In addition, the waveguide assembly 260 can also be referred to as an eyepiece.

[0058] Continue to refer Figure 6, the waveguide assembly 260 may further include a plurality of features 320, 330, 340, 350 located between the waveguides. In some embodiments, the features 320, 330, 340, 350 may be one or more lenses. The waveguides 270, 280, 290, 300, 310 and / or the plurality of lenses 320, 330, 340, 350 may be configured to send image information to the eye at various levels of wavefront curvature or light divergence. Each waveguide level may be associated with a particular depth plane and may be configured to output image information corresponding to the depth plane. The image injection device 360, 370, 380, 390, 400 may serve as a light source for the waveguide and may be used to inject image information into the waveguides 270, 280, 290, 300, 310, each of which may be configured to distribute incident light through each corresponding waveguide for output to the eye 210, as described herein. Light exits the output surfaces 410, 420, 430, 440, 450 of the image injection devices 360, 370, 380, 390, 400 and is injected into the respective input surfaces 460, 470, 480, 490, 500 of the waveguides 270, 280, 290, 300, 310. In some embodiments, each of the input surfaces 460, 470, 480, 490, 500 may be an edge of the respective waveguide, or may be a portion of a major surface of the respective waveguide (i.e., one of the waveguide surfaces that directly faces the world 510 or the viewer's eye 210). In some embodiments, a single beam (e.g., a collimated beam) may be injected into each waveguide so as to output an entire field of view of a cloned collimated beam directed toward the eye 210 at a particular angle (and divergence) corresponding to the depth plane associated with the particular waveguide. In some embodiments, a single one of image injection devices 360 , 370 , 380 , 390 , 400 may be associated with multiple (e.g., three) of waveguides 182 , 184 , 186 , 188 , 190 and inject light into multiple (e.g., three) of waveguides 270 , 280 , 290 , 300 , 310 .

[0059] In some embodiments, the image injection devices 360, 370, 380, 390, 400 are separate displays, each of which generates image information for injection into a respective waveguide 270, 280, 290, 300, 310. In some other embodiments, the image injection devices 360, 370, 380, 390, 400 are outputs of a single multiplexed display, which may pipe image information to each of the image injection devices 360, 370, 380, 390, 400, for example, via one or more light conduits (such as fiber optic cables). It will be appreciated that the image information provided by the image injection devices 360, 370, 380, 390, 400 may include light of different wavelengths or colors (e.g., different component colors as discussed herein).

[0060] In some embodiments, light injected into the waveguides 270, 280, 290, 300, 310 is provided by a light projector system 520 including a light module 540, which may include a light emitter, such as a light emitting diode (LED). Light from the light module 540 may be directed to a light modulator 530 and modified by the light modulator 530 (e.g., a spatial light modulator) via a beam splitter 550. The light modulator 540 may be configured to change the perceived intensity of the light injected into the waveguides 270, 280, 290, 300, 310. Examples of spatial light modulators include liquid crystal displays (LCDs), including liquid crystal on silicon (LCOS) displays. It should be understood that the image injection devices 360, 370, 380, 390, 400 are shown schematically and that in some embodiments, these image injection devices may represent different optical paths and positions in a common projection system that is configured to output light into associated waveguides in waveguides 270, 280, 290, 300, 310.

[0061] In some embodiments, the display system 250 may be a scanning fiber display that includes one or more scanning fibers configured to project light into one or more waveguides 270, 280, 290, 300, 310 and ultimately into the viewer's eye 210 in various patterns (e.g., raster scan, spiral scan, Lissajous pattern, etc.). In some embodiments, the image injection devices 360, 370, 380, 390, 400 shown may schematically represent a single scanning fiber or a bundle of scanning fibers configured to inject light into one or more waveguides 270, 280, 290, 300, 310. In some other embodiments, the image injection device 360, 370, 380, 390, 400 shown may schematically represent a plurality of scanning optical fibers or a plurality of scanning optical fibers, each of which is configured to inject light into an associated one of the waveguides 270, 280, 290, 300, 310. It should be understood that one or more optical fibers may be configured to transmit light from the optical module 540 to the one or more waveguides 270, 280, 290, 300, 310. It should be understood that one or more intermediate optical structures may be provided between the one or more scanning optical fibers and the one or more waveguides 270, 280, 290, 300, 310 to, for example, redirect light emitted from the scanning optical fibers into the one or more waveguides 270, 280, 290, 300, 310.

[0062] Controller 560 controls the operation of one or more of stacked waveguide assemblies 260, including the operation of image injection devices 360, 370, 380, 390, 400, light source 540, and light modulator 530. In some embodiments, controller 560 is part of local data processing module 140. Controller 560 includes programming (e.g., instructions in a non-transitory medium) that regulates the timing and provision of image information to waveguides 270, 280, 290, 300, 310, for example, according to any of the various schemes disclosed herein. In some embodiments, controller 560 can be a single integral device, or a distributed system connected by wired or wireless communication channels. In some embodiments, controller 560 can be a processing module 140 or 150 ( Figure 2 ) part.

[0063] Continue to refer Figure 6, the waveguides 270, 280, 290, 300, 310 can be configured to propagate light within each respective waveguide by total internal reflection (TIR). The waveguides 270, 280, 290, 300, 310 can each be planar or have other shapes (e.g., curved) having a top major surface and a bottom major surface and an edge extending between these top major surfaces and the bottom major surface. In the illustrated configuration, the waveguides 270, 280, 290, 300, 310 can each include an outcoupling optical element 570, 580, 590, 600, 610, which are configured to extract light outside the waveguide by redirecting the light propagating within each respective waveguide outside the waveguide to output image information to the eye 210. The extracted light can also be referred to as outcoupled light, and the outcoupling optical element light can also be referred to as a light extraction optical element. The extracted light beam can be output by the waveguide at a position where the light propagating in the waveguide impinges on the light extraction optical element. The outcoupling optical element 570, 580, 590, 600, 610 may, for example, include a grating of diffractive optical features, as discussed further herein. Although illustrated as being disposed at the bottom major surface of the waveguide 270, 280, 290, 300, 310 for ease of description and clarity of drawing, in some embodiments, the outcoupling optical element 570, 580, 590, 600, 610 may be disposed at the top and / or bottom major surface, and / or may be disposed directly in the volume of the waveguide 270, 280, 290, 300, 310, as discussed further herein. In some embodiments, the outcoupling optical element 570, 580, 590, 600, 610 may be formed in a material layer attached to a transparent substrate to form the waveguide 270, 280, 290, 300, 310. In some other embodiments, the waveguides 270, 280, 290, 300, 310 may be a single piece of material, and the outcoupling optical elements 570, 580, 590, 600, 610 may be formed on a surface of and / or within the interior of the piece of material.

[0064] Continue to refer Figure 6As discussed herein, each waveguide 270, 280, 290, 300, 310 is configured to output light to form an image corresponding to a particular depth plane. For example, the waveguide 270 closest to the eye can be configured to deliver collimated light (which is injected into such waveguide 270) to the eye 210. The collimated light can represent an optical infinity focal plane. The next up-going waveguide 280 can be configured to send the collimated light through a first lens 350 (e.g., a negative lens) before it can reach the eye 210; such a first lens 350 can be configured to produce a slightly convex wavefront curvature so that the eye / brain interprets the light from the next up-going waveguide 280 as coming from a first focal plane that is closer inward from optical infinity toward the eye 210. Similarly, the third upstream waveguide 290 causes its output light to pass through the first lens 350 and the second lens 340 before reaching the eye 210; the combined optical power of the first lens 350 and the second lens 340 can be configured to produce another incremental wavefront curvature so that the eye / brain interprets the light from the third waveguide 290 as coming from a second focal plane that is closer inward toward the person from optical infinity than the light from the next upstream waveguide 280.

[0065] The other waveguide layers 300, 310 and lenses 330, 320 are similarly configured, with the highest waveguide 310 in the stack sending its output through all the lenses between it and the eye for an aggregate focal power representing the focal plane closest to the person. To compensate for the stack of lenses 320, 330, 340, 350 when viewing / interpreting light from the world 510 on the other side of the stacked waveguide assembly 260, a compensating lens layer 620 can be provided at the top of the stack to compensate for the aggregate focal power of the underlying lens stack 320, 330, 340, 350. This configuration provides as many perceived focal planes as there are available waveguide / lens pairings. The out-coupling optics of the waveguide and the focusing aspects of the lens can be static (i.e., not dynamic or electro-active). In some optional embodiments, either or both can be dynamic using electro-active features.

[0066] In some embodiments, two or more of the waveguides 270, 280, 290, 300, 310 may have the same associated depth plane. For example, multiple waveguides 270, 280, 290, 300, 310 may be configured to output images set to the same depth plane, or multiple subsets of waveguides 270, 280, 290, 300, 310 may be configured to output images set to the same multiple depth planes, one set per depth plane. This may provide advantages for forming tiled images to provide an extended field of view at those depth planes.

[0067] Continue to refer Figure 6 , the out-coupling optical elements 570, 580, 590, 600, 610 can be configured to redirect light out of their respective waveguides and output the light with an appropriate amount of divergence or collimation for a particular depth plane associated with the waveguide. As a result, waveguides with different associated depth planes can have different configurations of out-coupling optical elements 570, 580, 590, 600, 610 that output light with different amounts of divergence depending on the associated depth plane. In some embodiments, the light extraction optical elements 570, 580, 590, 600, 610 can be volume or surface features that can be configured to output light at a specific angle. For example, the light extraction optical elements 570, 580, 590, 600, 610 can be volume holograms, surface holograms, and / or diffraction gratings. In some embodiments, features 320, 330, 340, 350 may not be lenses; instead, they may simply be spacers (eg, cladding and / or structures used to form air gaps).

[0068] In some embodiments, the outcoupling optical elements 570, 580, 590, 600, 610 are diffractive features that form a diffraction pattern, or "diffractive optical elements" (also referred to herein as "DOEs"). Preferably, the DOE has a sufficiently low diffraction efficiency that only a portion of the light beam is deflected toward the eye 210 through each intersection of the DOE, while the remainder continues to move through the waveguide via TIR. The light carrying the image information is thus split into a plurality of related exit beams that exit the waveguide at a plurality of locations, and the result is a fairly uniform pattern of exit emissions toward the eye 210 for that particular collimated beam that bounces within the waveguide.

[0069] In some embodiments, one or more DOEs may be switchable between an "on" state in which they actively diffract and an "off" state in which they do not significantly diffract. For example, a switchable DOE may include a polymer dispersed liquid crystal layer in which droplets contain a diffractive pattern in a host medium, and the refractive index of the droplets may be switched to substantially match that of the host material (in which case the pattern does not significantly diffract incident light), or the droplets may be switched to a refractive index that does not match that of the host medium (in which case the pattern actively diffracts incident light).

[0070] In some embodiments, a camera assembly 630 (e.g., a digital camera, including visible light and infrared light cameras) may be provided to capture images of the eye 210 and / or tissue surrounding the eye 210, for example, to detect user input and / or monitor a physiological state of the user. As used herein, a camera may be any image capture device. In some embodiments, the camera assembly 630 may include an image capture device and a light source to project light (e.g., infrared light) to the eye, which may then be reflected by the eye and detected by the image capture device. In some embodiments, the camera assembly 630 may be attached to the frame 80 ( Figure 2 ) and can be in electrical communication with processing modules 140 and / or 150, which can process image information from camera assembly 630. In some embodiments, one camera assembly 630 can be used for each eye to monitor each eye separately.

[0071] Reference now Figure 7 , which shows an example of an output beam output by a waveguide. One waveguide is shown, but it should be understood that the waveguide assembly 260 ( Figure 6 ) can function similarly, where the waveguide assembly 260 includes multiple waveguides. Light 640 is injected into the waveguide 270 at the input surface 460 of the waveguide 270 and propagates within the waveguide 270 by TIR. At the point where the light 640 impinges on the DOE 570, a portion of the light exits the waveguide as an exit beam 650. The exit beam 650 is shown as being substantially parallel, but as discussed herein, depending on the depth plane associated with the waveguide 270, the exit beam 650 can also be redirected at an angle (e.g., to form a diverging exit beam) to propagate to the eye 210. It should be understood that a substantially parallel exit beam can be indicative of a waveguide having an out-coupling optical element that couples light out to form an image that appears to be disposed at a depth plane at a greater distance (e.g., optical infinity) from the eye 210. Other waveguides or other groups of outcoupling optics may output a more divergent exit beam pattern, which would require the eye 210 to adapt to a closer distance to focus it on the retina and would be interpreted by the brain as light coming from a distance closer to the eye 210 than optical infinity.

[0072] In some embodiments, a full color image may be formed at each depth plane by superimposing images of each component color (eg, three or more component colors). Figure 8An example of a stacked waveguide assembly is shown, where each depth plane includes an image formed using multiple different component colors. The illustrated embodiment shows depth planes 240a-240f, but more or fewer depths may also be considered. Each depth plane may have three or more component color images associated with it, including: a first image G of a first color; a second image R of a second color; and a third image B of a third color. For the optical power (dpt) after the letters G, R, and B, different numbers are used in the figure to represent different depth planes. As an example, the number after each of these letters represents the optical power (1 / m), or the inverse distance of the depth plane from the viewer, and each box in the figure represents a separate component color image. In some embodiments, in order to account for differences in the eye's focus on light of different wavelengths, the precise placement of the depth planes of different component colors can be changed. For example, different component color images of a given depth plane can be placed on depth planes corresponding to different distances from the user. Such an arrangement can increase visual acuity and user comfort and / or can reduce chromatic aberration.

[0073] In some embodiments, light of each component color may be output by a single dedicated waveguide, and therefore, each depth plane may have multiple waveguides associated with it. In such embodiments, each box in the figure including the letters G, R, or B may be understood to represent a separate waveguide, and each depth plane may provide three waveguides, wherein each depth plane provides three component color images. Although for ease of description, the waveguides associated with each depth plane are shown adjacent to each other in the figure, it should be understood that in a physical device, the waveguides may all be arranged into a stack with one waveguide per layer. In some other embodiments, multiple component colors may be output by the same waveguide, such that, for example, each depth plane may provide only a single waveguide.

[0074] Continue to refer Figure 8 In some embodiments, G is green, R is red, and B is blue. In some other embodiments, other colors associated with other wavelengths of light (including magenta and cyan) may be used in addition to red, green, or blue, or one or more of red, green, or blue may be replaced.

[0075] It should be understood that references throughout this disclosure to light of a given color will be understood to include light of one or more wavelengths within a range of wavelengths that a viewer perceives as light of the given color. For example, red light may include light of one or more wavelengths within a range of about 620-780 nm, green light may include light of one or more wavelengths within a range of about 492-577 nm, and blue light may include light of one or more wavelengths within a range of about 435-493 nm.

[0076] In some embodiments, the light source 540 ( Figure 6 ) can be configured to emit light at one or more wavelengths (e.g., infrared and / or ultraviolet wavelengths) outside the visual perception range of a viewer. Additionally, the incoupling, outcoupling, and other light redirection structures of the waveguide of display 250 can be configured to direct this light out of the display and toward the user's eye 210, e.g., for imaging and / or user stimulation applications.

[0077] Reference now Fig. 9A In some embodiments, it may be desirable to redirect light impinging on a waveguide to couple the light into the waveguide. Incoupling optics may be used to redirect the light and couple the light into its corresponding waveguide. Fig. 9A A cross-sectional side view of an example of a plurality of stacked waveguides or stacked waveguide groups 660 is shown, each stacked waveguide including an incoupling optical element. Each waveguide can be configured to output light of one or more different wavelengths, or light of one or more different wavelength ranges. It should be understood that stack 660 can correspond to stack 260 ( Figure 6 ), and the stacked waveguide 660 can correspond to a portion of multiple waveguides 270, 280, 290, 300, 310, except that one or more lights from the image injection devices 360, 370, 380, 390, 400 are injected into the waveguide from a location where the light needs to be redirected for coupling.

[0078] The stacked waveguide group 660 shown includes waveguides 670, 680, and 690. Each waveguide includes an associated coupling optical element (which may also be referred to as a light input region on the waveguide), for example, a coupling optical element 700 disposed on a major surface (e.g., upper major surface) of the waveguide 670, a coupling optical element 710 disposed on a major surface (e.g., upper major surface) of the waveguide 680, and a coupling optical element 720 disposed on a major surface (e.g., upper major surface) of the waveguide 690. In some embodiments, one or more of the coupling optical elements 700, 710, 720 may be disposed on the bottom major surface of the corresponding waveguide 670, 680, 690 (particularly in the case where one or more coupling optical elements are reflective, deflecting optical elements). As shown, the coupling optical elements 700, 710, 720 may be disposed on the upper major surface (or the top of the next lower waveguide) of their corresponding waveguides 670, 680, 690, particularly in the case where those coupling optical elements are transmissive, deflecting optical elements. In some embodiments, the coupling-in optical elements 700, 710, 720 can be disposed in the body of the respective waveguides 670, 680, 690. In some embodiments, as discussed herein, the coupling-in optical elements 700, 710, 720 are wavelength selective such that they selectively redirect light of one or more wavelengths while transmitting light of other wavelengths. Although shown on one side or corner of their respective waveguides 670, 680, 690, it should be understood that in some embodiments, the coupling-in optical elements 700, 710, 720 can be disposed in other areas of their respective waveguides 670, 680, 690.

[0079] As shown, the coupling-in optical elements 700, 710, 720 can be laterally offset from each other. In some embodiments, each coupling-in optical element can be offset so that it receives light without the light passing through another coupling-in optical element. For example, each coupling-in optical element 700, 710, 720 can be configured to be offset from, for example, Figure 6 The different image injection devices 360 , 370 , 380 , 390 , and 400 shown receive light and may be separated (eg, laterally spaced apart) from the other incoupling optical elements 700 , 710 , 720 such that it receives substantially no light from the other incoupling optical elements 700 , 710 , 720 .

[0080] Each waveguide further includes an associated light distribution element, such as a light distribution element 730 disposed on a major surface (e.g., top major surface) of the waveguide 670, a light distribution element 740 disposed on a major surface (e.g., top major surface) of the waveguide 680, and a light distribution element 750 disposed on a major surface (e.g., top major surface) of the waveguide 690. In some other embodiments, the light distribution elements 730, 740, 750 may be disposed on the bottom major surface of the associated waveguides 670, 680, 690, respectively. In some other embodiments, the light distribution elements 730, 740, 750 may be disposed on the top and bottom major surfaces of the associated waveguides 670, 680, 690, respectively; or the light distribution elements 730, 740, 750 may be disposed on different major surfaces of the top and bottom major surfaces of different associated waveguides 670, 680, 690, respectively.

[0081] The waveguides 670, 680, 690 may be spaced apart and separated by, for example, layers of gas, liquid, and / or solid material. For example, as shown, layer 760a may separate waveguides 670 and 680; and layer 760b may separate waveguides 680 and 690. In some embodiments, layers 760a and 760b are formed of a low refractive index material (i.e., a material having a lower refractive index than the material forming the immediately adjacent waveguide in waveguides 670, 680, 690). Preferably, the refractive index of the material forming layers 760a, 760b is 0.05 or more, or 0.10 or less, less than the refractive index of the material forming waveguides 670, 680, 690. Advantageously, the low refractive index layers 760a, 760b may serve as a cladding layer that promotes total internal reflection (TIR) ​​of light passing through waveguides 670, 680, 690 (e.g., TIR between the top and bottom major surfaces of each waveguide). In some embodiments, the layers 760a, 760b are formed of air. Although not shown, it is understood that the top and bottom of the illustrated waveguide set 660 may include an adjacent cladding layer.

[0082] Preferably, for ease of manufacturing and other considerations, the materials forming waveguides 670, 680, 690 are similar or the same, and the materials forming layers 760a, 760b are similar or the same. In some embodiments, the materials forming waveguides 670, 680, 690 can be different between one or more waveguides, and / or the materials forming layers 760a, 760b can be different, while still maintaining the various refractive index relationships described above.

[0083] Continue to refer Fig. 9A, light rays 770, 780, 790 are incident on the waveguide group 660. It should be understood that light rays 770, 780, 790 may be injected through one or more image injection devices 360, 370, 380, 390, 400 ( Figure 6 ) are injected into waveguides 670, 680, and 690.

[0084] In some embodiments, the light rays 770, 780, 790 have different properties, such as different wavelengths or different wavelength ranges, which may correspond to different colors. The incoupling optical elements 700, 122, 720 each deflect incident light so that the light propagates through a corresponding one of the waveguides 670, 680, 690 by TIR. In some embodiments, the incoupling optical elements 700, 710, 720 each selectively deflect one or more specific wavelengths of light while transmitting other wavelengths to the underlying waveguide and associated incoupling optical element.

[0085] For example, the coupling-in optical element 700 can be configured to deflect light 770 having a first wavelength or wavelength range while transmitting light 780 and 790 having different second and third wavelengths or wavelength ranges, respectively. The transmitted light 780 impinges on and is deflected by the coupling-in optical element 710, which is configured to deflect light having the second wavelength or wavelength range. The light 790 is deflected by the coupling-in optical element 720, which is configured to selectively deflect light having a third wavelength or wavelength range.

[0086] Continue to refer Fig. 9A , the deflected light rays 770, 780, 790 are deflected so that they propagate through the corresponding waveguides 670, 680, 690; that is, the coupling-in optical element 700, 710, 720 of each waveguide deflects light into the corresponding waveguide 670, 680, 690 to couple the light into the corresponding waveguide. The light rays 770, 780, 790 are deflected at an angle that causes the light to propagate through the corresponding waveguide 670, 680, 690 via TIR. The light rays 770, 780, 790 propagate through the corresponding waveguide 670, 680, 690 via TIR until impinging on the corresponding light distribution element 730, 740, 750 of the waveguide.

[0087] Reference now Fig. 9B , showing Fig. 9A600, 680, 690. As described above, the in-coupling light rays 770, 780, 790 are deflected by the in-coupling optical elements 700, 710, 720, respectively, and then propagate through TIR in the waveguides 670, 680, 690, respectively. Then, the light rays 770, 780, 790 are irradiated onto the light distribution elements 730, 740, 750, respectively. The light distribution elements 730, 740, 750 deflect the light rays 770, 780, 790 so that they propagate toward the out-coupling optical elements 800, 810, 820, respectively.

[0088] In some embodiments, the light distribution elements 730, 740, 750 are orthogonal pupil expanders (OPEs). In some embodiments, the OPEs deflect or distribute light to the outcoupling optical elements 800, 810, 820, and in some embodiments, the OPEs also increase the beam or spot size of the light as it propagates to the outcoupling optical elements. In some embodiments, the light distribution elements 730, 740, 750 may be omitted, and the incoupling optical elements 700, 710, 720 may be configured to deflect light directly to the outcoupling optical elements 800, 810, 820. For example, referring to Fig. 9A , the light distribution elements 730, 740, 750 can be replaced by outcoupling optical elements 800, 810, 820, respectively. In some embodiments, the outcoupling optical elements 800, 810, 820 are exit pupils (EP) or exit pupil expanders (EPE), which guide the light in the viewer's eye 210 ( Figure 7 ). It should be understood that the OPE can be configured to increase the size of the eyebox in at least one axis, and the EPE can increase the eyebox in an axis that is, for example, orthogonal to the axis of the OPE. For example, each OPE can be configured to redirect a portion of the light impinging on the OPE to the EPE of the same waveguide, while allowing the remainder of the light to continue to propagate along the waveguide. Upon impinging on the OPE again, another portion of the remaining light is redirected to the EPE, and the remainder of the remaining light continues to propagate further along the waveguide, and so on. Similarly, upon impinging on the EPE, a portion of the impinged light is directed out of the waveguide toward the user, and the remainder of the light continues to propagate through the waveguide until it impinges on the EP again, at which time another portion of the impinged light is directed out of the waveguide, and so on. Therefore, each time a portion of the light is redirected by an OPE or EPE, the coupled-in light of a single beam of light can be "copied", thereby forming a field of cloned beams, such as Figure 6 As shown. In some embodiments, the OPE and / or EPE can be configured to modify the size of the light beam.

[0089] Therefore, reference Fig. 9A and 9BIn some embodiments, the waveguide group 660 includes waveguides 670, 680, 690; coupling-in optics 700, 710, 720; light distribution elements (e.g., OPEs) 730, 740, 750; and outcoupling optics (e.g., EPs) 800, 810, 820 for each component color. The waveguides 670, 680, 690 can be stacked with air gaps / cladding layers between each. The coupling-in optics 700, 710, 720 redirect or deflect the incident light (using different coupling-in optics that receive light of different wavelengths) into their waveguides. The light then propagates at an angle that will cause TIR within the corresponding waveguide 670, 680, 690. In the example shown, light ray 770 (e.g., blue light) is deflected by first coupling-in optical element 700 and then proceeds to jump to the waveguide, interacting with light distribution element (e.g., OPE) 730 in the manner previously described, and then interacting with outcoupling optical element (e.g., EPs) 800. Light rays 780 and 790 (e.g., green light and red light, respectively) will pass through waveguide 670, where light ray 780 impinges on coupling-in optical element 710 and is deflected by coupling-in optical element 710. Light ray 780 then jumps to waveguide 680 via TIR, proceeds to light distribution element (e.g., OPE) 740 reaching it and then to outcoupling optical element (e.g., EP) 810. Finally, light ray 790 (e.g., red light) passes through waveguide 690 to impinge on light coupling-in optical element 720 of waveguide 690. The light incoupling optical element 720 deflects the light 790 so that it propagates through TIR to the light distribution element (e.g., OPE) 750, and then propagates through TIR to the outcoupling optical element (e.g., EP) 820. The outcoupling optical element 820 then finally couples the light 790 out to the viewer, who also receives outcoupled light from the other waveguides 670, 680.

[0090] Fig. 9C Shows Fig. 9A and 9B A top plan view of an example of multiple stacked waveguides. As shown, the waveguides 670, 680, 690 and the light distribution elements 730, 740, 750 associated with each waveguide and the associated coupling-in optical elements 800, 810, 820 can be vertically aligned. However, as discussed herein, the coupling-in optical elements 700, 710, 720 are not vertically aligned; instead, the coupling-in optical elements are preferably non-overlapping (e.g., laterally spaced apart as seen in the top view). As further discussed herein, this non-overlapping spatial arrangement facilitates one-to-one injection of light from different sources into different waveguides, thereby allowing a particular light source to be uniquely coupled to a particular waveguide. In some embodiments, arrangements including non-overlapping, spatially separated coupling-in optical elements can be referred to as shifted pupil systems, and the coupling-in optical elements within these arrangements can correspond to sub-pupils.

[0091] Eye imaging and environmental imaging

[0092] As described above, a head-mounted display can be used to provide image content to a user that is integrated, combined, and / or superimposed on a view of the world in front of the wearer. Such a head-mounted display system can be configured to project light into the user's eyes to form augmented reality image content, and to transmit light from the environment in front of the user to the user. The head-mounted display system may include one or more cameras for imaging the environment and / or the user's eyes. An outward-facing camera can be used to directly image the environment, for example, to determine where to place the augmented reality image content relative to an object in the environment. For example, imaging the environment can provide the location of a table so that the head-mounted display can render an image of a person standing near the table rather than standing on or in the table. An inward-facing camera can be used to directly shape the eyes, for example, for eye tracking. Disclosed herein are examples of head-mounted display systems and / or imaging systems that can be configured to also image the eyes and / or the environment. In some designs, the system does not require inward-facing and / or outward-facing cameras to directly image the eyes and / or the environment, respectively. Such a system may employ one or more cameras configured to receive light from the eye / environment via the eyepiece, such as one or more waveguides in the eyepiece in optical communication with the one or more cameras. With the light collected by the one or more waveguides, the one or more cameras may generate an image of the eye or an image of the environment in front of the user. Using waveguides to collect light for imaging the eye and / or environment may potentially reduce the form factor of the head mounted display, making the head mounted display potentially more compact and / or aesthetically desirable.

[0093] Fig.10 An example imaging system 900 configured to image an eye is shown, integrated with an eyepiece 950 that can be used in a head-mounted display. The eyepiece 950, which can be disposed in front of the user's eye 210, can be used to inject image content into the eye and to image the eye. Fig.10 An eyepiece 950 is shown positioned in front of one eye 210. Figure 2 The various head mounted display systems shown may include a pair of eyepieces 950 and associated components disposed in front of respective left and right eyes 210. Fig.10 A single waveguide 940 is shown in FIG. 9 , but waveguide 940 may include one, two, three, four, six, seven, eight, or more waveguides (eg, one or more waveguide stacks).

[0094] The imaging system 900 may include a light source or illumination source 960 that illuminates the eye to facilitate image capture, an eyepiece 950 including a waveguide 940 configured to propagate light therein, and / or an imaging device 920, such as a camera for image capture. Also shown is an image projector 930 for producing an image that can be injected into the eye via the eyepiece 950. The eyepiece 950 may include one or more waveguides 940 configured to transmit light from the illumination source 960 and / or the image projector 930 to the eye, and to transmit light from the eye to the camera 920. The eyepiece 950 may further include one or more coupling optical elements 944 for coupling light out of the waveguide 940 and into the eye for illuminating the eye and for image injection, and / or coupling light out of the eye and into the waveguide for image capture. The eyepiece 950 may further include one or more coupling-in optical elements 942 for coupling light from the illumination source 960 and / or the image projector 930 into the waveguide 940 and one or more coupling-out optical elements 952 for coupling light out of the waveguide to the camera 920.

[0095] The eyepiece 950 may be disposed on a frame that is wearable on the head. The eyepiece 950 may be disposed in front of the eye 210. The eyepiece 950 may have an inner or nasal side that is closer to the wearer's nose and an opposite outer or temporal side that is closer to the temple and away from the wearer's nose. Fig.10 , coupling optical element 944 is medially or nasally relative to coupling-in optical element 942 and coupling-out optical element 952 (which is lateral or temporal to coupling optical element 944). Illumination source 960 is also more medially or nasally relative to image projector 930 (or image projector is lateral or temporal to illumination source 930). However, the relative positions may be different. For example, in some designs, illumination source 960 may be more lateral or temporal to image projector 930.

[0096] The waveguide 940 may include a sheet or layer having two major surfaces (a front surface and a rear surface) that have the largest surface areas and are disposed opposite each other. When the user wears the head mounted display, the front facing surface may be away from the user's eyes 210 (closer to the environment in front of the wearer), and the rear facing surface may be closer to the user's eyes (and away from the environment in front of the wearer). The waveguide 940 may include a transparent material (e.g., glass, plastic) having a refractive index greater than 1.0 so that light may be guided therein by total internal reflection between the major surfaces. For one or more embodiments described herein, elements with the same number may have the same function.

[0097] Coupling optics 944 for coupling light from the waveguide 940 to the eye 210 and / or from the waveguide to the eye may be disposed on or in the waveguide 940. Fig.10 As shown, the coupling optical element 944 can be disposed in an optical path between the user's eye 210 and the waveguide 940, so that light coupled from the waveguide 940 via the coupling optical element 944 can be incident on the user's eye 210 (e.g., to illuminate the eye and / or for image injection). The coupling optical element 944 may include a plurality of turning features configured to turn light guided within the waveguide out of the waveguide, or to turn light incident on the coupling optical element 944 into the waveguide at a certain angle to be guided therein by total internal reflection. The coupling optical element 944 and the turning features may be physically coupled to the waveguide 940. For example, the coupling optical element 944 may include a holographic or diffractive optical element (e.g., a surface relief grating) patterned (e.g., etched) in or on the waveguide 940. The coupling optical element 944 may include a layer disposed on the waveguide 940, or may be formed in the waveguide 940. For example, a volume hologram or other diffractive optical element may be formed by changing the refractive index of a material comprising the waveguide or a layer disposed thereon.Thus, the coupling optical element 944 may be disposed in the volume of the waveguide 940 or as a layer thereon.

[0098] Depending on the design, the coupling optical element 944 can be transmissive or reflective, and can operate in a transmissive or reflective manner. For example, the coupling optical element 944 can include a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates in a transmissive or reflective manner, respectively, thereby, for example, redirecting light transmitted through it or reflected from it. The coupling optical element 944 can include a polarization optical element, such as a polarization selective steering element (e.g., a polarizer). The polarization selective steering element can include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and can include a liquid crystal structure, such as a liquid crystal polarization grating. The coupling optical element 944 can be configured to guide light from the image projector 930 and / or the light source 960 guided within the waveguide 940 by total internal reflection (TIR) ​​to the user's eye 210 at an angle less than (e.g., more normal to) the critical angle so as to be emitted from the waveguide to the eye. Additionally or alternatively, the coupling optical element 944 can be configured to couple light from the eye 210 into the waveguide 940 at an angle greater than (e.g., less normal to) the critical angle so as to be guided therein to the camera 920 by total internal reflection.

[0099] like Fig.10As shown, an incoupling optical element 942 for coupling light from an illumination source 960 and / or an image projector 930 into a waveguide 940 may be disposed on or in the waveguide 940. The incoupling optical element 942 may be disposed in an optical path between the light source 960 and the waveguide 940 such that light coupled from the light source 960 via the incoupling optical element 942 is guided within the waveguide 940. The incoupling optical element 942 may include, for example, a plurality of turning features configured to turn light incident thereon at an angle into the waveguide to be guided in the waveguide by total internal reflection. The incoupling optical element 942 may include a liquid crystal structure, such as a liquid crystal polarization grating. Additionally or alternatively, the incoupling optical element 942 may include a blazed grating. The incoupling optical element 942 may include a layer disposed on the waveguide 940, or may be formed on or in the waveguide 940 (e.g., patterned), or may be otherwise fabricated therein. For example, a surface hologram or diffractive optical element (e.g., a surface relief grating) can be manufactured by patterning (e.g., etching) the surface of a waveguide or a layer thereon. A volume hologram or diffractive optical element can also be formed by changing the refractive index of the material comprising the waveguide or a layer thereon. Thus, the coupling-in optical element 942 can be disposed in the volume of the waveguide 940 or in a layer thereon. Depending on the design, the coupling-in optical element 942 can be transmissive or reflective and can operate in a transmissive or reflective manner. For example, the coupling-in optical element 942 can include a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates in a transmissive or reflective manner, respectively, for example, thereby deflecting light transmitted through it or reflected from it.

[0100] The coupling-in optical element 942 may include a reflective optical element (e.g., a mirror). For example, the coupling-in optical element 942 may include an off-axis reflector. Additionally or alternatively, the coupling-in optical element 942 and / or the coupling optical element 944 may include a polarization optical element, such as a polarization selective steering element (e.g., a polarizer). The polarization selective steering element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may include a liquid crystal structure, such as a liquid crystal polarization grating. For example, one or both of the coupling-in optical element 942 and / or the coupling optical element 944 may include a liquid crystal polarization grating (LCPG). The LCPG may potentially provide efficient diffraction over a wide wavelength. Thus, the LCPG may be used for the coupling-in optical element 942 and / or the coupling optical element 944. The LCPG may be polarization dependent. The LCPG or other type of liquid crystal grating, diffractive optical element, or optical element may include a pattern or arrangement of liquid crystal molecules configured to provide one or more functions, such as diverting light into or out of a waveguide. Thus, the coupling-in optical element 942 and / or the coupling optical element 944 may include a polarization grating. Additionally or alternatively, the coupling-in optical element 942 and / or the coupling optical element 944 may include a liquid crystal, and thus in some implementations, one or both may be a liquid crystal grating or a liquid crystal diffractive optical element. Additionally or alternatively, one or both of the coupling-in optical element 942 and / or the coupling optical element 944 may include a blazed grating. In some designs, the coupling-in optical element 942 includes a liquid crystal reflector, such as a cholesteric liquid crystal reflective lens (e.g., a reflective liquid crystal diffractive lens, a Bragg reflection structure, a reflective liquid crystal diffraction grating, etc.).Some non-limiting examples of liquid crystal gratings, liquid crystal polarization gratings, and other liquid crystal optical elements are discussed in the following published applications, which are hereby incorporated by reference in their entirety for all purposes: U.S. Publication No. 2018 / 0143438, filed on November 16, 2017, entitled “MULTILAYER LIQUID CRYSTAL DIFFRACTIVEGRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES”; U.S. Publication No. 2018 / 0143485, filed on November 16, 2017, entitled “SPATIALLY VARIABLE LIQUID CRYSTAL DIRRRACTION GRATINGS”; U.S. Publication No. 2018 / 0143486, filed on November 16, 2017, entitled “WAVEGUIDELIGHT MULTIPLEXER USING CROSSED and U.S. Publication No. 2018 / 0143509, entitled “WAVEGUIDE OPTICAL MULTIPLEXER USING CROSSED GRATINGS”; U.S. Publication No. 2018 / 0239147, filed on February 22, 2018, entitled “DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR”; U.S. Publication No. 2018 / 0239177, filed on February 22, 2018, entitled “VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION”; and U.S. Publication No. 2018 / 0239189, entitled “DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID DEVICES BASED ON VISUAL IMAGE DEVICES WITH VARIABLE FOCUS REFLECTOR”, filed on December 7, 2017. CRYSTAL (Diffractive Device Based on Cholesteric Liquid Crystal)" U.S. Publication No. 2018 / 0164627. However, the designs of the coupling-in optical element 942 and / or the coupling optical element 944 are not limited to these and may include other types of optical elements, diffractive optical elements, liquid crystal optical elements, liquid crystal gratings, and liquid crystal polarization gratings. More information about examples of cholesteric liquid crystal structures such as reflectors can also be found in the section below entitled "Cholesteric Liquid Crystal Mirrors". As described above, other liquid crystal optical elements as well as other non-liquid crystal optical elements may be used.Thus, many types of coupling optical elements (e.g., coupling-in optical element 942 and / or coupling optical element 944), diffractive optical elements, gratings, polarization gratings, etc., can be used, typically using those described herein and other types of gratings, diffractive optical elements, liquid crystal elements, and optical elements. In various implementations, coupling-in optical element 942 can be configured to couple light from image projector 930 and / or light source 960 into the waveguide at an angle greater than the critical angle so as to be guided to the eye to the user's eye 210 within the waveguide 940 by total internal reflection.

[0101] The waveguide 940 may include one or more waveguides. In some implementations, one or more waveguides 940 include a stack of waveguides. For example, in some designs, different waveguides in a waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eyes. For example, a first waveguide or waveguide group may be configured to output collimated light or light with a first divergence, as if projected from a first depth, and a second waveguide or waveguide group may be configured to output divergent (uncollimated) or light at a second divergence (greater than the first divergence), as if projected from a second depth closer than the first depth. In some designs, different waveguides may be configured to output light with different associated colors. For example, a first waveguide may be configured to output red light, a second waveguide may be configured to output green light, and a third waveguide may be configured to output blue light. A fourth waveguide may be configured to output and / or input infrared light.

[0102] For example Fig.10 The out-coupling optical element 952 shown for coupling light from the waveguide 940 to the camera 920 may include, for example, a plurality of turning features configured to turn light incident thereon at an angle such that the light is not guided within the waveguide and the light is turned out of the waveguide to the camera. The out-coupling optical element 952 may be disposed within the interior of the waveguide 940, or may be patterned (e.g., etched) in or on a surface (e.g., a major surface) of the waveguide 940. For example, a surface hologram or a diffractive optical element (e.g., a surface relief grating) may be fabricated by patterning (e.g., etching) the surface of the waveguide or a layer thereon. A volume hologram or diffractive optical element may also be formed by changing the refractive index of the material comprising the waveguide or a layer disposed thereon. Depending on the design, the out-coupling optical element 952 may be transmissive or reflective, and may operate in a transmissive or reflective manner. For example, the outcoupling optical element 952 may include, for example, a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates in a transmissive or reflective manner, respectively, for example, thereby redirecting light transmitted therethrough or reflected therefrom.

[0103] The outcoupling optical element 942 may include a reflective optical element (e.g., a mirror). For example, the outcoupling optical element 952 may include an off-axis reflector. In some designs, the outcoupling optical element 952 may include a polarization optical element, such as a polarization selective steering element (e.g., a polarizer). Thus, the polarization selective steering element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may include a liquid crystal structure such as a liquid crystal polarization grating. In some implementations, for example, the outcoupling optical element 952 may include a liquid crystal polarization grating (LCPG). LCPG can potentially provide efficient diffraction over a wide wavelength. Similarly, LCPG can be used for the outcoupling optical element 952. LCPG can be polarization dependent. LCPG or other types of liquid crystal gratings may include a pattern or arrangement of liquid crystal molecules that are configured to provide one or more functions, such as transferring light into or out of a waveguide. Thus, the outcoupling optical element 952 may include a polarization grating. Additionally or alternatively, the outcoupling optical element 952 may include a liquid crystal, and thus, in some implementations, may be a liquid crystal grating or other liquid crystal optical element, such as a liquid crystal diffractive optical element. Additionally or alternatively, the outcoupling optical element 952 may include a blazed grating. In some designs, the outcoupling optical element 952 includes a liquid crystal reflector, such as a cholesteric liquid crystal reflective lens (e.g., a reflective liquid crystal diffractive lens, a Bragg reflection structure, a reflective liquid crystal diffraction grating, etc.).Some non-limiting examples of liquid crystal gratings, liquid crystal polarization gratings, and other liquid crystal optical elements are discussed in the following published applications, which are hereby incorporated by reference in their entirety for all purposes: U.S. Publication No. 2018 / 0143438, filed on November 16, 2017, entitled “MULTILAYER LIQUID CRYSTALDIFFRACTIVE GRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES”; U.S. Publication No. 2018 / 0143485, filed on November 16, 2017, entitled “SPATIALLY VARIABLE LIQUID CRYSTAL DIRRRACTION GRATINGS”; U.S. Publication No. 2018 / 0143486, filed on November 16, 2017, entitled “WAVEGUIDE LIGHT MULTIPLEXER USING CROSSED and U.S. Publication No. 2018 / 0143509, entitled “WAVEGUIDE OPTICAL MULTIPLEXER USING CROSSED GRATINGS”; U.S. Publication No. 2018 / 0239147, filed on February 22, 2018, entitled “DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR”; U.S. Publication No. 2018 / 0239177, filed on February 22, 2018, entitled “VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION”; and U.S. Publication No. 2018 / 0239177, filed on December 7, 2017, entitled “DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL (Diffraction Device Based on Cholesteric Liquid Crystal)" is disclosed in U.S. Publication No. 2018 / 0164627. However, the design of the coupling optical element 952 is not limited to these, and may include other types of optical elements, diffractive optical elements, liquid crystal optical elements, liquid crystal gratings, and liquid crystal polarization gratings. More information about examples of cholesteric liquid crystal structures such as reflectors can also be found in the section below entitled "Cholesteric Liquid Crystal Mirrors". As described above, other liquid crystal optical elements and other non-liquid crystal optical elements can be used. Therefore, many types of coupling optical elements (e.g., coupling optical element 952), diffractive optical elements, gratings, polarization gratings, etc. can be used, typically using those described herein and other types of gratings, diffractive optical elements, liquid crystal elements, or optical elements.As described above, the outcoupling optical element 952 can be configured to redirect light guided within the waveguide 940 at an angle less than the critical angle so as not to be guided within the waveguide by total internal reflection but rather to be emitted to the camera 920.

[0104] In various designs, coupling optical element 944 can be transparent in the visible spectrum so that the user can see the environment in front of the user through coupling optical element 944 and eyepiece 950. For example, if coupling optical element 942 is used to receive light from image projector 930 and / or if illumination source 960 is configured to output visible light to illuminate eye 210 with visible light, coupling optical element 942 can also redirect light in the visible spectrum. In some embodiments, for example, if illumination source 960 is configured to output infrared light to illuminate eye 210 with infrared light, coupling optical element 942 is configured to redirect infrared light. In embodiments such as Fig.10 In some designs shown, the incoupling optical element 942 can be located more medially or nasally than the outcoupling optical element 952. However, in other designs, the incoupling optical element 942 can be located more laterally or temporally than the outcoupling optical element 952. Fig.10 In certain implementations shown in FIG. 9 , the outcoupling optical element 952 may be adjacent to the incoupling optical element 942, although non-adjacent positioning is possible.

[0105] like Fig.10 As shown, the illumination source 960 can be disposed on the same side of the eyepiece 950 as the eye 210 (e.g., rearward or proximal). (The proximal end can refer to the side closest to the eye 210.) Optionally, the illumination source 960 can be disposed on the side opposite to the eye 210 (e.g., forward or distal). The illumination source 960 can be configured to guide light into at least one of the major surfaces of the waveguide 940 via the coupling optical element 942. The light source 960 can be configured to emit invisible light (e.g., infrared). The light source 960 can include one or more LEDs. The LED can include an infrared LED. The light source 960 can be configured to emit coherent light. In some designs, the light source 960 includes a laser (e.g., an infrared laser). In some designs, the light source 960 emits pulsed light. For example, the camera 920 can be configured to capture images periodically. Therefore, the illumination source 960 can be pulsed to coincide with the time period when the camera acquires an image. When the camera does not acquire an image, the intensity output from the illumination source 960 can be reduced. By concentrating the total energy of the illumination in a short time, an increased signal-to-noise ratio can be obtained without exposing the eye 210 to unsafe intensity levels. In some cases, for example, the camera 920 captures an image every 30 milliseconds, and the camera's exposure time is several milliseconds. The illumination source 960 can be configured to output pulses of similar period and duration to match the pulses of the camera 920.

[0106] In some implementations, different light sources having different wavelengths are alternately pulsed to provide illumination at different wavelengths at different times, as described below.

[0107] The coupling-in optical element 942 can, for example, be in direct optical communication with the illumination source 960 and / or the image projector 930 so as to guide light therein from the image projector 930 and / or the light source 960. For example, light emitted by the light source 960 can be incident on the coupling-in optical element 942 before optically interacting with the coupling optical element 944 and / or the coupling optical element 952.

[0108] like Figures 11A-11E As shown, light 902 projected from an image projector 930 can form an image on the retina. The image projector 930 may include a light source, a modulator, and / or projection optics. The light source for the image projector 930 may include one or more LEDs, lasers, or other light sources, and may include one or more visible light sources. The modulator may include a spatial light modulator, such as a liquid crystal spatial light modulator. Such a spatial light modulator may be configured, for example, to modulate the intensity of light at different spatial locations. The projection optics may include one or more lenses. Other types of image projectors 930 capable of projecting and / or forming images may be used. For example, the image projector 930 may include a scanning optical fiber.

[0109] The image projector 930 and the coupling optical element 942 can be in direct optical communication with each other. The image projector 930 can be aligned with the coupling optical element 942 into which the light from the image projector 930 is directed. In some cases, the image projector 930 is disposed adjacent to the corresponding coupling optical element 942 and / or waveguide 940. The image projector 930 can also be disposed in an optical path including the coupling optical element 942, the coupling optical element 944, and the eye 210.

[0110] Image projector 930 may be a separate component from illumination source 960, such as Fig.10 as well as Figures 11A-11E As shown. However, in some cases, image projector 930 can be used as an illumination source. For example, in addition to injecting an image into eye 210, image projector 930 can be used to direct visible light and / or infrared light into the eye to illuminate the eye for image capture. However, alternatively, one or more separate light sources 960 can be used to illuminate eye 210 for image capture.

[0111] The light emitted by the illumination source 960 may include light of a specific wavelength range, such as invisible light. The illumination source 960 may be configured to project invisible (e.g., infrared) light onto / into the eye 210 to image one or more parts (e.g., cornea, retina) of the eye 210. In some example implementations, the light source 960 may be configured to emit light within a range of about 850nm to 940nm. The light source 960 may be configured to emit light extending over a wavelength range of at least about 20nm. Other ranges are also possible. The wavelength range emitted may be any range between 5nm, 10nm, 15nm, 50nm, 75nm, 100nm, 150nm, 200nm, or any of these values. The light source 960 may be configured to emit light of a broadband wavelength across any range such as within the infrared spectrum.

[0112] The imaging device 920, which may include a camera, may include a detector array and possible imaging optics. The detector array may include, for example, a CCD or CMOS detector array, and the imaging optics may include one or more lenses. The one or more lenses may have positive optical power and an associated focal length. In some designs, the camera 920 is focused at infinity. For example, the optics may have a focal length f, and the detector array may be disposed at a distance away from the optics corresponding to the focal length, so that objects at large distances are imaged onto the detector array. Similarly, collimated light from the eye or an object in the environment will be focused on the detector array to form an image of the eye or object thereon.

[0113] The imaging device 920 may be disposed on the opposite side of the waveguide 940 from the illumination source 960 and / or the eye 210. In some designs, the imaging device 920 may be disposed on the same side of the waveguide 940 as the illumination source 960 and / or the eye 210. Fig.10 As shown, the imaging device 920 may be positioned near the outer edge or temporal edge of the eyepiece 950, although other locations are possible.

[0114] Figures 11A-11E Shows Fig.10 Operation of example imaging system 900 . Fig.11A An illumination source 960 is shown emitting light 902 toward an incoupling optical element 942 on a waveguide 940. As shown, the light 902 can be directed to the eyepiece 950 generally at normal incidence, although other angles are possible. In some designs, the light source 960 is configured to emit collimated light into the eyepiece 950. Fig. 11BAs shown, the illumination light 902 can be coupled into the waveguide 940 via an incoupling optical element 942. In some designs, the incoupling optical element 942 includes a diffractive optical element (e.g., a grating, a holographic element) on which light incident is diffracted at an angle greater than the critical angle of the waveguide so that the incoupled light 904 is guided within the eyepiece 950 by total internal reflection (TIR). In some designs, the incoupling optical element 942 can be configured to direct the light toward the coupling optical element 944. The incoupling optical element 942 can be polarization selective. For example, the incoupling optical element 942 can include a polarization selective steering element, such as a polarization grating, for example a liquid crystal polarization grating. Fig. 11C It is shown how the incoupled light 904 propagates through the waveguide 940 by TIR.

[0115] Fig.11D An exemplary imaging system 900 is shown that couples light out of the eyepiece 950. As the incoupled light 904 propagates through the waveguide 940, some of the light may be incident on the coupling optical element 944. The coupling optical element 944 may be configured to couple the incoupled light 904 out of the eyepiece 950 and toward the user's eye 210. The coupling optical element 944 may be configured to couple the light as collimated light toward the eye 210. The coupling optical element 944 may be tuned to light having a particular wavelength range. For example, the coupling optical element 944 may be configured to couple infrared light (e.g., between approximately 700 nm and 15,000 nm) out of the waveguide 940. In some designs, the coupling optical element 944 may be configured to couple multiple wavelengths of light out of the eyepiece 950. For example, the coupling optical element 944 may be tuned for both infrared and visible light. The coupling optical element 944 may also be configured to couple light into the waveguide 940, as described more fully below.

[0116] The coupling optical element 944 can be configured to increase one or more dimensions of the eye box for the user. For example, the one or more dimensions can be measured along a first axis (e.g., an x-axis). The eyepiece 950 can further include an orthogonal pupil expander (OPE). The OPE can have at least one light redirecting element disposed on or in the waveguide (e.g., on one of the major surfaces), or the OPE can be disposed within the waveguide 940. The OPE can include features similar to or the same as those described above with respect to the light distribution elements 730, 740, 750. In some implementations, the light redirecting element can include a diffractive optical element. The OPE can be configured to increase the size of the eye box along a second axis (e.g., a y-axis) that is orthogonal to the first axis.

[0117] Fig.11DSome of the light exiting the eyepiece 950 toward the user's eye 210 is shown. In some designs, the coupling optical element 944 is configured so that the incoupled light 904 incident on the coupling optical element 944 at various portions of the coupling optical element 944 along a first axis (e.g., parallel to the x-axis) exits the eyepiece 950 at each portion of the coupling optical element 944 along the first axis. This can provide the user with light for projecting an image or illuminating the eye with respect to different eye positions or locations.

[0118] like Figures 11D-11E As shown, the coupling optical element 944 can be configured to couple the coupled light 904 out of the eyepiece 950 as collimated light. The light can also be directed in a generally normal direction relative to the major surfaces of the eyepiece 950 and / or the waveguide 940. The collimated light can be directed into the eye and focused by the eye (e.g., the cornea and natural lens of the eye) onto the retina. The light 908 incident on the retina can provide illumination to image the retina and / or provide image content to the eye. Some of the light 908, for example, can be reflected or scattered away from the retina, emitted from the eye and provide a retinal image to be captured. The light source 960 can be an extended light source so that the light will illuminate an area of ​​the retina.

[0119] Figures 12A-12E Shows Figures 11A-11E How the imaging system 900 may additionally or alternatively be used for image collection of the eye 210. Fig. 12A Light 910 is shown reflecting from the retina leaving the eye 210. As shown, light 910 scattered or reflected from the retina, passing through the natural lens of the eye, the pupil in the eye, and the cornea can be collimated. The light can also be incident on the eyepiece 950 at normal incidence (e.g., at right angles to the major surfaces of the waveguide 940 and / or coupling optical element 944). The coupling optical element 944 can be configured to couple the light 910 reflected from the retina into the waveguide 940.

[0120] Fig. 12BAn exemplary imaging system 900 is shown coupling light into an eyepiece 950. The coupling optical element 944 may include turning features such as a diffractive optical element or other structure that redirects light at an angle greater than a critical angle to guide the light within the waveguide 940. The coupling optical element 944 may be configured to direct the in-coupled light 914 generally toward the light source 960 and / or the imaging device 920. The coupling optical element 944 may be configured to couple out of the waveguide 940 less than a fraction of the light that propagates toward the camera 920. For example, a partially reflective element (e.g., a semi-transparent mirror) may be disposed on or in the waveguide 940 such that a portion of the in-coupled light 914 continues to propagate in the waveguide 940 by total internal reflection while reducing leakage of the in-coupled light 914 from the waveguide 940 along the portion of the waveguide 940 where the coupling optical element 944 is disposed. The fraction of light that does not leak out may be any fraction between 0 and 1. For example, the fraction may be 0.90, where 90% of the light propagating through the waveguide 940 along the coupling optical element 944 is retained within the waveguide at each reflection of the light. Other fractions are also possible (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or a range between any of these values). Such partially reflective elements may be similarly used in the implementations described below.

[0121] like Fig. 12C As shown, the collimated incoupled light 914 may continue to propagate through the waveguide 940 toward the imaging device 920 . Fig.12D 942. It is shown how some of the incoupled light 914 continues to propagate until it is incident on one or more outcoupling optical elements 952. In order to reduce the amount of leakage of the incoupled light 914 from the incoupling optical element 942, the incoupling optical element 942 can be configured to couple very little of the light propagating toward the camera 920 out of the waveguide. For example, a partially reflective element (e.g., a semi-transparent mirror) can be disposed on or in the waveguide 940 so that a portion of the incoupled light 914 continues to propagate within the waveguide 940 by total internal reflection while reducing leakage of the incoupled light 914 from the waveguide 940 along the portion of the waveguide 940 in which the incoupling optical element 942 is disposed. The portion of light that does not leak out can be any fraction between 0 and 1. For example, the portion can be 0.90, where 90% of the light propagating through the waveguide 940 along the coupling optical element 944 is retained within the waveguide at each reflection of the light. Other fractions are also possible (eg, 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or a range between any of these values).Such partially reflective elements may be similarly used in the implementations described below.

[0122] like Fig.12EAs shown, the outcoupling optical element 952 can be configured to couple light guided in the waveguide 940 out of the waveguide 940 and into the imaging device 920. As a result, light propagating in the waveguide 940 incident on the outcoupling optical element 952 is redirected so that it exits the waveguide 940, for example, from a major surface of the waveguide 940 (e.g., the front side or the back side of the waveguide 940) and is directed to the imaging device 920. The outcoupling optical element 952 can be configured to direct light 926 to exit the waveguide 940 perpendicularly (e.g., normal to) the major surface of the waveguide 940. In some designs, the outcoupling optical element 952 is configured to direct the collimated light 924 onto the imaging device 920 at normal incidence to the photosensitive portion of the imaging device 920. As described above, the camera 920 can be focused at infinity, for example, the imaging optics can be configured to focus the collimated light onto the detector array.

[0123] Thus, the waveguide 940 can be configured to guide light coupled from the user's eye 210 into the waveguide 940 to be received by the imaging device 920 (e.g., a camera) to capture an image of at least a portion of the user's eye 210. The same waveguide 940 can be configured to guide light coupled from the image projector 930 so that the light from the image projector 930 can be guided to the user's eye 210 so that the image from the image projector 930 is located in the user's field of view. In some implementations, the same waveguide is configured to guide light coupled from the illumination source 960 so that the light from the illumination source can be guided to the user's eye 210 to illuminate the eye so that an image of the eye can be captured by the camera 920.

[0124] In some implementations, the same coupling optical element 944 can be configured to (i) couple light from the user's eye 210 into the waveguide 940 to be received by the imaging device 920, and (ii) couple light from the image projector 930 out of the waveguide 940 to the user's eye 210 to project image content into the user's field of view. In some implementations, the same coupling optical element 944 can be configured to couple light from the illumination source 960 out of the waveguide to the user's eye 210 so that the light from the illumination source can illuminate the eye.

[0125] In other designs, different waveguides may be used and / or different coupling optical elements 944 may be used. For example, in some designs, a first waveguide 940 may be configured to direct light coupled from a user's eye 210 to be received by a camera 920 to capture an image of at least a portion of the user's eye 210, and a second waveguide may be configured to direct light coupled from an image projector 930 such that light from the image projector 930 may be directed to the user's eye 210. The first waveguide and the second waveguide may be stacked on top of each other. Additionally or alternatively, another waveguide may be configured to direct light coupled from an illumination source 960 such that light from the illumination source may be directed to the user's eye 210 to illuminate the eye.

[0126] In addition, in some implementations, the first coupling optical element 944 can be configured to (i) couple light from the user's eye 210 into the waveguide 940 to be received by the imaging device 920, and (ii) couple light from the image projector 930 out of the waveguide 940 to the user's eye 210 to project image content into the user's field of view. Additionally or alternatively, another coupling optical element can be configured to couple light from the illumination source 960 out of the waveguide to the user's eye 210 so that the light from the illumination source can illuminate the eye.

[0127] In some designs, the coupling optical element 944 may include multiple diffractive optical elements (DOEs). For example, a first DOE may be configured to couple light from the user's eye 210 into the waveguide 940 to be received by the imaging device 920. A second DOE may be configured to couple light from the image projector 930 out of the waveguide 940 to the user's eye 210 to project image content into the user's field of view. Optionally, a third DOE may be configured to couple light from the light source 960 out of the waveguide 940 to the user's eye 210 to illuminate the eye. The first and second (and possibly third) DOEs may be stacked, for example, in some implementations, so that light from the environment in front of the user passes through the first DOE, then impinges on the second DOE, then impinges on the third DOE and impinges on the user's eye. However, the order may be different.

[0128] In some designs, the first DOE and the second DOE are integrated into a single element or volume of the waveguide 940. In some implementations, for example, both the first DOE and the second DOE are superimposed on each other (e.g., occupying the same or approximately the same volume) within the waveguide 2102. For example, the first and second DOEs can be recorded in the same medium.

[0129] As described above, image capture of the eye, such as the retina, can facilitate eye tracking. For example, Fig.13AAn imaging system 900 is shown that is configured to image various portions of the eye 210 (e.g., the retina) at different times, for example, when the eye is in different positions. Phase A and Phase B may refer to images of the eye 210 during different orientations of the eye. Fig.13A Imaging of the eye 210 during both Phase A and Phase B imaging and its results are shown.

[0130] In some implementations, light emissions 928 (e.g., from illumination source 960 as described above or from one or more illumination sources configured and / or positioned differently) can be used to obtain one or more images of the retina 962, such as Fig.13A The image of the retina 962 may include one or more regions 964 , 966 imaged during different orientations of the eye 210 . Fig.13A Two regions 964, 966 of an image of the retina 962 are shown. For example, region 964 of the retina imaged in phase A may be imaged while the eye 210 is directed at an angle normal to the waveguide 940. Image data for region 966 of the retina imaged in phase B may be obtained while the eye 210 is oriented at an acute angle to the waveguide 940. Using one or more orientations of the eye 210 during one or more phases of imaging, a composite image or map of the retina 962 may be obtained. Such as the data module 140 (see Figure 2 ) can be used to find overlapping image data between two adjacent regions. Using the overlapping regional image data, a composite image of the retina 962 can be determined. A larger size (e.g., full size) composite image or map of the user's retina can be stored.

[0131] As described herein, a head mounted display may be used to map the retina of a user's eye based on the direction in which the user's eye is directed. To provide realistic and intuitive interaction with objects in the user's environment using eye gaze and / or to identify the wearer of the head mounted display device, the head mounted display system may use retinal mapping in conjunction with unique eye features of the user and other conditions that may have some influence on eye measurements. For example, an image may be identified based on the location of blood vessels in a corresponding retinal image.

[0132] Retinal mapping may involve a process that enables a computing device to learn how to associate a user's eye gaze (e.g., as identified in a retinal image) with a gaze point in 2D or 3D space. Eye gaze may be associated with a single point in 2D or 3D space. Eye gaze may also be associated with multiple points in space, which may describe the motion of a virtual object (e.g., a series of points, the position of a moving image).

[0133] The head-mounted display system can determine the user's eye gaze based on the retinal image. The head-mounted display system can obtain the retinal image using a sensor (e.g., an eye camera such as imaging device 920). The head-mounted display system can image one or both eyes of the user as the user changes his or her eye gaze (e.g., when the user looks around to follow a moving or shifting calibration target or gaze target). In order to map the user's retina, the head-mounted display system can present a virtual target, such as a gaze target, for the user to view. The virtual target can be associated with one or more known gaze points in 2D or 3D space. When the user gazes at the target, the head-mounted display system can acquire the retinal image and associate the image with the gaze point. The head-mounted display system can calculate and / or generate a mapping matrix based on the association of each retinal image and the gaze point associated with the target.

[0134] The retinal mapping results can reflect the uniqueness of each person's eyes. For example, the head-mounted display system can generate a mapping matrix customized for one or both eyes of a particular individual. For example, a user may have a different amount of eye movement or eye gaze in response to a particular target. Additionally or alternatively, a user may have a different location, size, shape, and / or orientation of blood vessels in the retina. As a result, by generating calibration results specific to a single user, the head-mounted display system can allow more accurate user interaction with eye gaze and / or can allow identification of a particular user.

[0135] Thus, when a user puts on the head mounted display device, the system can detect whether the user is a previous user or a new user. A confusion matrix can be calculated, where the scores of specific eye gaze images stored in the system memory are compared with corresponding images of the current user. The confusion matrix may include comparison scores for multiple eye gazes and associated retinal images. Based on the comparison scores, the system may be able to make a determination about the identity of the user (e.g., whether the user is an individual associated with a stored retinal image or composite map) and / or a confidence level in the determination. The confidence level may, for example, include an identity coefficient. A stored image (e.g., a composite image or map) may be compared to an image obtained later (referred to as an instant or real-time image obtained about the current user). If the system detects that the user is a new user, the system may provide an alert, or other measures may be taken.

[0136] The system can apply filtering, such as digital filtering or image mapping processing, to the image of the retina captured by the camera. Such filtering or imaging processing can, for example, enhance features that can be used for identification, splicing, combining composite images, eye tracking, etc. Such filtering or image mapping processing can include edge enhancement. Although other types of filters can be used, such filters can include, for example, Frangi filters. Such filters or processing (e.g., edge enhancement or Frangi filters) can be used to enhance and / or detect image features such as blood vessels or tubular structures or fibers in retinal images.

[0137] Fig. 13B A pattern of sequentially displayed fixation targets that can be used in a retinal mapping process is shown. These virtual targets to which the user's eyes will direct their gaze can result in redirecting the eye gaze to a variety of different directions, during which the retina can be imaged. The resulting images associated with different gaze directions correspond to different parts of the retina. As described above, when the eye gazes in different directions to view fixation targets at different locations on the display, the images captured by the camera include different parts of the retina. These images can be combined to form a larger retinal map or composite image.

[0138] Fig. 13B Virtual targets are shown at sixteen different locations in the user's field of view (FOV) 1200. In various implementations, a virtual target will be presented at a given location at a given time. During the time when the virtual target is presented to the user at that particular location, one or more retinal images will be obtained. The image or images may be associated with the target location and / or corresponding gaze direction. More or fewer target locations may be used. Fig. 13BIn the example shown, sixteen target locations 1202a–1202p are shown. More or fewer target locations may be used. The target locations may also be different. The order in which the targets are displayed at different locations may be different. For example, the target may move from the left side of the user's field of view to the user's field of view, back from right to left, and then from left to right in a raster pattern, thereby lowering the position of the target in the field of view, where each lateral channel spans the field of view. However, other patterns and methods are also possible. Similarly, the targets may be rendered the same or differently at different locations. For example, the rendered targets may be different sizes, shapes, colors, etc. The targets may be rendered sequentially to the user during the eye tracking calibration process. For example, as described above, the head-mounted display system may render the targets in a serpentine pattern. For example, target 1202a may be followed by 1202b, then 1202c, then 1202d, then 1202h, then 1202g, and so on. Other patterns are also possible. For example, the targets may be displayed in a more random or non-sequential pattern. In some embodiments, a single target is displayed to the user, and the target moves around the user's field of view (e.g., passing through or temporarily stopping at positions 1202a-1202p during the target's movement). The head-mounted display system can acquire images of the user's retina while the user is looking at these targets. For example, the head-mounted display system can acquire a first image when the user is looking at a target at a first position 1202a, a second image when the user is looking at a target at a second position 1202b, and a third image when the user is looking at a target at a third position 1202c, and so on. The wearable system can associate the first image with the first position 1202a, the second image with the second position 1202b, the third image with the third position 1202c, and so on. Adjacent images can be stitched together in a database to create a complete or partial retinal map. For example, features or portions of features common to multiple images (e.g., blood vessels or portions thereof) can be used to stitch two images together with appropriate registration. In various implementations, adjacent target locations will produce overlapping images that can be aligned and stitched together. For example, target location 1202a and target location 1202b, and target location 1202b and target location 1202c can produce overlapping and adjacent retinal images that can be stitched together. Thus, many different retinal images can be obtained at different eye gazes, thereby combining a larger image of the retina (e.g., a composite image or map).

[0139] As described above, eye tracking can be performed using a synthetic retinal image or mapping. For example, after the target is no longer displayed, the user can move his eye gaze when he looks at different real objects in front of the user and the head-mounted display or the augmented reality (virtual) image content displayed by the head-mounted display. One or more retinal images can be obtained at these times. The term "instantaneous" or "real-time" image can be used in this article to describe these images obtained after calibration, which can be used for eye tracking (or other purposes, such as obtaining biological data). These "instantaneous" or "real-time" images may correspond to a portion of a synthetic retinal image or mapping. The system can be configured to fully match the "instantaneous" or "real-time" retinal image with a portion of a composite retinal image or retinal mapping. Such a match can be based on a "instantaneous" or "real-time" retinal image and a portion of a composite retinal image or mapping that have common features or portions of features (blood vessels or portions thereof). Based on the position where the "instantaneous" or "real-time" retinal image overlaps with a portion of a synthetic retinal image or mapping, the gaze direction can be inferred. Different gaze directions will produce retinal images corresponding to different portions of the retinal mapping. Thus, identifying the location of the "instantaneous" or "real-time" retinal image on the synthetic retinal image or map will provide information about the direction of the user's gaze. Such or similar methods can be used to perform eye tracking, for example, tracking eye movement and changes in eye gaze. As described above, edge enhancement, edge detection, or other digital filtering and / or processing can be used to enhance features of different images and / or associate features of different images with the synthetic retinal image or retinal map.

[0140] In various implementations after completing an initial calibration process in which a virtual target or gaze target is displayed (e.g., at multiple locations) to assemble a synthetic retinal image or map, the synthetic retinal image or map may still be refined. For example, as additional retinal images are obtained, the additional images may be used to further refine or improve the synthetic retinal image or map. Thus, as additional "instantaneous" or "real-time" retinal images are obtained, for example, for the purpose of providing eye tracking, the "instantaneous" or "real-time" may be used to further refine or improve the synthetic retinal image or map. As the user continues to look at various locations in the display (with or without the aid of calibration targets), the retinal synthetic image or map may be further refined using additional images acquired after the initial calibration in which the virtual target or gaze target is displayed. Thus, the quality of the synthetic retinal image or map may be increased.

[0141] Additional non-limiting examples of how eye tracking may be accomplished and / or how a synthetic retinal image or map may be generated and the retinal images used are described in U.S. Publication No. 2017 / 0205875, filed on January 17, 2017, entitled “EYE IMAGE COLLECTION,” the disclosure of which is incorporated herein by reference in its entirety.

[0142] Thus, as described above, a larger portion of the retina can be recorded and mapped by obtaining retinal images and / or other images of the eye using an imaging system such as described herein, and such images can facilitate eye tracking. For example, when the eye is in an arbitrary position, it can be captured Fig.13A The image of the eye 210 shown. The processing electronics or processor (e.g., the same or different processing electronics or processor as described above for forming the composite image) can then compare the captured image of the user's retina in real time with the stored composite or larger size (e.g., full size) image to track the movement of the eye. A given image of the user's retina captured in real time may show a specific portion of the user's retina. As described above, by comparing such a captured image with a stored image of a larger portion of the user's retina mapped by the user, the system can determine which portion of the user's retina is shown in the captured image, and thus the position / orientation of the eye that will produce such an image can be determined. See Figure Example 13A, which shows two different images of the retina portion produced when the eye is in two different positions and / or orientations. Therefore, the position and / or orientation of the eye can be determined by capturing different images of the retina and determining which portion of the retina is visible. This determination can be performed even if a composite image is not formed but multiple images of the retina for different eye positions / orientations are recorded and stored in a database. When a future image of the retina is obtained, the image can be compared with an image in a stored image database to determine which image in the database is similar to the most recently obtained image of the eye. Matching the most recent image with one or more images in a database having associated positions and / or orientations may enable determination of the orientation and / or position of the more recent image.Other eye tracking methods may be used based on images captured using the designs described herein.

[0143] As described herein, retinal images may also be used for other purposes. For example, a retinal image may be used to verify that the user is the same user for whom a synthetic retinal image or map was acquired. A retinal image obtained while the user is wearing the head-mounted display system (e.g., during a calibration process and / or during later use) may be compared to a previously-acquired synthetic retinal image or map that is stored (e.g., created the previous day or when the head-mounted display was previously activated). If the recently-acquired retinal image does not match a portion of the synthetic retinal image or map, it may be concluded that the current user is different from the previous user (e.g., for whom the synthetic virtual image or map was created). Such an approach may be used for security, for example, to verify that the current user of a head-mounted display device is the owner or a typical user of the device. Thus, biometric data obtained via retinal imaging may be used for security purposes.

[0144] Retinal imaging can also be used to collect biometric data to monitor the health of a user. Medically relevant data can be obtained from retinal images. Such medical data may be useful for monitoring the health of a user.

[0145] Although various applications of eye imaging, such as eye tracking, collection of biometric data for health monitoring and security, are discussed herein in the context of retinal imaging, other portions of a user (e.g., a user's eyes) may be imaged for these and other purposes.

[0146] Although the above description describes that eyepiece 950 can be used to facilitate imaging of the eye, the eyepiece can also be used to image the world in front of the user. For example, Figures 14A-14B An example imaging system 900 is shown that can be used to image a portion of an environment located in front of a user and / or an object in the portion of the environment. The imaging system 900 used can be related to Figures 11A-11E and / or about Figures 12A-12E The system described is similar to the system described above, except that light from the environment is collected toward the eyepiece and the user by the eyepiece 950. For example, Fig.14A Light 970 from the environment is shown, which is reflected and / or emitted by one or more physical objects 972 in the environment and is directed toward the user and the eyepiece 950. As shown, the light 970 from the environment can be approximately collimated (e.g., at infinity), for example because the physical objects 972 in the environment may be located at a sufficiently large distance from the imaging system 900 so that the light reaching the imaging system 900 is collimated or approximately collimated. In some implementations, the imaging system 900 can be configured to image the environment and / or objects in the environment without using any optical elements (e.g., lenses, mirrors) having optical power in the imaging system 900.

[0147] Fig.14Aand 14B The imaging system 900 shown is similar to the imaging systems described above. The imaging system includes an eyepiece 950 that includes one or more waveguides 940 that include a coupling optical element 944 that is configured to guide light from an image projector 930 (not shown) into the eye 210 to form an image therein. The one or more waveguides may include a plurality of waveguides (e.g., a stack of waveguides) that are configured to couple in / out a plurality of corresponding colors / wavelengths. Each waveguide in the waveguide stack may be configured to guide light of a particular color (e.g., red, green, blue). For example, the farthest waveguide (e.g., a stack of waveguides) may be configured for visible light (e.g., red, blue, green) such that the waveguide is configured to couple in and out visible light of the same wavelength. Additionally or alternatively, a waveguide configured to couple in and out invisible (e.g., infrared) light may be disposed proximate to the eye 210. Such a plurality of waveguides corresponding to the waveguide 940 may be used in any other implementation described herein. The imaging system 900 may also include an imaging device (e.g., a camera) 920 and an outcoupling optical element 952 configured to redirect light reflected from the eye 210 propagating within the waveguide 940 to the camera. Fig.14A and 14B In the embodiment of the present invention, the illumination source 960 is excluded because the illumination source may not be needed to image the environment in front of the user. However, an illumination source (e.g., the light source 960 described above) may be used in some designs.

[0148] The eyepiece 950, waveguide 940, coupling optical element 944, out-coupling optical element 952, and camera 920 may be the same or similar as described above. For example, the coupling optical element 944 may be physically coupled to the waveguide 940. For example, the coupling optical element 944 and / or the out-coupling optical element 952 may be disposed in an optical path between the environment in front of the eyepiece 950 and the camera 920, such that light from the environment is coupled into the waveguide 940 via the coupling optical element 944 and is coupled out of the waveguide via the out-coupling optical element to be incident on the camera 210 (e.g., to form an image of at least a portion of the environment). The coupling optical element 944 may include a plurality of steering features configured to turn light guided within the waveguide out of the waveguide or to turn light incident on the coupling optical element 944 at an angle into the waveguide to be guided therein by total internal reflection. The outcoupling optical element 952 may include a plurality of steering features configured to steer light (from the environment) guided within the waveguide at an angle such that the light is not guided in the waveguide by total internal reflection but is directed out toward the camera. The coupling optical element 944, the outcoupling optical element 952, and the steering features associated therewith may be physically coupled to the waveguide 940. For example, the coupling optical element 944 and / or the outcoupling optical element 952 may include one or more holographic or diffractive optical elements (e.g., surface relief gratings) patterned (e.g., etched) in or on the waveguide 940. The coupling optical element 944 and / or the outcoupling optical element 952 may include a layer disposed on the waveguide 940, or may be formed in the waveguide 940. For example, a volume holographic or diffractive optical element may be formed by changing the refractive index of a material comprising the waveguide or a layer disposed thereon. Thus, the coupling optical element 944 and / or the outcoupling optical element 952 may be disposed in the volume of the waveguide 940 or a layer disposed thereon. Depending on the design, the coupling optical element 944 and / or the coupling-out optical element 952 may be transmissive or reflective, and may operate in a transmissive or reflective manner. For example, the coupling optical element 944 and / or the coupling-out optical element 952 may include a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates in a transmissive or reflective manner, respectively, for example, thereby redirecting light transmitted therethrough or reflected therefrom. The coupling optical element 944 and / or the coupling-out optical element 952 may include a polarization optical element, such as a polarization selective steering element (e.g., a polarizer). The polarization selective steering element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may include a liquid crystal structure, such as a liquid crystal polarization grating. In some implementations, the reflective optical element may include a reflector (e.g., a mirror). For example, other elements of the waveguide 940 may also be similar to those described above.

[0149] Fig. 14B Shows Fig.14AOperation of the imaging system 900 is shown. Light 970 from the environment is coupled into the waveguide 940 via the coupling optical element 944. The coupling optical element 944 can be configured to deflect the collimated light at an angle greater than the critical angle of the waveguide 940, so that at least a portion of the collimated light is directed within the waveguide toward the camera 920 by total internal reflection. The outcoupling optical element 952 can be configured to receive at least a portion of the light from the environment in front of the user, at least a portion of which is coupled into the waveguide 940 via the coupling optical element 944 and directed therein. The outcoupling optical element 952 can be configured to couple the incoupled light out of the waveguide 940 to the camera 920, so that an image of the environment can be captured by the camera 920. The image of the environment can be communicated to processing electronics (e.g., one or more processors), such as the data module 140 (see Figure 2 ). Data module 140 may be configured to reproduce a modified image of the environment in an augmented reality context. Processing electronics may communicate with camera 920 via wired or wireless electronic signals. Additionally or alternatively, processing electronics may communicate with camera 920 using one or more remote receivers. Processing electronics may reside remotely (e.g., a cloud computing device, a remote server, etc.).

[0150] Thus, the imaging system 900 can be used to directly image the environment, which can be useful for a variety of reasons. For example, imaging the environment can be used to determine where to place augmented reality image content relative to objects in the environment. For example, imaging the environment can provide the location of a table so that the head mounted display can render an image of a person standing near the table rather than on or in the table. The imaging system 900 described for imaging the environment can also be used to image the eye 210, for example with respect to Fig.10 , 11A -11E and / or as described in 12A-12E.

[0151] It may be desirable to use imaging system 900 to image a wide field of view of an environment. Fig. 14C An imaging system 900 is schematically shown for collecting light from the environment using a powered optical element or lens, such as a refractive optical element 980 (e.g., a wide field lens), located in front of the eyepiece. The refractive optical element 980 may have positive optical power. The refractive optical element 980 (e.g., a positive lens) focuses collimated light 970 from the environment toward the waveguide 940. Fig. 14COther types of lenses other than the lenses shown. The transmitted light (not shown) can pass through a powered optical element or lens, such as a refractive optical element 990 (e.g., a negative lens) configured to have a negative optical power equal to and opposite to that of the refractive optical element 980. The negative lens 990 can have a similar or identical optical power to the positive lens 980 to compensate for or cancel out the optical power of the positive lens or a portion thereof. In this way, light from the environment (e.g., the distal end of the waveguide 940) can pass through the negative lens 990, the eyepiece 950, and the positive lens 980, with substantially no net change in the optical power introduced into the eye by the two lenses. The negative lens 990 can be configured to compensate for or cancel out the optical power of the positive lens 980, so that when viewing the environment in front of the eyepiece 950, the user will not experience the optical power of the positive lens. The negative lens 990 will also cancel out the effect of the positive lens 980 to invert the image of objects in the environment in front of the wearer. Although some light rays converge, some light 970 from the environment can be coupled into the waveguide 940 by the coupling optical element 944. The incoupled light incident on the outcoupling optical element 952 may be emitted from the waveguide 940 .

[0152] Implementations may be used outside of an augmented reality environment (e.g. Figures 14A-14C Implementations described herein). For example, it is intended to implement an imaging system 900 configured to image an environment in a wearable device (e.g., glasses (including afocal glasses) or bifocals). Such an imaging system 900 may not require an image projector 930 and / or a light source 960. Additionally or alternatively, such an imaging system 900 may not require coupling optical elements configured for corresponding image projectors 930 and / or light sources 960.

[0153] It may be advantageous to implement an imaging system 900 for imaging an environment on a viewing screen (e.g., a television screen, a computer screen) such as a handheld device (e.g., a cell phone, a tablet computer). The imaging system 900 may improve video chat capabilities. For example, a viewer who sees a chat partner looking at the screen may appear to be directly at the viewer. This will be possible because the light captured by the imaging system 900 will be captured in the same area that the user is looking at (e.g., as opposed to the viewing screen, but with light captured by a separate outward-facing camera located at a different location).

[0154] In it Fig. 14CIn an implementation where the imaging system 900 is also used to image the eye 210, the light source 960 and / or the image projector 930 may be configured to inject light into the waveguide 940. Because the light reflected from the eye that is coupled into the waveguide will pass through the refractive optical element 990 (e.g., a negative lens), a positive power refractive optical element may be disposed between the light source 960 and / or the image projector 930 and the waveguide 940. The positive lens may be configured to compensate or cancel out any optical power provided by the refractive optical element 990 before the in-coupled light from the light source and / or the light projector is incident on the eye 210. Fig. 14C Other types of lenses other than those shown in FIG. 9 may be used as optical element 990. Alternatively or additionally, processing electronics in communication with the light source and / or image projector may be configured to alter the image sufficient to present an undistorted image to a user after the light has passed through refractive optical element 990. In some designs, corresponding in-coupling optical elements, out-coupling optical elements, and / or coupling optical elements may be configured to operate on non-collimated light (e.g., diverging, converging light).

[0155] In various implementations, the same waveguide 940 may be used to (i) propagate light from the eyepiece 950 and the environment in front of the user to the camera 940, and (ii) propagate light from the image projector 930 to the eye 210 to form image content therein. Using the same waveguide 940 may simplify the system and / or the eyepiece, and may make the system and / or the eyepiece more compact, potentially providing a reduced form factor. Reducing the thickness of the eyepiece 950 by reducing the number of waveguides 940 may also be advantageous for other reasons. Lower costs and a more simplified manufacturing process may be some such advantages.

[0156] Also in various designs, the same or different imaging systems may be used in the same head mounted display to transmit light from the eye to the camera 940 via a waveguide in the eyepiece 950, for example as described above. Such a system may also use the eyepiece to transmit light from an illumination source to the eye 210 to illuminate the eye. In some designs, the eyepiece may additionally be used to transmit light from the image projector 930 to the eye 210 to form image content therein. Using the eyepiece to help image the environment and image the eye (and possibly illuminate the eye) may simplify the system and / or may make the system more compact, thereby potentially providing a reduced form factor.

[0157] Additionally, in some implementations, the same waveguide 940 may be used to (i) propagate light from the environment in front of the eyepiece 950 to the camera 940, and (ii) propagate light from the eye 210 to the camera to capture an image of the eye. The same waveguide may be used to propagate light from the image projector 930 to the eye 210 to form image content therein and / or to propagate light from the illumination source 960 to the eye 210 to illuminate the eye for image capture. Using the same waveguide 940 may simplify the system and / or eyepiece, and may make the system and / or eyepiece more compact, thereby potentially providing a reduced form factor. Reducing the thickness of the eyepiece 950 by reducing the number of waveguides 940 may also be advantageous for other reasons. Lower costs and a more simplified manufacturing process may be some such advantages.

[0158] Similarly, in addition to coupling light from the environment into the waveguide 940, the same coupling optical element 944 can be configured to direct light from the image projector 930 to the eye 210 to form image content therein and / or direct light from the eye into the waveguide 940 to be directed therein to the camera 920. Additionally or alternatively, the same coupling optical element 944 can be configured to couple light from the illumination source 960 guided within the waveguide 940 out of the waveguide 940 to the eye 210 of the user.

[0159] As described above, one or more of the coupling optical element 944, the coupling-in optical element 942, or the coupling-out optical element 952 may include a polarization-selective coupling element. Thus, in various designs, light input into the eyepiece 950 or the waveguide 940 is polarized so as to be properly acted upon by the polarization-selective steering element.

[0160] Thus, in some embodiments, illumination source 960 comprises a polarized light source having an appropriate polarization to be properly acted upon by a polarization-selective coupling / steering element.

[0161] One or more polarization-specific filters and polarization-modifying elements may be included in various imaging systems 900, such as those in which image projector 930 and / or light source 960 are positioned directly opposite one another via waveguide 940. For example, in a configuration in which polarization-sensitive elements are aligned on opposite sides of waveguide 940 at the same lateral position, the polarization-sensitive elements may help reduce directional light emission into imaging device 920 and / or reduce saturation of imaging device 920. Figures 15A-15B Such a configuration is shown in FIG. Fig.15AThe illustrated light source 960 may be configured to direct light through a polarization-specific filter 982, such as a polarizer (e.g., a linear polarizer) and / or through a polarization-modifying element 986 (e.g., a polarization rotator) configured to change the polarization state of the incident light. A retarder, such as a half-wave retarder, may, for example, rotate a linear polarization. Thus, a properly oriented half-wave retarder or half-wave plate may rotate s-polarized light to p-polarized light and vice versa. Thus, in various implementations, a polarization-specific filter 982 and / or a polarization-modifying element 986 is disposed in the optical path between the light source 960 and the coupling-in optical element 942 so as to provide a properly oriented polarization to the coupling-in optical element. In some implementations, the imaging system 900 does not include a polarization-modifying element, but instead includes a properly oriented polarization filter, such as a polarizer.

[0162] The light emitted by the light source 960 may pass through the arrangement of optical elements in a particular order. Fig.15A As shown, light from light source 960 may first pass through polarization-specific filter 982 (e.g., polarizer) and then through polarization-modifying element 986 (e.g., rotator). After the light has passed through polarization-modifying element 986, the light may be incident on incoupling optical element 942, which may direct the light into waveguide 940 to be guided therein.

[0163] For example, the light source 960 can be configured to emit light of mixed polarization (e.g., s-polarization and p-polarization). The polarization-specific filter 982 can be configured to transmit only light of a first polarization state (e.g., p-polarization). As the light continues, the polarization modification element 986 can be configured to change the polarization state of the light (e.g., from p-polarization to s-polarization). The coupling optical element can be configured to convert the s-polarized light into an angle greater than the critical angle of the waveguide so that the s-polarized light is guided within the waveguide. When the coupled-in light 904 propagates through the waveguide 940, the coupled-in light 904 can be substantially polarized with a second polarization (s-polarization). The coupling optical element 944 can be configured to redirect only light of a second polarization state (s-polarization). The coupling optical element 944 can be configured to couple the coupled-in light 904 out of the waveguide 940 and couple to the eye 210 to provide illumination for image capture.

[0164] In order to prevent direct illumination (e.g., saturation) of the imaging device 920, a polarization-modifying element 958 and / or a polarization-specific filter 984 may be disposed in or on the waveguide 940 so that only light of a specific polarization state (e.g., p-polarization) may pass through the polarization-specific filter 984 and reach the imaging device 920. The polarization-modifying element 958 (e.g., a half-wave plate) may be configured to change the state of polarization (e.g., from s-polarization to p-polarization). The polarization-specific filter 984 may be configured to transmit only light of a specific polarization (e.g., p-polarized light) therethrough. In this way, light passing through the polarization-specific filter 982 will not be configured to be directly transmitted through the polarization-specific filter 984. In the above, for example, Fig.10 , 11A -11E and 12A-12E (e.g., where the image projector 930 and / or the light source 960 are located on the same optical axis, such as Fig.15A ), the polarization-specific filter 982, the polarization-modifying element 986, the coupling-in optical element 942, the polarization-modifying element 958 and / or the polarization-specific filter 984 may be configured according to Fig.15A Polarization-specific filter 984 may be a transflective polarizer (eg, a polarizer beam splitter) configured to transmit light of a first polarization and redirect or reflect light of a second polarization different from the first polarization.

[0165] A partially reflective element (e.g., a semi-transparent mirror) may be included to redirect the incoupled light 904 toward the imaging device 920. The partially reflective element may be disposed between the incoupled optical element 942 and the polarization changing element 986 such that a portion of the incoupled light 914 is reflected toward the imaging device 920 while reducing leakage of the incoupled light 914 from the waveguide 940. The portion of light that does not leak may be any fraction between 0 and 1. For example, the portion may be 0.90, where 90% of the light propagating through the waveguide 940 along the coupling optical element 944 is retained within the waveguide 940 at each reflection of the light. Other portions are also possible (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or a range between any of these values).

[0166] Fig. 15B The propagation of light reflected or scattered off the retina is shown. Some of the light 910 incident on the coupling optical element 944 having a second polarization (s-polarization) that is reflected by the retina is turned by the coupling optical element 944 at an angle greater than the critical angle of the waveguide 940 and can thereby be guided therein. Some of the light may not be coupled into the waveguide 940 and will be transmitted therethrough as uncoupled light 912. Incoupled light 904 can propagate through the waveguide 940 and toward the camera.

[0167] Other implementations may benefit from the use of polarization selective elements near the light source and the camera. For example, various systems may be configured to provide illumination with a first polarization and capture images through a camera using light with a different polarization. For example, when imaging the retina, this configuration can be used to reduce unwanted reflections, such as reflections from the cornea. The reflection from the cornea will be specular. Therefore, if light of a first polarization is incident on the cornea, the light reflected from the cornea will maintain the first polarization. In contrast, the retina is diffuse. If light of a first polarization is incident on the retina, the light reflected from the retina will not only maintain the first polarization. Diffuse reflection is more likely to result in the generation of unpolarized light. Therefore, there will be a second polarization different from the first polarization in the reflected light. Similarly, by illuminating with a first polarization and imaging with a second different polarization, the retina can be imaged with reduced glare from the cornea.

[0168] Thus, in various implementations, polarization-specific filters 982, 984 can be used together to reduce unwanted reflected light from the eye 210 (e.g., from the cornea). For example, unwanted light, glare, or glint may be reflected from the cornea, which may saturate the image captured by the imaging device 920. The light reflected from the cornea may be specular and retain its polarization. In contrast, light reflected from the retina may be more diffusely reflected and may be weakly uniformly polarized. Similarly, a combination of polarizers can be used to remove some or most of the unwanted reflected light. Initially, polarized light can be used to illuminate the eye 210. In some designs, a polarized illumination source (e.g., light source 960) can be used. Additionally or alternatively, a first polarizer (e.g., polarization-specific filter 982) can be located at the beginning of the light path of the illumination source to provide an initial polarization of the light. A second polarizer (e.g., polarization-specific filter 984) can be positioned at the light path before the light enters the imaging device 920. The second polarizer can be rotated 90° from the first polarizer (e.g., polarizers 982, 984 can be "crossed"). As a result, the eye will be illuminated with some light of the first polarization reflected from the cornea at the first polarization. This light will not pass through polarizer 984 near the camera. However, light reflected from the retina will include the second polarization. Similarly, light diffusely reflected from the retina will pass through polarization 984 near the camera and will enable the camera to capture an image of the retina. Thus, in such a configuration, unwanted light received from the eye (e.g., from the cornea) that may enter the imaging device 920 can be reduced or eliminated. Other configurations are also possible. For example, a polarization-selective in-coupling optical element 942 for coupling light from the light source 960 into the waveguide 940 and a polarization-selective out-coupling optical element for coupling light out of the waveguide to the camera 920 can be employed to have different polarization-sensitive characteristics. For example, the polarization-selective in-coupling optical element can selectively divert light from the illumination source having a first polarization into the waveguide, while the out-coupling optical element can selectively divert light having a second, different polarization from the waveguide to the camera. The effect can again be to reduce or remove unwanted light received from the eye (e.g., from the cornea) before entering the imaging device 920.

[0169] Various imaging systems 900 are discussed herein that can collect light to image the retina using an eyepiece 950. However, the imaging system 900 can be configured to image other portions of the eye, such as the front of the eye. Fig.16900 is shown how the imaging system 900 can be used to image the front of the eye 210 (e.g., the cornea). The imaging system 900 can include one or more elements of the example imaging system 900 described above. In addition, the example imaging system 900 can include one or more optical elements or lenses with focal power, such as refractive optical elements 980, 990 with focal power. For example, a positive optical power lens or positive lens 980 can be disposed proximal to the eyepiece 950 between the eye 210 and the eyepiece (e.g., closer to the eye 210). A negative optical power lens or negative lens 990 can be disposed distal to the eyepiece 950 between the eyepiece and the environment in front of the user. One or both of the lenses 980, 990 can be variable focus elements (e.g., zoom lenses) and / or can include liquid crystal elements. In some designs, one or both of the lenses 980, 990 include Fresnel lenses. The lenses 980, 990 can be combined with liquid crystals to produce Fresnel lens functions. Such functionality may allow for variable focus of one or both of the lenses 980, 990. In some designs, one or more of the lenses 980, 990 may be integrated with the eyepiece 950 and / or manufactured (e.g., formed) on or in the eyepiece 950.

[0170] In various embodiments, the coupling optical element 944 is configured to redirect collimated light reflected from the eye 210 into the optical waveguide to be guided therein. Thus, the positive lens 980 can be configured to collimate light reflected from the eye 210, such as the front portion of the eye (e.g., the cornea). Thus, the positive lens 980 can have a focal length that is equal to or substantially equal to the distance of the lens to the portion of the eye 210 to be imaged (e.g., the cornea).

[0171] The negative lens 990 may have a similar or identical optical power to the positive lens 980 to compensate for or offset the optical power of the positive lens. In this manner, light from the environment (e.g., the distal end of the waveguide 940) may pass through the negative lens 990, the eyepiece 950, and the positive lens 980 with substantially no net change in the optical power introduced by the two lenses. Thus, the negative lens 990 may be configured to offset or cancel the optical power of the positive lens 980 such that the user will not experience the optical power of the positive lens when viewing the environment in front of the eyepiece 950. The negative lens 990 will also offset the effect of the positive lens 980 to invert the image of objects in the environment in front of the wearer.

[0172] Fig.16Light 928 scattered therefrom is shown incident on the cornea. The imaging system 900 can be configured to capture this light 988 reflected from the cornea. For example, the positive lens 980 can collect a portion of the light 988 scattered from the cornea and collimate the light 988. The light 988 collimated by the positive lens 980 is incident on the coupling optical element 944, which is configured to redirect the collimated light into the waveguide 940 at an angle greater than the critical angle of the waveguide so that the light is guided therein by TIR. The coupling optical element 944, the decoupling optical element 952 and / or the waveguide 940 can be as described above. The generated decoupled light 906 can be led from the waveguide 940 to a camera (not shown) by the decoupling optical element 952.

[0173] Fig.16 Light 928, such as collimated light, is shown, which may come from the eyepiece 950 as described above. An illumination source 960 may couple light into the waveguide 940, and a coupling element 944 may couple light from the illumination source 960 out of the waveguide. The coupling element 944 may be configured to couple the light out of the waveguide 940 as collimated light. The light illuminates the front of the eye (e.g., the cornea) and is scattered therefrom. As described above, this scattered light 988 may be collected by the positive lens 980 and the imaging system 900 to form an image of the front of the eye 210. Also as described above, the illumination 928 directed onto the eye 210 may be invisible (e.g., infrared) light.

[0174] Fig.16 Also shown is an optional arrangement for illuminating the eye 210. In some designs, one or more light sources 934, such as LEDs or emitters, can be arranged relative to the eye 210 to guide light thereon without being guided through the waveguide 940 and to the eye 210 via TIR. In some implementations, the eyepiece 950 or the waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210. In some designs, multiple such light sources 934 can be arranged in a pattern (e.g., a circular or annular pattern) near and / or around the eye. In some designs, the pattern of light sources 934 can define an illumination axis that is parallel (e.g., coaxial) to the optical axis of one or more lenses 980, 990. The one or more light sources 934 can be similar to the one or more light sources 960 described above and can be, for example, pulsed. Similarly, the one or more light sources 934 can include an infrared light source such as an infrared LED or another type of invisible light. Alternatively, the one or more light sources can include a visible light source that emits visible light. Alternatively, the one or more light sources can emit visible light and invisible light (e.g., infrared light).

[0175] Fig.17 Another example imaging system 900 is shown that is configured to image a portion of the eye 210, such as the front portion of the eye (eg, the cornea). Fig.16 Compared to the transmissive optical element (lens) 980 shown, Fig.17 The imaging system 900 shown employs a reflective optical element 996 configured to collimate light from the eye. Reflective optical elements will have smaller aberrations than transmissive optical elements, and chromatic aberration is generally not applicable to reflective optical elements, such as Fig.17 Thus, by using a reflective surface to collect light from the eye 210, fewer aberrations (e.g., chromatic aberrations) may be introduced into the captured image of the eye.

[0176] Fig.17 For example, an imaging system 900 is shown that includes a curved transmissive optical element 996 having a wavelength-dependent reflective coating 998. The curved transmissive optical element 996 can be disposed at the distal end of the waveguide 940 (on the ambient side of the eyepiece 950). Thus, the curved transmissive optical element 996 can be disposed between the environment in front of the wearer and the waveguide 940 and / or the coupling optical element 944. Similarly, the waveguide 940 and / or the coupling optical element 944 can be disposed between the curved transmissive element 996 and the eye 210.

[0177] The wavelength-dependent reflective coating 998 can be configured to reflect light of a specific wavelength or wavelength range. In some implementations, for example, the wavelength-dependent reflective coating 998 can be configured to reflect invisible light (e.g., infrared light) within a certain wavelength range, while the wavelength-dependent reflective coating 998 can be configured to transmit visible light. In some cases, the wavelength-dependent reflective coating 998 can be disposed on a surface of the curved transmissive optical element 996.

[0178] As described above, in various designs, the coupling optical element 944 is configured to redirect collimated light reflected from the eye 210 into the waveguide 940 to be guided therein. Thus, the reflective optical element 996 can be configured to collimate light reflected from the eye 210, such as the front portion of the eye (e.g., cornea). Thus, the curved reflective optical element 996 can have a positive optical power for light incident on the near side of the curved reflective optical element 996 that is reflected from the wavelength-dependent reflective coating 998. In particular, in various designs, the reflective optical element 994 can have a focal length that is equal to or substantially equal to the distance from the reflective optical element 996 to the portion of the eye 210 to be imaged (e.g., cornea, iris, etc.). Example values ​​for the focal length can be, for example, 2 cm to 8 cm. In some implementations, the focal length is between 4 cm and 6 cm. In some designs, the focal length is about 5 cm. The focal length can be within any range formed by any of these values ​​or can be outside such a range in different designs.

[0179] In various implementations, the reflective optical element 996 is disposed on the far side of the eyepiece 950 in front of the eyepiece. Thus, the reflective optical element 996 is disposed between the eyepiece 950 and the environment in front of the user. Similarly, the eyepiece 950 is disposed between the reflective optical element 996 and the eye 210.

[0180] The curved transmissive optical element 996 can have a curved reflective surface having a curvature of any shape. In some implementations, the surface is rotationally symmetric. In some implementations, the surface can be spherical or non-spherical (e.g., parabolic). Non-rotationally symmetric shapes are also possible. However, in various designs, the reflective surface has positive optical power. The reflective optical element 996 can include, for example, a concave mirror at least for certain wavelengths and / or polarizations.

[0181] The curved transmissive optical element 996 can be configured to have negligible focal power when transmitting. Likewise, the curved transmissive optical element 996 can be configured to transmit light without introducing convergence or divergence. In one example, the curved transmissive optical element 996 can have a curvature of an inner radius that is substantially the same as a curvature of an outer radius. A thin optical element 996 can, for example, reduce optical aberrations for light transmitted therethrough, can be lighter and / or more compact.

[0182] In various designs, the reflective optical element 996 includes a material that transmits visible light so that the user can see the environment in front of the wearer. In some cases, in order to enhance transmission, the curved transmissive optical element 996 can be coated with an anti-reflective coating on the outer surface (e.g., the distal surface). The anti-reflective coating can be configured to reduce the reflection of visible light such as red, green and / or blue light, for example. However, the reflective optical element 996 can be configured to reflect a portion of the light scattered from the eye 210 to form an image of the eye. Therefore, the reflective optical element 996 can operate differently with respect to different light. For example, the reflective optical element 996 can operate differently with respect to different wavelengths. The reflective optical element 996 can be configured to reflect infrared light and transmit visible light.

[0183] As described above, one or more light sources 934 can be configured to illuminate eye 210 with infrared light. Fig.17, the resulting light 988 reflected from the eye 210 (e.g., cornea) can be divergent. The curved transmissive optical element 996 can be configured to receive the light 988 reflected from the eye (e.g., cornea, iris). The wavelength-dependent reflective coating 998 can be configured to reflect the light 988 reflected from the eye because the wavelength illumination used to illuminate the eye is the same wavelength (e.g., 850 nm) that is reflected by the reflective coating on the curved transmissive optical element 996. For example, the eye can be illuminated with infrared light (e.g., 850 nm), and the curved transmissive optical element 996 can be configured to reflect the infrared light (e.g., 850 nm) and pass visible light. The shape of the curved transmissive optical element 996 can also be configured to collimate the light 988 reflected from the eye and reflect the light to the coupling optical element 944, which redirects the collimated light into the waveguide 940 to be TIR-guided therein.

[0184] exist Fig.17 In some other designs, as in some other designs, one or more light sources 934, such as LEDs or emitters, can be arranged relative to the eye 210 to guide light thereon, without being guided through the waveguide 940 and onto the eye 210 via TIR. In some implementations, the eyepiece 950 or the waveguide 940 is not in the light path between the one or more light sources 934 and the eye 210. In some designs, multiple such light sources 934 can be arranged in a pattern (e.g., a circular or annular pattern) near and / or around the eye. In some designs, the pattern of light sources 934 can define an illumination axis that is parallel (e.g., coaxial) to the optical axis of one or more lenses 980, 990. The one or more light sources 934 can be similar to the one or more light sources 960 described above and can be pulsed, for example. Similarly, the one or more light sources 934 can include an infrared light source such as an infrared LED or another type of invisible light. However, other types of light sources can be used.

[0185] Fig.18 Another example imaging system 900 is shown that is configured to image a portion of the eye 210, such as the front portion of the eye (e.g., the cornea). Fig.18In the implementation shown, polarization selection is used to help control the path of light reflected from the eye. In particular, in various designs, the coupling optical element 944 is polarization selective. For example, light having a first polarization is transmitted through the coupling optical element 944, and light having a second, different polarization is turned into the waveguide 940 by the coupling optical element 944 to be coupled therein by TIR. Therefore, in various implementations, the eye 210 is illuminated with polarized light, or a polarizer (not shown) is disposed between the eye and the waveguide 940 so that the light from the eye incident on the waveguide is polarized. For example, the transmitter 934 can emit polarized light, or a polarizer can be disposed in front of the transmitter 934 so that the eye 210 is illuminated by polarized light. Therefore, in various designs, the polarization of the polarized light incident and / or reflected from the eye 210 received by the optical coupling element 944 can be a first polarization so that the light is directed to the reflector 996.

[0186] Likewise, in various implementations, the coupling optical element 944 (and / or the coupling-out optical element 952) is configured to transmit light having a first polarization state (e.g., a first linear, circular, or elliptical polarization state (e.g., p-polarization, left-handed circular or elliptical polarization, etc.)) and to redirect light having a second polarization state (e.g., a second linear, circular, or elliptical (e.g., s-polarization, right-handed circular or elliptical polarization, etc.)) into and / or out of the waveguide, respectively. In some implementations, the eye illuminator 934 may emit only or primarily the first polarization (e.g., p-polarization) or further include a polarization modifying element (e.g., a polarizer) configured to transmit only light of the first polarization state (e.g., p-polarization). Additionally, the coupling optical element 944 and / or the coupling-out optical element 952 may be configured to redirect light of the second polarization (e.g., s-polarization) into and / or out of the waveguide, respectively.

[0187] and Fig.17 Similar to the imaging system 900 shown, Fig.17 The imaging system 900 of the illustrated curved reflector 998 includes a curved transmissive optical element 996 having a wavelength-dependent reflective coating 998. The wavelength-dependent reflective coating 998 can be configured to reflect light of a specific wavelength or wavelength range. In some implementations, for example, the wavelength-dependent reflective coating 998 can be configured to reflect invisible light (e.g., infrared light) within a certain wavelength range, while the wavelength-dependent reflective coating 998 can be configured to transmit visible light. In some cases, the wavelength-dependent reflective coating 998 can be disposed on a surface of the curved transmissive optical element 996.

[0188] In various implementations, the curved transmissive optical element 996 is disposed on the far side of the eyepiece 950 in front of the eyepiece. Thus, the reflective optical element 996 is disposed between the eyepiece 950 and the environment in front of the user. Similarly, the eyepiece 950 is disposed between the reflective optical element 996 and the eye 210.

[0189] Thus, light from the eye 210 having a first polarization (e.g., p-polarization) is incident on the coupling optical element 944 and passes therethrough to the curved transmissive optical element 996. The imaging system 900 also includes a polarization modifying optical element 978, such as a retarder (e.g., a quarter-wavelength retarder). The retarder 978 is transmissive and imparts a quarter-wavelength delay to the light transmitted therethrough. The light is incident on and reflected from the curved transmissive optical element 996. The wavelength-dependent reflective coating 998 can be configured to reflect light of the wavelength reflected from the eye. Thus, the light is reflected from the curved surface of the curved transmissive optical element 996 and is collimated. The collimated light passes through the retarder 978 again, thereby imparting another quarter-wavelength delay to the light transmitted therethrough. The delay introduced on the two lights passing through the retarder (e.g., a fully retarded wave) rotates the polarization. Thus, the first polarization (e.g., p-polarization) transmitted through the polarization selective coupling optical element 944 on the first pass is converted into a second polarization (s-polarization) and is diverted into the waveguide 940 to be guided to the camera 920 by TIR. As described above, in various designs, the coupling optical element 944 is configured to divert collimated light reflected from the eye 210 into the waveguide 940 to be guided therein. Thus, the reflective optical element 996 can be configured to collimate light reflected from the eye 210, such as the front portion of the eye (e.g., cornea). Thus, the curved reflective optical element 996 can have a positive optical power. In particular, in various designs, the reflective optical element 994 can have a focal length equal to or substantially equal to the focal length from the reflective optical element 996 to the portion of the eye 210 to be imaged (e.g., cornea, iris, etc.). Example values ​​for the focal length can be, for example, 2 cm to 8 cm. In some implementations, the focal length is between 4 cm and 6 cm. In some designs, the focal length is approximately 5 cm.

[0190] In various designs, the reflective optical element 996 can include a curved surface configured to reflect light. In some cases, the curved surface can be spherical or rotationally symmetric. The reflective optical element 996 can include a concave mirror at least for some wavelengths and / or polarizations.

[0191] In various designs, the reflective optical element 996 includes a material that transmits visible light so that the user can see the environment in front of the wearer. Therefore, the wavelength-dependent reflective coating 998 disposed on the surface of the curved transmissive optical element 996 can transmit visible light or at least a specific wavelength of visible light. The curved transmissive optical element 996 can also be coated with an anti-reflective coating on the outer surface (e.g., the distal surface). The anti-reflective coating can be configured to reduce the reflection of red, green and / or blue light. However, the reflective optical element 994 can be configured to reflect a portion of the light scattered from the eye 210 to form an image of the eye. Therefore, the reflective optical element 996 can operate differently with respect to different light. For example, the reflective optical element 996 can operate differently with respect to light of different polarization states (and / or wavelengths). The reflective optical element 996 can be configured to transmit visible light and reflect infrared light.

[0192] like Fig.17 As shown, for example Fig.18 One or more light sources 934 of LEDs or emitters in the eye 210 can be arranged relative to the eye 210 to guide light thereon without being guided through the waveguide 940 and onto the eye 210 via TIR. Thus, in some implementations, the eyepiece 950 or waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210. In some designs, multiple such light sources 934 can be arranged in a pattern (e.g., a circular or annular pattern) near and / or around the eye. The one or more light sources 934 can be similar to the one or more light sources 960 described above, and can be pulsed, for example. Similarly, the one or more light sources 934 can include an infrared light source such as an infrared LED or another type of invisible light. In particular, in various implementations, the light source 934 can emit light reflected by a wavelength-dependent reflective coating 998 and / or a curved transmissive optical element 996. However, other types of light sources can be used.

[0193] Although the polarization selective coupling optical element 944 is configured to be polarization selective depending on the type of linear polarization incident thereon, other polarization selective coupling optical elements may be polarization selective for other types of polarization states, such as different types of circular or elliptical polarizations. The polarization selective coupling optical element 944 may, for example, be configured such that a first polarization, such as a first circular or elliptical polarization (e.g., left-handed polarization or LHP polarization), is transmitted through the polarization selective coupling optical element 944, such that a second polarization, such as a second circular or elliptical polarization (e.g., right-handed polarization or RHP), is diverted into the optical waveguide, and vice versa. Such a polarization selective coupling optical element 944 may include a liquid crystal, such as a cholesteric liquid crystal. Some examples of liquid crystal optical elements are discussed in the following sections: the section entitled “Cholesteric Liquid Crystal Mirror”; U.S. Publication No. 2018 / 0164627, filed on December 7, 2017, entitled “DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL”; U.S. Publication No. 2018 / 0239147, filed on February 22, 2018, entitled “DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR”; U.S. Publication No. 2018 / 0239147, filed on February 22, 2018, entitled “VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION (Variable Focus Virtual Image Device Based on Polarization Conversion)" U.S. Publication No. 2018 / 0239177; for all purposes, the entire contents of which are incorporated herein by reference.

[0194] A polarization modifying element such as a circular polarizer or a retarder may be disposed between the eye and the polarization selective coupling optical element 944 to convert the light reflected from the eye to a first polarization (e.g., LHP). The LHP light will pass through the polarization selective coupling optical element 944, reflect from the reflector 998, change polarization to RHP, and be diverted by the polarization selective coupling optical element 944 into the waveguide to reach the camera.

[0195] In some implementations, the reflector 996 can be polarization selective in its reflectivity, such that only light of a specific polarization state is reflected and / or light of a different polarization state is transmitted.Such an optical element can include a liquid crystal, such as a cholesteric liquid crystal. Examples of such optical elements are discussed in the following sections: the section entitled “Cholesteric Liquid Crystal Mirror”; U.S. Publication No. 2018 / 0164627, filed on December 7, 2017, entitled “DIFFRACTIVE DEVICESBASED ON CHOLESTERIC LIQUID CRYSTAL”; U.S. Publication No. 2018 / 0239147, filed on February 22, 2018, entitled “DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR”; U.S. Publication No. 2018 / 0239147, filed on February 22, 2018, entitled “VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION (Variable Focus Virtual Image Device Based on Polarization Conversion)" U.S. Publication No. 2018 / 0239177; for all purposes, the entire contents of which are incorporated herein by reference. Such an optical element can reflect light of a first polarization state such as a first circular or elliptical polarization state (left-handed circular or elliptical polarization) and transmit light of a second polarization state such as a second circular or elliptical polarization state (e.g., right-handed circular or elliptical polarization), and vice versa. In some embodiments, the liquid crystal is disposed on a curved surface of the reflector 996 so that when reflecting, the reflector has an optical power such as positive optical power. In various other implementations, the liquid crystal optical element can be flat or planar. For example, the liquid crystal can be disposed on a flat or planar substrate or layer. Despite being flat, optical power can be included in the liquid crystal optical element. Such an element can be referred to as a cholesteric liquid crystal reflective lens. Thus, light from the eye can be collimated and reflected to the coupling optical element 998. For example, the reflector can reflect light of a first polarization state (e.g., left-handed circular or elliptical) and transmit light of a second polarization state (e.g., right-handed circular or elliptical polarization). Thus, the eye 210 is illuminated with left-handed circularly polarized light, or light reflected from the eye is transmitted through a polarizer (e.g., a circular or elliptical polarizer) that transmits light having a first polarization (e.g., left-handed circular or elliptical polarization). The coupling optical element 944 can also be polarization selective and can transmit LHP light and redirect RHP light into the waveguide. The LHP light from the eye passes through the coupling optical element 944. This transmitted LHP light is also incident on and reflected from the wavelength selective liquid crystal reflector 996.In some designs, the wavelength selective liquid crystal reflector 996 converts the first polarization state (e.g., LHP) to a second polarization state (e.g., RHP) upon reflection. The light in the second polarization state (e.g., RHP) is directed to the coupling optical element 944, which redirects the light in the second polarization state (RHP) into the waveguide 940 and to the camera 920.

[0196] In some designs, the coupling optical element 944 does not include a liquid crystal grating, but instead includes, for example, a surface relief diffraction grating or a holographic grating. As described above, these coupling optical elements 944 that do not include cholesteric liquid crystals may also include volume diffraction or holographic optical elements or gratings.

[0197] Therefore, light scattered from the eye is reflected by the reflective optical element 996 back to the waveguide 940 to be coupled into the waveguide through the coupling element 944. However, conversely, a portion of the unpolarized light from the environment in front of the wearer corresponding to the second polarization state (e.g., RHP) will be transmitted through the reflective optical element 996. Thus, the wearer can see the object through the reflective optical element 996.

[0198] However, in various designs, the reflective optical element 996 will have negligible power when transmitting. For example, the reflective optical element 996 can have curved surfaces of the same curvature on both sides of the optical element so that the overall power of the optical element for light transmitted therethrough is negligible.

[0199] As described above, in various implementations, the reflective optical element 996 includes a cholesteric liquid crystal reflective lens, a cholesteric liquid crystal reflective element, such as discussed in the following sections: the section entitled “Cholesteric Liquid Crystal Mirror”; U.S. Publication No. 2018 / 0164627, filed on December 7, 2017, entitled “DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL”; U.S. Publication No. 2018 / 0239147, filed on February 22, 2018, entitled “DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR”; U.S. Publication No. 2018 / 0239147, filed on February 22, 2018, entitled “VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION (Variable Focus Virtual Image Device Based on Polarization Conversion)"; for all purposes, the entire contents of which are incorporated herein by reference. Such an optical element can operate with respect to a specific wavelength or wavelength range. Thus, light such as infrared light reflected from the eye can be acted upon by a cholesteric liquid crystal reflective element. However, light that is not within this wavelength range, such as visible light from the environment, can pass through the cholesteric liquid crystal reflective element without being operated by the cholesteric liquid crystal reflective element. Therefore, the cholesteric liquid crystal reflective element has a negligible focal length for the visible light from the environment passing therethrough.

[0200] As described above, in some implementations, the illumination source 960 couples light into the waveguide 940, which is then transferred out of the waveguide to illuminate the eye 210. In such embodiments, the coupling optical element 944 can be polarization selective. For example, the coupling optical element 944 can transmit a first polarization (p-polarization) and transmit a second polarization (s-polarization).

[0201] Thus, if light from illumination source 906 propagates through waveguide 940 and is turned by coupling optical element 944, the illumination will be s-polarized. A polarization modifying optical element (e.g., a quarter-wave retarder) may be disposed between waveguide 940 and eye 210 to cause rotation of the polarized light reflected from the eye. Light from light source 960 that is reflected from eye 210 will pass through the quarter-wave retarder twice, and as a result, the s-polarized light that is emitted from the waveguide by coupling element 944 to illuminate the eye will be converted to p-polarized light.

[0202] This p-polarized light will be transmitted through coupling optical element 944 and the waveguide and incident on reflective optical element 996 .

[0203] The imaging system 900 may further include a second polarization modification element 978, which may include, for example, a retarder or a wave plate as described above. The retarder may include, for example, a quarter-wave retarder. The second polarization modification element 978 may be disposed at the distal end of the waveguide 940 between the waveguide and the reflector 996. The second polarization modification element 978 may also be disposed between the coupling element light 944 and the reflector 996. The light from the eye 210 (p-polarization) transmitted through the coupling element 944 passes through the second polarization modification element 978 and is converted into circular polarization. If the reflector 996 reflects circularly polarized light, the light will be reflected back to the waveguide 940 after passing through the polarization modification element 978 again. Passing twice through the polarization modification element (e.g., a quarter-wave retarder) 978 will convert the light into s-polarized light, which will be diverted by the coupling element 944 into the waveguide to be guided therein and reach the camera (not shown).

[0204] like Fig.18 As shown, the light 988 reflected from the eye 210 is divergent. This light is incident on a reflector 996 that is curved or has a positive optical power and can be collimated thereby. The coupling optical element 944 configured to redirect the collimation into the waveguide 940 will thus direct this collimated light from the curved reflective optical element 996 toward the imaging device 920 (not shown). Therefore, the light reflected from the eye 210 that is collimated by the curved reflective optical element 996 is coupled into the waveguide 940 and directed therein toward the outcoupling optical element 952. The outcoupling optical element 952 can be configured to direct the light out of the eyepiece 950 and to a camera (not shown).

[0205] There can be a variety of variations in the configuration of the imaging system. Different types of reflectors 996 and coupling elements 944 can be used. Reflectors 996 and coupling elements 944 can, for example, be configured to operate with respect to linearly polarized light or circularly or elliptically polarized light. As discussed, reflector 996 has optical power. Reflector 996 and coupling element 944 can include a cholesteric liquid crystal grating reflector and / or a lens with no optical power. A polarization modifying element 978 such as a retarder can be included between coupling element 944 and the reflector and / or between coupling element 944 and the eye. In some embodiments, a polarizer such as a circular polarizer or a linear polarizer can be disposed between the eye and coupling element 944. If, for example, non-polarized light is reflected from the eye, a polarizer (e.g., a circular polarizer or a linear polarizer) can be disposed between the eye and coupling element 944. In some such cases, coupling element 944 is polarization selective.

[0206] In such Fig.17 and 18 In the configuration shown, where light reflected from the eye passes through the waveguide 940 to the curved reflective optical element 996 to be collimated and redirected back to the waveguide, background noise is introduced. This background noise is generated by the light that originally passed from the eye through the coupling optical element 944. As described above, the coupling optical element 944 can be configured to redirect collimated light into the waveguide 940 to be guided therein and reach the camera 920 that forms an image. However, the coupling optical element 944 will redirect some of the non-collimated light incident thereon. Therefore, when initially passing through the coupling optical element 944 and the waveguide 940 to reach the curved reflective optical element 996, some of the non-collimated (divergent) light reflected from the eye will be coupled into the waveguide through the coupling optical element 944 and contribute background noise to the image of the eye formed by the camera 920. This noise will be superimposed on the image formed by the collimated light that is retroreflected by the curved reflective optical element 996, which is coupled into the waveguide by the coupling optical element 944 and guided therein and reaches the camera 920.

[0207] In some designs, this noise can be subtracted from the image. The process of subtracting the noise from the signal may involve (a) measuring the amount of light coupled by the coupling optical element 944 when it initially passes through the coupling optical element 944 to the curved reflective optical element 996, which is redirected and reaches the camera 920 (referred to as N) and (b) measuring the total signal at the camera 920 when the light passes through the coupling optical element 944 and the waveguide 940 to the curved reflective optical element 996, is collimated and reflected back to the coupling optical element and is redirected to the camera 920. This total signal will also include some noise N because non-collimated light reflected from the eye will have passed through the coupling optical element 944 to the curved reflective optical element 996, so some of the non-collimated light will be redirected by the coupling optical element 944 to the camera 920. If the noise N can be measured separately from the total signal T containing the noise superimposed on the eye image, the noise N can be subtracted from the total signal T, as shown in the following equation:

[0208] I=TN

[0209] Where I represents the image with the noise component N removed.

[0210] The above two measurements (a) and (b) can be obtained in various ways. For example, Fig.19 As shown, a shutter 936 can be disposed between the curved reflective optical element 996 and the waveguide 940 and the coupling optical element 944. The shutter 936 can be configured to block light when the shutter is in a first state and transmit light when the shutter is in a second state. The shutter 936 can include, for example, a liquid crystal shutter.

[0211] Thus, when the shutter 936 is in a first state where light reflected from the eye 210 is incident on the coupling optical element 944 and passes through it toward the curved reflective optical element 996, a noise component N can be measured, however, the noise component N can be prevented from reaching the curved reflective optical element by the closed shutter. As described above, some of the light reflected from the eye 210, although primarily uncollimated, does couple into the coupling optical element 944 and is diverted into the waveguide and guided therein to the camera 920. As described above, this light image does not contribute to the formation, but does generate background noise. When the shutter 936 is closed, the camera 920 can record this noise N.

[0212] When the shutter 936 is in a second state in which the shutter is open, a total signal T including both the noise N and the image can be measured. The light reflected from the eye 210 is again incident on the coupling optical element 944. Some of this light reflected from the eye 210, although primarily uncollimated, is coupled into the coupling optical element 944 and diverted into the waveguide and directed therein to the camera 920. However, most of this light reflected from the eye 210 passes through the coupling optical element 944, through the open shutter 936, and to the curved reflective optical element 996. The curved reflective optical element 996 collimates at least a portion of the light and reflects it back to the coupling optical element 944, which diverts the collimated light into the waveguide 920 and directs it to the camera 920 to form an image of the eye 210. The camera 920 can capture an image of the eye 210.

[0213] Processing electronics (e.g., processing electronics 140) in communication with camera 920 can receive the noise component N measured when shutter 936 is in a first closed state and the total signal T measured when shutter 936 is in a second open state and can subtract the two (TN). In this way, noise N contributed by non-collimated light reflected from eye 210 that is coupled into coupling optical element 944 when initially passing through coupling optical element 944 can be subtracted from the total image signal T. The processing electronics can communicate with camera 920 via wired electronic signals. Additionally or alternatively, the processing electronics can communicate with camera 920 using one or more remote receivers. The processing electronics can reside remotely (e.g., a cloud computing device, a remote server, etc.).

[0214] Other ways of performing the measurements of (a) and (b) to obtain N and T and subtracting N from T may be employed. For example, if the curved reflective optical element 996 is wavelength selective, such as Fig.18As shown, the eye may be illuminated with different wavelengths of light at different times. For example, to perform measurement (a) and quantify the noise N, the eye may be illuminated with a wavelength that is not reflected by the curved reflective optical element 996. However, to perform measurement (b) and quantify the total signal T, the eye may be illuminated with a wavelength that is reflected by the curved reflective optical element 996. The noise N may then be subtracted from the total T as described above (e.g., TN).

[0215] Figures 20A-20E An example imaging system 900 is shown that is configured to measure and subtract a noise component N using wavelength modulation as described above. Figures 20A-20E The imaging system 900 in the embodiment includes a wavelength selective curved transmissive optical element 996 (e.g., as described above with reference to Fig.17 and Fig.18 996 has a wavelength-dependent reflective coating 998 on its curved surface. The imaging system 900 may also include one or more light sources or illumination sources (not shown) configured to illuminate the eye 210. The one or more light sources may be configured to emit infrared light. However, the one or more light sources may be configured to emit light of different colors or wavelengths at different times. Such wavelength modulation may enable N to be measured separately so that N can be subtracted from the total signal T.

[0216] In various implementations, for example, one or more illumination sources 960, 934 may be configured to emit one or more wavelengths λ reflected by the curved reflective optical element in a first state. 反射 , and in the second state emits one or more wavelengths λ that are not reflected 未反射 In the second state, no more than a negligible amount of wavelength λ is emitted 反射 , which wavelength is reflected by the curved reflective optical element. Similarly, in the first state, no more than a negligible amount of the unreflected wavelength λ is emitted 未反射 .

[0217] In some examples, the reflected wavelength λ 反射 It can be between about 800nm ​​and 950nm. 反射 It can be between about 835nm and 915nm. 反射 It can be between about 840nm and 870nm. In some designs, the reflection wavelength λ 反射 About 850 nm. Light emission 928 from one or more light sources 960 can illuminate the eye.

[0218] like Fig. 20B As shown, the wavelength λ which is not reflected by the curved reflective optical element 944 未反射988 (and no more than a negligible amount of light reflected by the curved optical element λ 反射 ) reflects off a portion of the eye 210 (eg, the cornea). Because this light includes wavelengths λ that are not reflected by the curved reflective optical element 944 未反射 , so light ray 916 is shown propagating through curved reflective optical element 996 to the environment in front of the user.

[0219] Although the light 988 incident on the coupling optical element 944 is not collimated, the coupling optical element couples at least some of the light 914 into the waveguide 940 to be directed to the camera 920. Therefore, the camera 920 can capture an image (image #1) corresponding to the noise component N, which is produced by the non-collimated light that is diverted by the coupling optical element 944 when it initially passes through to the curved reflective optical element 996. This image (image #1) is background noise and is indeed not a recognizable image of the eye. The processing electronics 140 is shown as receiving this first image (image #1).

[0220] exist Figures 20C-20E In the embodiment, an illumination source (not shown) emits one or more wavelengths λ which are reflected by the curved reflective optical element. 反射 , and does not exceed a negligible amount of wavelength λ 未反射 Not reflected. The wavelength λ 反射 It may be, for example, 850 nm.

[0221] like Fig. 20C As shown, some of the light 988 reflected from the eye 210 that is incident on the coupling optical element 944 when passing through the coupling optical element 944 for the first time is coupled into the waveguide 940 by the coupling optical element 944 (e.g., Fig. 20B ) and is directed toward camera 920. In addition, the wavelength λ is selectively reflected 反射 The light-bent transmissive optical element 996 reflects and collimates the incoupled light 918 reflected from the eye 210 and incident on the curved transmissive optical element. Fig.20E As shown in , coupling optical element 944 turns the collimated reflected light and couples it into waveguide 940 and toward camera 920. Fig.20E Two components reaching the camera 920 are shown, light 988 reflected from the eye 210 incident on the coupling optics 944 upon a first pass through the coupling optics 944 (which is coupled into the waveguide 940 by the coupling optics), and light reflected and collimated by the curved transmissive optics 996 coupled into the waveguide by the coupling optics. The camera 920 may capture an image (image #2) corresponding to this total image component T. The processing electronics 140 is shown receiving this second image (image #2).

[0222] As described above, the processing electronics may subtract the noise TN from the image. In this example, image #1 may be subtracted from image #2. Thus, the processing electronics 140 may be configured to modify the second image based on the first image. However, other approaches are possible. For example, the processing electronics 140 may be configured to create a new image that represents a version of the second image with reduced optical noise. An implementation for subtracting noise from an image may be used in the implementations described above. For example, Fig.10 , Figures 11A-11E and / or Figures 12A-12E The illustrated implementation may include a shutter 936 and / or a curved transmissive optical element 996 having a wavelength-dependent reflective coating 998 that is configured to selectively reflect the uncoupled light 912 and direct the light to an imaging device 920 .

[0223] As described above, Image #1 is obtained when the illumination has one or more wavelengths λ that are not reflected by the curved reflective optical element. 未反射 of light, and not exceeding a negligible amount of wavelength λ 反射 Image #2 is obtained when the illumination has one or more wavelengths λ that are reflected by the curved reflective optical element. 反射 of light, and not exceeding a negligible amount of wavelength λ 未反射 Thus, one or more illumination sources 960, 934 may be configured to modulate wavelengths. For example, in some designs, one or more illumination sources 960, 934 may include a first illumination source configured to output one or more wavelengths λ that are not reflected by the curved reflective optical element. 未反射 , and does not exceed a negligible amount of wavelength λ 反射 The one or more illumination sources may further include a second illumination source configured to output one or more wavelengths λ reflected by the curved reflective optical element. 反射 , and does not exceed a negligible amount of wavelength λ 未反射 The intensity of the first and second illumination sources can be alternately increased and decreased, turned on and off, attenuated and not attenuated, passed and blocked to provide modulation of the wavelength of light illuminating the eye. For example, during a first time interval, the first illumination source can be blocked while the second illumination source is not blocked. During a subsequent second time interval, the second illumination source can be blocked while the first illumination source is not blocked. This process can be repeated to provide modulation of the wavelength of light illuminating the eye. In other designs, the wavelength of the light source can be tuned and detuned to cause the wavelength to vary between λ and λ. 反射 and λ 未反射 Other arrangements are possible.

[0224] As described above, the imaging system 900 can be included in a head-mounted display such as an augmented reality display, which also provides the ability to image the eye by collecting light with the eyepiece 950. Such an imaging system 900 can be used for eye tracking. Multiple images of the retina of the eye or the front of the eye can be obtained. The movement and / or repositioning of the eye can be determined from these images to track the position and / or orientation of the eye. These imaging systems can also be used for bio-imaging and / or for identifying users. For example, an image of the user's eye (e.g., retina or iris) can be obtained and recorded. A subsequent image of the wearer's eye (e.g., retina or iris) can be obtained at a later time. The two images can be compared to determine whether the wearer in the subsequent instance is the wearer in the first instance. However, other uses of the imaging system are possible.

[0225] Although the illumination system may be described above as being waveguide-based and including one or more waveguides, other types of light steering optical elements may be used in place of the waveguides. Such light steering optical elements may include steering features to direct light out of the light steering optical element, for example, onto a spatial light modulator. Thus, in any of the examples described herein and in any of the examples below, any reference to a waveguide may be replaced with a light steering optical element in place of the waveguide. Such a light steering optical element may include, for example, a polarizing beam splitter, such as a polarizing beam splitting prism.

[0226] As described above, the systems described herein may allow for the collection of biometric data and / or biometric identification. For example, an eye or a portion thereof (e.g., a retina) may be imaged to provide such biometric data and / or biometric identification. When a user (presumably the same user) wears the head-mounted display system, images of the eye, such as the retina, may be obtained at different times. A collection of such images may be recorded, for example, in a database. These images may be analyzed to collect biometric data. Such biometric data may be useful for monitoring the health or medical condition of the user. Different medical parameters may be monitored by imaging a patient (e.g., the patient's eye (e.g., retina)). Medical parameters may be recorded and compared to subsequent measurements obtained while the user is wearing the head-mounted display system.

[0227] In addition, if a person starts wearing a head-mounted display system and an image of the user's eyes is captured that does not match the image stored in the database, it can be concluded that the person currently wearing the head-mounted display system is different from the previous user. This is useful in determining whether the target user is wearing a headset or the headset is worn by a new user. Such features can allow certain medical, safety, and / or convenient applications or functions. For example, the head-mounted display can be configured to identify the wearer based on the characteristics of the wearer's eyes. For example, the system can be configured to determine an individual based on the wearer's retina (e.g., blood vessels), cornea, or other eye features. For example, in some implementations, a series of markers can be determined for a particular wearer. Based on the series of markers, the system may be able to determine that the previous user is wearing a headset, or, alternatively, that another user is wearing a headset. The markers may include the shape or center of the user's cornea, the configuration of the blood vessels in the user's retina, the intensity and / or position of the reflection of light from the cornea, the shape of the aspect of the eye, and / or any other biomarker. In some implementations, a confusion matrix can be determined. As discussed above, for example, in developing retinal maps using virtual / fixation targets located at various locations (see, e.g., Fig. 13B ), a system can have a user look at or eye gesture in a set of predetermined directions and develop a matrix of features of the eye or part of the eye (e.g., cornea, retina, etc.) associated with each direction or eye gesture. Using such a matrix, the system can determine the identity of the individual. Other approaches are possible.

[0228] Similarly, as described above, multiple configurations of the system are possible. For example, Fig.21 An example eyepiece 900 is shown that can be used to project light into a user's eye while imaging the user's eye. The eyepiece 900 shown includes a coupling optical element 2104, a light distribution element 2108 and a light consolidating element 2116, and an outcoupling optical element 2120 located on the opposite side of the coupling optical element 2112. Each of these optical elements can be disposed within or on a waveguide 2102. The waveguide 2102 can correspond to, for example, one of the waveguides 670, 680, 690 described herein (e.g., see Figures 9A-9C ). The coupling-in optical element 2104 may correspond to one of the coupling-in optical elements 700, 710, 720 described herein and / or to the coupling-in optical element 942 (see, e.g. Fig.10 ) and can be configured to inject image content from a projector into the waveguide and / or inject illumination from the light source 960. The light distribution element 2108 can correspond to one of the light distribution elements 730, 740, 750 described herein (e.g., see Figures 9A-9C And can be used to distribute light in a given direction and redirect light from the coupling optical element 2104 to the coupling optical element 2112. The coupling optical element 2112 can correspond to the coupling optical element 944 described herein (e.g., see Fig.10 In some designs, the coupling optical element 2112 includes the functionality described herein with respect to the outcoupling optical elements 800, 810, 820 (see Figures 9A-9C ). The light combining element 2116 can be configured to receive the lateral spatial extent of the light from the coupling optical element 2112 and redirect the light to the outcoupling optical element 2120. The outcoupling optical element 2120 can correspond to the outcoupling optical element 952 described herein (e.g., see Fig.10 ).

[0229] The coupling optical element 2104 can be disposed within or on the waveguide 2102 to receive light, such as from a projector (e.g., image projector 930) and / or an illuminator (e.g., light source 960). The light can be delivered to the associated light distribution optical element 2108 via the waveguide 2102. Any of the coupling optical element 2104, the light distribution optical element 2108, or the coupling optical element 2112 can be disposed on a major surface of the waveguide (e.g., on the top or bottom surface) or within the waveguide. Similarly, any one or combination of the light combining element 2116 and / or the coupling optical element 2120 can be disposed on a major surface of the waveguide 2102 (e.g., the top surface or both major surfaces) or within the waveguide.

[0230] The coupling optical element 2112 can receive light from the light distribution element 2108 (e.g., via TIR) and expand the light to enter the user's eyes. Thus, the coupling optical element 2112 can be disposed in front of the user's eyes and project image content therein. Additionally or alternatively, the coupling optical element 2112 can be configured to provide illumination light onto and / or into the user's eyes.

[0231] Light reflected from the eye (e.g., illumination light from an illumination source) can be reflected and captured by the coupling optical element 2112. Thus, in some embodiments, the coupling optical element 2112 can be used to couple out light received from the light distribution element 2108 and couple in light received from the eye into the waveguide 2102.

[0232] In some embodiments, the coupling optical element 2112 can include one or more diffractive optical elements (DOEs) such that the coupling optical element 2112 has dual functionality. A first DOE (e.g., grating, holographic region) can also be configured to couple light out, and a second DOE can be configured to couple reflected light from the eye into the waveguide 2102. In some embodiments, the first and second DOEs are superimposed within the waveguide 2102 (e.g., occupy the same or approximately the same volume).

[0233] Optionally, in some embodiments, the coupling optical element 2112 includes at least two DOEs stacked on top of or in front of another DOE. Fig.21 , the first DOE of the coupling optical element 2112 can be arranged above, and the second diffraction element can be arranged below the first DOE. In other implementations, the order of each DOE can be reversed.

[0234] Cholesteric Liquid Crystal Mirror

[0235] Some liquid crystals are in a phase called a chiral phase or a cholesteric phase. In the cholesteric phase, the liquid crystal can exhibit a twist of the molecules along an axis perpendicular to the director, wherein the molecular axis is parallel to the director. As described herein, a cholesteric liquid crystal (CLC) layer includes a plurality of liquid crystal molecules in a cholesteric phase, which extend in a direction, for example, perpendicular to the direction of the director (e.g., the layer depth direction), and rotate or twist in sequence, for example, in a clockwise or counterclockwise rotation direction. The director of the liquid crystal molecule in the chiral structure can be characterized as a helix having a pitch (p), which corresponds to a length in the layer depth direction, which corresponds to the net rotation angle of the liquid crystal molecule in the chiral structure through a full rotation along a first rotation direction. In other words, the pitch refers to the distance over which the liquid crystal molecule undergoes a full 360° twist. Liquid crystals that exhibit chirality may also be described as having a twist angle or rotation angle (φ), which may refer, for example, to a relative azimuthal rotation between successive liquid crystal molecules in a layer normal direction, and may have a net twist angle or net rotation angle, which may refer, for example, to a relative azimuthal rotation between the topmost liquid crystal molecule and the bottommost liquid crystal molecule across a specified length (e.g., the length of the chiral structure or the thickness of the liquid crystal layer). As described herein, a chiral structure refers to a plurality of liquid crystal molecules in a cholesteric phase that extend in a direction, for example, perpendicular to the direction of the director (e.g., the layer depth direction) and are continuously rotated or twisted in a rotation direction, for example, clockwise or counterclockwise. In one aspect, the director of the liquid crystal molecules in the chiral structure may be characterized as a helix having a pitch.

[0236] Fig. 22A cross-sectional side view of a cholesteric liquid crystal (CLC) layer 1004 including a plurality of uniform chiral structures according to an embodiment is shown. In the CLC layer 1004, adjacent chiral structures in a lateral direction (e.g., x-direction) have similarly arranged liquid crystal molecules. In the illustrated embodiment, the chiral structures 1012-1, 1012-2, ... 1012-i are similarly configured so that liquid crystal molecules of different chiral structures located at approximately the same depth, e.g., liquid crystal molecules closest to the light incident surface 1004S, have the same rotation angle, the continuous rotation angle of continuous liquid crystal molecules located at approximately the same depth, and the net rotation angle of the liquid crystal molecules of each chiral structure.

[0237] The CLC 1004 includes a CLC layer 1008 including liquid crystal molecules arranged into a plurality of chiral structures 1012-1, 1012-2, ... 1012-i, wherein each chiral structure includes a plurality of liquid crystal molecules, wherein i is any suitable integer greater than 2. In operation, when incident light having a combination of a left-handed circularly polarized beam and a right-handed circularly polarized beam is incident on a surface 1004S of the CLC layer 1008 by Bragg reflection, light having one of the circular polarization handednesses is reflected by the CLC layer 1004, while light having the opposite polarization handedness is transmitted through the CLC layer 1008 without substantial interference. As described herein and throughout the disclosure, handedness is defined as viewed along the propagation direction. According to an embodiment, the incident light is reflected when the polarization direction or handedness of the light beam 1016-L, 1016-R matches such that it has the same rotation direction as the liquid crystal molecules of the chiral structures 1012-1, 1012-2, ... 1012-i. As shown, incident on the surface 1004S is a light beam 1016-L having left-handed circular polarization and a light beam 1016-R having right-handed circular polarization. In the illustrated embodiment, the liquid crystal molecules of the chiral structures 1012-1, 1012-2, ... 1012-i continuously rotate in a clockwise direction (i.e., positive x direction) in the direction of travel of the incident light beams 1016-L, 1016-R, which is the same rotation direction as the light beam 1016-R having right-handed circular polarization. As a result, the light beam 1016-R having right-handed circular polarization is substantially reflected, while the light beam 1016-L having left-handed circular polarization is substantially transmitted through the CLC layer 1004.

[0238] As described above, the CLC layer can be configured as a Bragg reflector by matching the handedness of polarization of incident elliptically or circularly polarized light with the rotation direction of the liquid crystal molecules of the chiral structure of the CLC layer. In addition, one or more CLC layers with different pitches can be configured as wavelength-selective Bragg reflectors with high bandwidth. Based on the concepts described herein with respect to various embodiments, the CLC layer can be configured as an off-axis or on-axis mirror that is configured to selectively reflect a first wavelength range, such as infrared wavelengths (e.g., near infrared), while transmitting another wavelength range, such as visible wavelengths.

[0239] Fig.23 An example of an eye tracking system 2300 employing a cholesteric liquid crystal reflector (CLCR), such as a wavelength selective CLCR 1150 configured to image a viewer's eye 302, is shown in accordance with various embodiments. Fig. 22 Unlike the CLC layer 1004 described above, the chiral structures adjacent in the lateral direction (e.g., the x direction) in the wavelength selective CLCR 1150 have liquid crystal molecules arranged differently. That is, the chiral structures are configured so that liquid crystal molecules of different chiral structures located at approximately the same depth (e.g., liquid crystal molecules closest to the light incident surface 1004S) have different rotation angles. As a result, light incident on the CLCR 1150 is rotated at an angle (θ) relative to the layer depth direction. R ) reflections, as further described below in the context of eye tracking system 2300.

[0240] Eye tracking can be a useful feature in an interactive visual or control system, including a wearable display system for virtual / augmented / mixed reality display applications and other applications described elsewhere in this specification. In order to achieve effective eye tracking, it may be desirable to obtain an image of the eye 302 at a low angle, for which it may be necessary to place the eye tracking camera 702b near the center of the viewer's eye. However, this position of the camera 702b may interfere with the user's viewing. Optionally, the eye tracking camera 702b can be set at a lower position or side. However, since the eye image is captured at a steeper angle, this position of the camera may increase the difficulty of obtaining robust and accurate eye tracking. By configuring the CLCR 1150 to selectively reflect infrared (IR) light 2308 (e.g., having a wavelength of 850nm) from the eye 302 while transmitting visible light 2304 from the world, the camera 702b can be placed away from the user's viewing while capturing normal or low-angle eye images. Such a configuration will not interfere with the user's viewing because visible light will not be reflected. As shown, the same CLCR 1150 can also be configured as an IR illumination source 2320 by reflecting IR light from an IR source, such as an IR LED, into the eye 302. The low viewing angle of the IR illuminator can result in less occlusion, such as from eyelashes, and this configuration allows for more robust detection of specular reflections, which can be a useful feature in modern eye tracking systems.

[0241] Still reference Fig.23According to various embodiments, the CLCR 1150 includes one or more cholesteric liquid crystal (CLC) layers, each of which includes a plurality of chiral structures, wherein each chiral structure includes a plurality of liquid crystal molecules extending in a layer depth direction (e.g., z direction) and continuously rotating along a first rotation direction, as described above. The arrangement of the liquid crystal molecules of the chiral structure changes periodically in a lateral direction perpendicular to the layer depth direction, so that the one or more CLC layers are configured to substantially Bragg reflect a first incident light having a first wavelength (λ1) while substantially transmitting a second incident light having a second wavelength (λ2). As described above, when viewed in the layer depth direction, each of the one or more CLC layers is configured to substantially Bragg reflect the first and second incident lights having an elliptically or circularly polarized polarization that matches the first rotation direction, and is configured to substantially transmit the first and second incident lights having an elliptically or circularly polarized polarization that is opposite to the first rotation direction. According to an embodiment, the arrangement of the liquid crystal molecules that periodically vary in the lateral direction is arranged to have a period in the lateral direction such that a ratio between the first wavelength and the period is between about 0.5 and about 2.0. According to an embodiment, the first wavelength is in the near infrared range between about 600 nm and about 1.4 μm, for example about 850 nm, and the second wavelength is in the visible range having one or more colors as described elsewhere in the specification. According to various embodiments, the liquid crystal molecules of the chiral structure are pre-tilted relative to a direction normal to the layer depth direction. As configured, one or more CLC layers are configured such that a first incident light is incident at an angle (θ) relative to the layer depth direction (z direction). R ) reflection, the angle exceeding about 50°, about 60°, about 70° or about 80° relative to the depth direction of the layer.

[0242] Configured as such, the wavelength selective CLCR 1150 includes one or more cholesteric liquid crystal (CLC) layers, each cholesteric liquid crystal layer including a plurality of liquid crystal molecules extending in a layer depth direction and continuously rotating in a first rotation direction, wherein the arrangement of the liquid crystal molecules of a chiral structure periodically changes in a lateral direction perpendicular to the layer depth direction, such that the one or more CLC layers are configured to substantially Bragg reflect a first incident light having a first wavelength, such as an IR wavelength, while substantially transmitting a second incident light having a second wavelength, such as a visible wavelength.

[0243] Similar liquid crystal layers and structures can be used in the above combination Figure 17-20E The reflector 996 and coating 998 are shown. The coating 998 may include, for example, a liquid crystal coating and may be wavelength and / or polarization selective in certain implementations. However, other types of coatings 998 and reflectors 996 may be employed.

[0244] In the foregoing description, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. Accordingly, the description and drawings should be regarded in an illustrative rather than a restrictive sense.

[0245] In fact, it should be understood that the systems and methods of the present disclosure each have several innovative aspects, any single aspect of which is not solely responsible for or necessary for the desired properties disclosed herein. The various features and processes described above can be used independently of each other or can be used in combination in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure.

[0246] Certain features described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although features may be described above as being performed in certain combinations, even initially exemplified as such, in some cases, one or more features from the exemplified combinations can be deleted from the combination, and the exemplified combinations may involve sub-combinations or variations of sub-combinations. No single feature or group of features is necessary or indispensable for every embodiment.

[0247] Conditional terms used herein, such as (among other things) "can", "can", "may", "can", "for example", etc. are generally intended to express that certain embodiments include and other embodiments do not include certain features, elements and / or steps, unless otherwise specifically stated or otherwise understood in the context. Therefore, such conditional terms are generally not intended to imply that features, elements and / or steps are necessary for one or more embodiments in any way, or that one or more embodiments necessarily include logic for determining whether these features, elements and / or steps are included in or will be performed in any specific embodiment with or without programmer input or prompts. The terms "include", "comprise", "have", etc. are synonyms and are used inclusively in an open-ended manner, and do not exclude additional elements, features, actions, operations, etc. In addition, the term "or" is used with its inclusive meaning (rather than its exclusive meaning), so when used, for example, to connect a list of elements, the term "or" means one, some or all of the elements in the list. In addition, the articles "one", "an" and "said" used in this application and the attached examples should be interpreted as meaning "one or more" or "at least one", unless otherwise specifically stated. Similarly, although the operation can be depicted in a specific order in the accompanying drawings, it should be appreciated that such operation does not need to be performed in the specific order or in a sequential order, or all illustrated operations are performed to achieve the desired result. In addition, the accompanying drawings can schematically depict one or more example processes in the form of a flow chart. However, other operations not shown can be incorporated into the schematically illustrated example methods and processes. For example, one or more additional operations can be performed before, after, simultaneously or during any illustrated operation. In addition, in other embodiments, the operation can be rearranged or reordered. In some cases, multitasking and parallel processing can have advantages. In addition, the separation of various system components described in the above embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems can generally be integrated together in a single software product or packaged into multiple software products. In addition, other embodiments are within the scope of the following examples. In some cases, the actions listed in the examples can be performed in different orders, and still achieve the desired result.

[0248] Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the present disclosure, the principles and novel features disclosed herein. Various example systems and methods are provided below.

[0249] Example

[0250] Example Part I

[0251] 1. A head mounted display system configured to project light into a user's eyes to display augmented reality image content in the user's field of view, the head mounted display system comprising:

[0252] a frame configured to be supported on a head of the user;

[0253] an image projector configured to project an image into the eyes of the user to display image content in the field of view of the user;

[0254] camera;

[0255] at least one waveguide;

[0256] at least one coupling optical element configured such that light is coupled into the waveguide and guided therein; and

[0257] at least one outcoupling element configured to couple light guided in the waveguide out of the waveguide and guide the light to the camera,

[0258] Wherein, the camera is arranged in the optical path relative to the at least one outcoupling optical element to receive at least a portion of the light, the at least a portion of the light is coupled into the waveguide via the coupling element and guided therein, and is coupled out of the waveguide through the outcoupling coupling element, so that an image can be captured by the camera.

[0259] 2. A system according to Example 1, wherein the at least one coupling optical element is configured such that light from the environment in front of the user wearing the head-mounted display system is coupled into and guided into the at least one waveguide so that an image of the environment can be captured by the camera.

[0260] 3. A system according to any of the above examples, wherein the at least one coupling optical element is configured so that light reflected from the eye of the user wearing the head-mounted display system is coupled into and guided in the at least one waveguide so that an image of the eye can be captured by the camera.

[0261] 4. A system according to any of the above examples, wherein the at least one coupling optical element is configured so that light reflected from the eye of the user wearing the head-mounted display system is coupled into and guided in the waveguide so that an image of the eye can be captured by the camera, and the system is configured to image the front of the eye.

[0262] 5. A system according to any of the above examples, wherein the at least one coupling optical element is configured so that light reflected from the eye of the user wearing the head-mounted display system is coupled into and guided in the waveguide so that an image of the eye can be captured by the camera, and the system is configured to image the corneal surface of the eye.

[0263] 6. A system according to any of the above examples, wherein the at least one coupling optical element is configured so that light reflected from the eye of the user wearing the head-mounted display system is coupled into and guided in the waveguide so that an image of the eye can be captured by the camera, and the system is configured to image the retina of the user's eye.

[0264] 7. The system according to any of the above examples further includes an eyepiece disposed on the frame, the eyepiece being configured to direct light into the user's eyes to display augmented reality image content to the user's field of view, at least a portion of the eyepiece being transparent and disposed at a position in front of the user's eyes when the user wears the head-mounted display, so that the transparent portion transmits light from the environment in front of the user to the user's eyes to provide a view of the environment in front of the user.

[0265] 8. A system as described in any of Examples 7-8, wherein the eyepiece is configured to receive light from the image projector and direct the light into the user's eyes to display augmented reality image content to the user's field of view.

[0266] 9. The system of any of Examples 7-9, wherein the eyepiece comprises the at least one waveguide.

[0267] 10. The system of Example 7, wherein the image projector is configured to direct light into an edge of the eyepiece.

[0268] 11. A system according to Example 9 or 10, wherein the image projector is configured to direct light into an edge of the at least one waveguide.

[0269] 12. The system of any of the above examples further comprises at least one coupling optical element configured to couple light from the image projector into the at least one waveguide so as to guide the light from the image projector to provide the image content to the user's eyes.

[0270] 13. A system according to any of the above examples, wherein the at least one coupling optical element is also configured to couple light from the image projector guided in the waveguide out of the at least one waveguide so that image content can be viewed by the user's eyes.

[0271] 14. A system according to any of the above examples, wherein the same coupling optical element is configured to couple light from the image projector guided within the waveguide out of the waveguide so that image content can be viewed by the user's eyes, and to couple light into the at least one waveguide to be guided therein to the camera.

[0272] 15. The system according to any of the above examples also includes at least one image content coupling optical element, which is configured to couple light from the image projector guided within the waveguide out of the at least one waveguide so that the image content can be viewed by the user's eyes.

[0273] 16. A system according to any of the above examples, wherein the at least one coupling optical element faces the eye of the user wearing the head-mounted imaging system to receive light from the eye.

[0274] 17. A system according to any of the above examples, wherein the at least one coupling optical element is configured so that light from the environment in front of the user wearing the head-mounted imaging system is coupled into and guided in the at least one waveguide so that an image of the environment can be captured by the camera.

[0275] 18. A system according to any of the above examples, wherein the at least one coupling optical element faces the environment in front of the user wearing the head-mounted imaging system to receive light from the environment.

[0276] 19. A system according to Example 15, wherein the at least one image content coupling-out optical element is configured to couple light from the image projector guided in the waveguide out of the at least one waveguide, and the at least one coupling optical element is configured so that the light is coupled into the waveguide and guided therein to the camera, wherein the at least one image content coupling-out optical element and the at least one coupling optical element are superimposed on each other.

[0277] 20. A system according to Example 15, wherein the at least one image content coupling-out optical element is configured to couple light from the image projector guided within the waveguide out of the at least one waveguide, and the at least one coupling optical element is configured such that the light is coupled into the waveguide and guided therein to the camera, wherein the at least one image content coupling-out optical element and the at least one coupling optical element are stacked on each other.

[0278] 21. A system according to Example 15, wherein the at least one image content coupling-out optical element is configured to couple light from the image projector guided within the waveguide out of the at least one waveguide, and the at least one coupling optical element is configured such that the light is coupled into the waveguide and guided therein to the camera, wherein the at least one image content coupling-out optical element and the at least one coupling optical element are integrated in the same diffractive optical element.

[0279] 22. A system according to Example 15, wherein the at least one coupling optical element is configured such that light is coupled into a first waveguide and guided therein to the camera, and the at least one image content coupling optical element is configured to couple light from the image projector that is guided into a second waveguide out of the second waveguide.

[0280] 23. A system according to any of the above examples, wherein the at least one coupling optical element is configured such that light is coupled into a first waveguide and directed therein to the camera, and the image projector is configured to couple light into a second waveguide to provide image content to the eye.

[0281] 24. A system according to any of the above examples, wherein the image projector includes a light source, a modulator, and a projection optical device.

[0282] 25. A system according to any of the above examples, wherein the image projector includes a scanning optical fiber.

[0283] 26. A system according to any of Examples 24 or 25, wherein the modulator comprises an optical modulator.

[0284] 27. A system according to Example 26, wherein the light modulator includes a spatial light modulator.

[0285] 28. A system according to any of the above examples, wherein the camera includes a detector array and an imaging optical device.

[0286] 29. The system of Example 28, wherein the imaging optics are configured to focus collimated light onto the detector array.

[0287] 30. A system according to any of the above examples, wherein the at least one waveguide includes a material transparent to visible light, the material having a refractive index sufficient to guide light in the waveguide by total internal reflection.

[0288] 31. A system according to any of the above examples, wherein the at least one waveguide comprises a stack of waveguides.

[0289] 32. A system according to Example 31, wherein different waveguides in the waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eyes.

[0290] 33. A system according to Example 31 or 32, wherein different waveguides in the waveguide stack are configured to output light having different colors.

[0291] 34. A system according to any of Examples 31, 32 or 33, wherein the different waveguides include first, second and third waveguides, and the system is configured such that the first waveguide is used for red light, the second waveguide is used for green light, and the third is used for blue light.

[0292] 35. A system according to any one of Examples 12 to 34, wherein the coupling optical element includes a diffractive optical element or a reflector.

[0293] 36. A system according to any one of Examples 12 to 34, wherein the coupling optical element includes a diffractive optical element.

[0294] 37. A system according to any of the above examples, wherein the coupling optical element includes a diffractive optical element.

[0295] 38. A system according to any of the above examples, wherein the coupling optical element includes liquid crystal.

[0296] 39. A system according to any of the above examples, wherein the coupling optical element includes a liquid crystal polarization grating.

[0297] 40. A system according to any of the above examples, wherein the outcoupling optical element includes a diffractive optical element.

[0298] 41. A system according to any of the above examples, wherein the outcoupling optical element includes liquid crystal.

[0299] 42. A system according to any of the above examples, wherein the outcoupling optical element includes a liquid crystal polarization grating.

[0300] 43. A system according to any of the above examples, wherein the coupling element is configured to increase the size of the eye box along at least one axis.

[0301] 44. The system of Example 43 further comprises an orthogonal pupil expander comprising at least one light redirecting element positioned in or on the at least one waveguide, the at least one light redirecting element being configured to increase the size of the eye box along an axis orthogonal to the at least one axis.

[0302] 45. The system of Example 44, wherein the at least one light redirecting element comprises a diffractive optical element.

[0303] 46. ​​A system according to any of the above examples, wherein the same coupling element is configured to (a) couple light into the at least one waveguide to be received by the camera, and (b) couple light from the image projector out of the at least one waveguide to the user's eyes.

[0304] 47. A system according to any of the above examples, wherein the same coupling element is configured to (a) couple light from the environment into the at least one waveguide to be received by the camera, and (b) couple light from the image projector out of the at least one waveguide to the user's eyes.

[0305] 48. A system according to any of the above examples, wherein the same coupling element is configured to (a) couple light from the eye into the at least one waveguide to be received by the camera, and (b) couple light from the image projector out of the at least one waveguide to the user's eye.

[0306] 49. The system of any of the above examples further comprises a reflective surface having an optical power, the reflective surface being configured to receive light reflected from the user's eye that passes through the eyepiece and direct the light back to the eyepiece.

[0307] 50. A system according to Example 49, wherein the at least one coupling element is configured such that light from the user's eye that passes through at least one waveguide and reflects from the reflective surface back to at least one waveguide is coupled into and guided therein.

[0308] 51. A system according to any one of Examples 49 to 50, wherein a camera is disposed in an optical path relative to the at least one out-coupling optical element to receive at least a portion of light from the user's eye, at least a portion of which is reflected from the reflective surface and coupled into the waveguide via a coupling element and guided therein and coupled out of the waveguide through the out-coupling coupling element.

[0309] 52. A system as described in any of Examples 49 to 51, wherein the reflective surface reflects infrared light but transmits visible light.

[0310] 53. A system according to any of Examples 49 to 52, wherein the reflective surface is curved.

[0311] 54. A system according to any of Examples 49 to 53, wherein the reflective surface is disposed on a curved optical element.

[0312] 55. A system according to any of Examples 49 to 54, wherein the reflective surface is disposed on a concave mirror.

[0313] 56. A system according to any of Examples 49 to 55, wherein the reflective surface has positive optical power with respect to reflection and negligible optical power with respect to transmission.

[0314] 57. A system according to any of Examples 49 to 56, wherein the reflective surface is configured to collimate light from the user's eyes.

[0315] 58. A system according to any of Examples 49 to 57, wherein the reflective surface is configured to collimate light from the retina of the user's eye.

[0316] 59. A system as described in any of Examples 49 to 58, wherein the reflective surface is configured to collimate light from a front region of the user's eye.

[0317] 60. A system according to any of Examples 49 to 59, wherein the reflective surface is configured to collimate light from the cornea of ​​the user's eye.

[0318] 61. A system as described in any of Examples 49 to 60, wherein the reflective surface is formed on a curved optical element and includes an infrared reflective coating.

[0319] 62. A system according to Example 61, wherein the curved optical element has negligible focal length with respect to light transmitted therethrough.

[0320] 63. A system according to example 61 or 62, wherein the curved optical element has a first curved surface and a second curved surface located on opposite sides of the curved optical element, and the first curved surface and the second curved surface have the same curvature.

[0321] 64. A system according to any of Examples 49 to 63, further comprising a delay device arranged relative to the reflective surface and the coupling optical element to rotate the polarization of light that passes through at least one waveguide and is reflected from the reflective surface back to the at least one waveguide and the coupling optical element.

[0322] 65. A system according to any of the above examples, wherein at least one coupling element includes a polarization selective steering element.

[0323] 66. A system according to any of the above examples, wherein at least one coupling element includes a polarization grating.

[0324] 67. A system according to any of the above examples, wherein at least one coupling element is configured to divert light guided within at least one waveguide out of the waveguide as collimated light directed toward the user's eyes.

[0325] 68. A system according to any of the above examples, wherein at least one coupling element is configured to redirect collimated light from the reflective surface into at least one waveguide.

[0326] 69. A system according to any of the above examples, wherein at least one outcoupling element comprises an off-axis reflector.

[0327] 70. A system according to any of the above examples, wherein at least one outcoupling element includes a polarization selective steering element.

[0328] 71. A system according to any of the above examples, wherein at least one outcoupling element comprises a polarization grating.

[0329] 72. A system according to any of the above examples, wherein at least one outcoupling element comprises liquid crystal.

[0330] 73. A system according to any of the above examples, wherein at least one outcoupling element comprises a liquid crystal polarization grating.

[0331] 74. The system of any of the above examples further comprises a circular polarizer.

[0332] 75. A system according to any of the above examples, wherein the coupling element includes a polarization selective steering element.

[0333] 76. A system according to any of the above examples, wherein the coupling element includes a polarization grating.

[0334] 77. A system according to any of the above examples, wherein at least one coupling element includes a diffractive optical element.

[0335] 78. A system according to any of the above examples, wherein at least one coupling element includes a diffraction grating.

[0336] 79. A system according to any of the above examples, wherein the coupling element includes an off-axis reflector.

[0337] 80. A system according to any of Examples 49 to 79, wherein the reflective surface includes a liquid crystal reflector.

[0338] 81. A system according to any of Examples 49 to 80, wherein the reflective surface includes a cholesteric liquid crystal reflective lens.

[0339] 82. A system according to any of the above examples, wherein the same waveguide (a) guides light coupled from a user's eye into the at least one waveguide to be received by the camera so as to capture an image of at least a portion of the user's eye, and (b) guides light coupled from the image projector so that light from the projector can be guided to the user's eye so that the image from the image projector is in the user's field of view.

[0340] 83. A system according to any of the above examples, wherein the same coupling element (a) couples light from the user's eyes into the at least one waveguide to be received by the camera, and (b) couples light from the image projector out of the at least one waveguide to the user's eyes.

[0341] 84. The system of any of Examples 49 to 83, further comprising an electronic device configured to cause the camera to capture a first image when light reflected from the reflective surface is blocked.

[0342] 85. A system according to Example 84, wherein the electronic device is configured to cause the camera to capture a second image when light reflected from the reflective surface is not blocked.

[0343] 86. A system according to Example 85, wherein the electronic device is configured to use the first image to modify the second image.

[0344] 87. A system according to example 85 or 86, wherein the electronic device is configured to subtract the second image based on the first image.

[0345] 88. A system according to any of the above examples, wherein the system is configured to perform eye tracking based on an image of the eye.

[0346] 89. A system according to Example 88, wherein performing eye tracking based on the image of the eye includes storing an image of the retina of the eye.

[0347] 90. A system according to any of the above examples, wherein the system is configured to:

[0348] obtaining an image of a portion of the retina of the eye using the camera;

[0349] comparing one or more stored images of the retina to the image of the portion of the retina; and

[0350] The user's gaze is determined based on a comparison of the one or more stored images with an image of a portion of the retina obtained from a camera.

[0351] 91. A system according to example 90, wherein determining the user's gaze includes determining which portion of the retina corresponds to the image of the portion of the retina.

[0352] 92. A system according to any of Examples 90 to 91, wherein determining the gaze of the user includes determining the orientation of the eyes.

[0353] 93. A system according to any of the above examples, wherein the system is configured to obtain biometric data based on one or more images of the user's eyes obtained using the camera.

[0354] 94. A system according to any of the above examples, wherein the system is configured to identify a user via biometric sensing based on one or more images of the eye obtained using the camera.

[0355] 95. A system according to any of the above examples, wherein the system is configured to provide illumination of a first polarization and to preferentially capture images with the camera using light of a second polarization different from the first polarization.

[0356] 96. A system according to any of the above examples, wherein the system is configured to illuminate the user's eyes using a first polarized light and to preferentially capture an image of the user's eyes with the camera using a second polarized light different from the first polarization.

[0357] 97. A system according to Example 95 or 96, wherein the first polarization and the second polarization are orthogonal.

[0358] 98. A system according to any of the above examples, further comprising a light source configured to provide illumination for capturing images using the camera.

[0359] 99. The system of any of the above examples further comprises a light source configured to illuminate the user's eyes.

[0360] 100. The system of Examples 98 or 99, wherein the light source comprises one or more infrared light sources.

[0361] 101. A system as described in any of Examples 98 to 100, wherein the light source includes one or more infrared light emitting diodes (LEDs).

[0362] 102. A system according to any one of Examples 98 to 101, wherein the light source is pulsed.

[0363] 103. The system of any of Examples 98 to 102, further comprising an off-axis reflector configured to receive light from the light source and illuminate the user's eyes with the light.

[0364] 104. A system as described in any of Examples 98 to 103, wherein the light source is configured to input light into the waveguide to provide the illumination.

[0365] 105. A system as described in any of Examples 98 to 104, wherein the light source is configured to input light into a waveguide positioned relative to the eye to provide illumination to the eye.

[0366] 106. The system of example 104 or 105 further comprises an illumination incoupling optical element configured to couple light from the light source into the waveguide.

[0367] 107. A system according to any of Examples 98 to 103, wherein the light source is configured to input light into the at least one waveguide to provide illumination.

[0368] 108. The system of Example 107 further comprises an illumination incoupling optical element configured to couple light from the light source into the at least one waveguide to provide illumination.

[0369] 109. A system according to any of Examples 98 to 103, wherein the light source is configured to input light into the same waveguide as used to project image content into the user's eyes.

[0370] 110. A system according to any of Examples 98 to 104, wherein the light source is configured to provide illumination to an eye of a user to input light into the same waveguide as used to direct the light to the camera.

[0371] 111. A system as described in any of Examples 98 to 105, wherein the light source is configured to input light into the same waveguide as used to direct light from the user's eye to the camera.

[0372] 112. The system of any one of Examples 109 to 111, further comprising an illumination incoupling optical element configured to couple light from the light source into the waveguide.

[0373] 113. A system according to any of Examples 106, 108 or 112, wherein the illumination coupling optical element is polarization selective and couples in light having a first polarization.

[0374] 114. A system according to examples 98 to 113, wherein the light source is a polarized light source configured to output polarized light having a first polarization.

[0375] 115. A system according to any of Examples 98 to 114, wherein the light source is configured to direct polarized light having a first polarization onto the eye.

[0376] 116. The system of Examples 98 to 115, further comprising an illumination polarizer having a first polarization disposed in the light path between the light source and the eye to polarize light directed to the eye.

[0377] 117. A system according to Example 116, wherein an illumination polarizer is disposed in the optical path between the light source and the waveguide and is configured to provide illumination.

[0378] 118. The system of any of Examples 98 to 117, further comprising an image acquisition polarizer located in the optical path between the eye and the camera.

[0379] 119. A system according to Example 118, wherein the image acquisition polarizer is located at the proximal end of the camera.

[0380] 120. A system according to Example 118 or 119, wherein the image acquisition polarizer is disposed in an optical path between (a) at least one waveguide configured to direct light to the camera and (b) the camera.

[0381] 121. A system according to any of Examples 118 to 120, wherein the image acquisition polarizer reduces the amount of the first polarized light reaching the camera.

[0382] 122. A system according to examples 118 to 121, wherein the image acquisition polarizer includes a polarizer configured to selectively couple light having a second polarization different from the first polarization to the camera.

[0383] 123. The system according to any of the above examples further includes at least one light combining element, which is arranged on the optical path between the at least one coupling element and the at least one out-coupling optical element to reduce the lateral spatial range of light from the at least one coupling element before reaching the at least one out-coupling optical element.

[0384] 124. A system according to any of the above examples, wherein the at least one light combining element includes a diffractive optical element.

[0385] 125. A system according to any of the above examples, wherein the at least one light combining element comprises a hologram or a diffraction grating.

[0386] 126. A system according to any of the above examples, wherein the at least one waveguide includes a material transparent to infrared light, the material having a refractive index sufficient to guide the infrared light in the waveguide by total internal reflection.

[0387] 127. A system according to any of the above examples, wherein the at least one coupling optical element includes an exit pupil expander.

[0388] 128. A system according to any of the above examples, wherein the system includes optical focal length to increase the collimation of light reflected from the eye that is coupled into the waveguide to be directed to the camera.

[0389] 129. A system according to any of the above examples, wherein the system includes optical focal length to increase the collimation of light reflected from the front of the eye that is coupled into the waveguide to be directed to the camera.

[0390] 130. A system according to any of the above examples, wherein the system includes optical focal length to increase the collimation of light reflected from the cornea of ​​the eye that is coupled into the waveguide to be guided to the camera.

[0391] 131. A system according to any of Examples 128 to 130, wherein the optical power includes positive optical power.

[0392] 132. A system according to any of Examples 128 to 131, wherein the optical power is provided by a lens.

[0393] 133. A system of any of Examples 88 to 132, wherein the one or more stored images of the retina of the eye include a composite image of the retina of the eye generated using multiple images of different portions of the retina of the eye.

[0394] 134. A system of any of Examples 88 to 133, wherein the composite image of the retina comprises multiple images of the retina stitched together.

[0395] 135. A system according to any of Examples 88 to 134, wherein the multiple images of the retina stitched together include images obtained when a gaze target is displayed in the user's field of view at respective locations.

[0396] 136. A system according to any of Examples 88 to 135, wherein the one or more stored images of the retina include images obtained when a gaze target is displayed in the user's field of view at respective locations.

[0397] 137. A system according to any of Examples 88 to 136, wherein the system is further configured to update the composite image using the obtained image of a portion of the retina of the eye.

[0398] 138. A system according to any of Examples 88 to 137, wherein updating the composite image of the retina using the obtained image of the portion of the retina includes stitching the obtained image into a portion of the composite image that corresponds to the portion of the retina shown in the obtained image.

[0399] 139. A system according to any of Examples 88 to 138, wherein the system is further configured to apply a digital filter to the obtained image of the portion of the retina of the eye to obtain a filtered image of the portion of the retina.

[0400] 140. The system of Example 139, wherein the system is further configured to compare one or more stored images of the retina with a filtered image of a portion of the retina.

[0401] 141. A system according to any of Examples 139 to 140, wherein the digital filter includes a Frangi filter.

[0402] 142. A system according to any of Examples 88 to 139, wherein the system is configured to apply edge enhancement to the obtained image of the portion of the retina.

[0403] 143. A system according to any of the above examples, wherein the system is configured to use an image of the retina to perform user identification verification.

[0404] 144. A system according to any of the above examples, wherein the system is configured to:

[0405] obtaining an image of a portion of the retina of the eye using the camera;

[0406] One or more stored images of the retina are compared to the image of the portion of the retina.

[0407] 145. A system of example 144, wherein the one or more stored images of the retina of the eye include a composite image of the retina of the eye generated using multiple images of different portions of the retina of the eye.

[0408] 146. A system of any of Examples 144 to 145, wherein the composite image of the retina comprises multiple images of the retina stitched together.

[0409] 147. A system according to any of Examples 144 to 146, wherein the multiple images of the retina stitched together include images obtained when a gaze target is displayed in the user's field of view at respective locations.

[0410] 148. A system according to any of Examples 144 to 146, wherein the one or more stored images of the retina include images obtained when a gaze target is displayed in the user's field of view at respective locations.

[0411] 149. A system according to any of Examples 144 to 148, wherein the system is further configured to update the composite image using the obtained image of a portion of the retina of the eye.

[0412] 150. A system according to any of Examples 144 to 149, wherein updating the composite image of the retina using the acquired image of the portion of the retina includes stitching the acquired image into a portion of the composite image that corresponds to the portion of the retina shown in the acquired image.

[0413] 151. A system according to any of Examples 144 to 150, wherein the system is further configured to apply a digital filter to the obtained image of the portion of the retina of the eye to obtain a filtered image of the portion of the retina.

[0414] 152. A system according to example 151, wherein the system is further configured to compare one or more stored images of the retina with a filtered image of a portion of the retina.

[0415] 153. A system according to any of Examples 144 to 152, wherein the digital filter includes a Frangi filter.

[0416] 154. A system according to any of Examples 144 to 153, wherein the system is configured to apply edge enhancement to the obtained image of the portion of the retina.

[0417] Example Part II

[0418] 1. A head mounted display system configured to project light into eyes of a user to display augmented reality image content in a field of view of the user, and configured to image at least a portion of an environment in front of a user wearing the head mounted display system, the head mounted display system comprising:

[0419] a frame configured to be supported on a head of the user;

[0420] an image projector configured to project an image;

[0421] camera;

[0422] an eyepiece disposed on the frame, the eyepiece being configured to direct light into the user's eyes to display augmented reality image content to the user's field of view, at least a portion of the eyepiece being transparent and disposed at a position in front of the user's eyes when the user wears the head mounted display, such that the transparent portion transmits light from an environment in front of the user to the user's eyes to provide a view of the environment in front of the user, the eyepiece comprising:

[0423] (a) at least one waveguide;

[0424] (b) at least one in-coupling optical element configured to couple light from the image projector into the at least one waveguide so as to guide the light from the image projector therein;

[0425] (c) at least one coupling optical element configured to couple light from the image projector guided in the waveguide out of the waveguide and direct the light to an eye of a user; and

[0426] (d) at least one outcoupling element configured to couple light guided within the waveguide out of the waveguide and guide the light to the camera,

[0427] wherein an image projector is arranged in an optical path relative to the at least one coupling-in optical element to couple light from the image projector into the waveguide to be guided therein, so that the light is coupled out of the waveguide to the user's eye by the at least one coupling element so that the image from the projector is within the user's field of view,

[0428] wherein the coupling element is configured such that light from the environment in front of the user wearing the head mounted display is coupled into the waveguide and guided therein,

[0429] wherein a camera is arranged in an optical path relative to the at least one outcoupling optical element to receive at least a portion of the light from the environment in front of the user, the at least a portion of the light being coupled into the waveguide via a coupling element and guided therein, and being coupled out of the waveguide by the outcoupling coupling element so that an image can be captured by the camera, and

[0430] wherein the same waveguide (a) guides light coupled into the waveguide from the environment to be received by the camera so as to capture an image of at least a portion of the environment in front of a user, and (b) guides light coupled from the projector so that light from the projector is directed to the user's eyes so that the image from the projector is within the user's field of view.

[0431] 2. The system of Example 1, wherein the image projector comprises a light source, a modulator, and projection optics.

[0432] 3. A system according to Example 1 or 2, wherein the image projector includes a scanning optical fiber.

[0433] 4. The system of Example 2 or 3, wherein the modulator comprises an optical modulator.

[0434] 5. The system of Example 4, wherein the light modulator comprises a spatial light modulator.

[0435] 6. A system according to any of the above examples, wherein the camera includes a detector array and an imaging optical device.

[0436] 7. The system of Example 6, wherein the imaging optics are configured to focus collimated light onto the detector array.

[0437] 8. A system according to any of the above examples, wherein the at least one waveguide comprises a material transparent to visible light, the material having a refractive index sufficient to guide light in the waveguide by total internal reflection.

[0438] 9. A system according to any of the above examples, wherein the at least one waveguide comprises a stack of waveguides.

[0439] 10. A system according to Example 9, wherein different waveguides in the waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eyes.

[0440] 11. A system according to Example 9 or 10, wherein different waveguides in the waveguide stack are configured to output light having different colors.

[0441] 12. A system according to any of Examples 9, 10 or 11, wherein the different waveguides include first, second and third waveguides, and the system is configured such that the first waveguide is used for red light, the second waveguide is used for green light, and the third is used for blue light.

[0442] 13. A system according to any of the above examples, wherein the coupling-in optical element comprises a diffractive optical element or a reflector.

[0443] 14. A system according to any of the above examples, wherein the coupling optical element includes a diffractive optical element.

[0444] 15. A system according to any of the above examples, wherein the outcoupling optical element includes a diffractive optical element.

[0445] 16. A system according to any of the above examples, wherein the coupling element is configured to increase the size of the eye box along at least one axis.

[0446] 17. The system of Example 16 further comprises an orthogonal pupil expander comprising at least one light redirecting element positioned in or on the at least one waveguide, the at least one light redirecting element being configured to increase the size of the eye box along an axis orthogonal to the at least one axis.

[0447] 18. The system of Example 17, wherein the at least one light redirecting element comprises a diffractive optical element.

[0448] 19. A system according to any of the above examples, wherein the same coupling element is configured to (a) couple light from the environment into the at least one waveguide to be received by the camera, and (b) couple light from the image projector out of the at least one waveguide to the user's eyes.

[0449] 20. A system according to any of the above examples, wherein the same coupling element is configured to (a) couple light from the environment into the at least one waveguide to be received by the camera, and (b) couple light from the image projector out of the at least one waveguide to the user's eyes.

[0450] 21. A system according to Example 20, wherein the at least one coupling element is configured such that light from the user's eye that passes through the eyepiece and reflects from the reflective surface back to the eyepiece is coupled into and guided therein.

[0451] 22. A system according to any one of Examples 20 to 21, wherein a camera is disposed in an optical path relative to the at least one out-coupling optical element to receive at least a portion of light from the user's eye, at least a portion of which is reflected from a reflective surface and coupled into the waveguide via a coupling element and guided therein and coupled out of the waveguide through the out-coupling coupling element.

[0452] 23. A system according to any of Examples 20 to 22, wherein the reflective surface reflects infrared light but transmits visible light.

[0453] 24. A system according to any of Examples 20 to 23, wherein the reflective surface is curved.

[0454] 25. A system according to any of Examples 20 to 24, wherein the reflective surface is disposed on a curved optical element.

[0455] 26. A system according to any one of Examples 20 to 25, wherein the reflective surface is disposed on a concave mirror.

[0456] 27. A system according to any of Examples 20 to 26, wherein the reflective surface has positive optical power with respect to reflection and negligible optical power with respect to transmission.

[0457] 28. A system according to any of Examples 20 to 27, wherein the reflective surface is configured to collimate light from the user's eyes.

[0458] 29. A system according to any of Examples 20 to 28, wherein the reflective surface is configured to collimate light from the retina of the user's eye.

[0459] 30. A system according to any of Examples 20 to 29, wherein the reflective surface is configured to collimate light from a front region of the user's eye.

[0460] 31. A system according to any of Examples 20 to 30, wherein the reflective surface is configured to collimate light from the cornea of ​​the user's eye.

[0461] 32. A system according to any of Examples 20 to 31, wherein the reflective surface is formed on a curved optical element having an infrared reflective coating located on the reflective surface.

[0462] 33. A system according to Example 33, wherein the curved optical element has negligible focal length with respect to light transmitted therethrough.

[0463] 34. A system according to Example 32 or 33, wherein the curved optical element has a first curved surface and a second curved surface located on opposite sides of the curved optical element, and the first curved surface and the second curved surface have the same curvature.

[0464] 35. A system according to any of the above examples, further comprising a delay device arranged relative to the reflective surface and the coupling optical element to rotate the polarization of light that passes through the eyepiece and is reflected from the reflective surface back to the eyepiece and the coupling optical element.

[0465] 36. A system according to any of the above examples, wherein at least one coupling element includes a polarization selective steering element.

[0466] 37. A system according to any of the above examples, wherein at least one coupling element comprises a polarization grating.

[0467] 38. A system according to any of the above examples, wherein at least one coupling element is configured to divert light guided within at least one waveguide out of the waveguide as collimated light directed toward an eye of a user.

[0468] 39. A system according to any of the above examples, wherein at least one coupling element is configured to redirect collimated light from the reflective surface into at least one waveguide.

[0469] 40. The system of any of the above examples, wherein at least one outcoupling element comprises an off-axis reflector.

[0470] 41. A system according to any of the above examples, wherein at least one outcoupling element includes a polarization selective steering element.

[0471] 42. A system according to any of the above examples, wherein at least one outcoupling element comprises a polarization grating.

[0472] 43. A system according to any of the above examples, further comprising a circular polarizer.

[0473] 44. A system according to any of the above examples, wherein the coupling element includes a polarization selective steering element.

[0474] 45. A system according to any of the above examples, wherein the coupling-in element includes a polarization grating.

[0475] 46. ​​A system according to any of the above examples, wherein the coupling-in element includes an off-axis reflector.

[0476] 47. A system according to any of Examples 20 to 34, wherein the reflective surface includes a liquid crystal reflector.

[0477] 48. A system according to any of Examples 20 to 34 or 37, wherein the reflective surface includes a cholesteric liquid crystal reflective lens.

[0478] 49. A system according to any of the above examples, wherein the same waveguide (a) guides light coupled from a user's eye into the at least one waveguide to be received by the camera so as to capture an image of at least a portion of the user's eye, and (b) guides light coupled from the image projector so that light from the projector can be guided to the user's eye so that the image from the image projector is in the user's field of view.

[0479] 50. A system according to any of the above examples, wherein the same coupling element (a) couples light from the user's eyes into the at least one waveguide to be received by the camera, and (b) couples light from the image projector out of the at least one waveguide to the user's eyes.

[0480] 51. The system of any of the above examples, further comprising an electronic device configured to cause the camera to capture a first image when light reflected from the reflective surface is blocked.

[0481] 52. A system according to Example 51, wherein the electronic device is configured to cause the camera to capture a second image when light reflected from the reflective surface is not blocked.

[0482] 53. A system according to Example 52, wherein the electronic device is configured to use the first image to modify the second image.

[0483] 54. A system according to Example 53, wherein the electronic device is configured to subtract the second image based on the first image.

[0484] 55. A system according to any of the above examples, wherein the system is configured to perform eye tracking based on the image of the eye.

[0485] 56. A system according to Example 55, wherein performing eye tracking based on the image of the eye includes storing an image of the retina of the eye.

[0486] 57. A system according to any of the above examples, wherein the system is configured to:

[0487] storing an image of a retina of said eye;

[0488] capturing an image of a portion of the retina of the eye;

[0489] comparing a stored image of the retina to an image of the portion of the retina; and

[0490] Based on a comparison of the stored image with an image of the portion of the retina, the user's gaze is determined.

[0491] 58. A system according to Example 57, wherein determining the user's gaze includes determining which portion of the retina corresponds to the image of the portion of the retina.

[0492] 59. A system according to any of Examples 57 to 58, wherein determining the user's gaze includes determining an orientation of the eyes.

[0493] 60. A system according to any of the above examples, further comprising a light source configured to illuminate the user's eyes.

[0494] 61. A system according to Example 60, wherein the light source includes one or more infrared light sources configured to direct infrared light to the user's eyes.

[0495] 62. A system according to example 60 or 61, wherein the light source includes one or more infrared light emitting diodes (LEDs).

[0496] 63. A system according to any one of Examples 60 to 62, wherein the light source is pulsed.

[0497] 64. The system of any of the above examples further comprises an off-axis reflector configured to receive light from the light source and illuminate the user's eyes with the light.

[0498] 65. A head-mounted imaging system configured to image at least a portion of an environment in front of a user wearing the head-mounted imaging system, the head-mounted imaging system comprising:

[0499] a frame configured to be supported on a head of the user;

[0500] camera;

[0501] an eyepiece disposed on the frame, at least a portion of the eyepiece being transparent and disposed at a position in front of an eye of the user when the user wears the head-mounted imaging system, such that the transparent portion transmits light from an environment in front of the user to the user's eye to provide a view of the environment in front of the user, the eyepiece comprising:

[0502] (a) at least one waveguide;

[0503] (b) at least one in-coupling optical element configured such that light from the environment in front of a user wearing the head-mounted imaging system is coupled into and guided within the waveguide; and

[0504] (c) at least one outcoupling element configured to couple light guided within the waveguide out of the waveguide and guide the light to the camera,

[0505] Wherein, a camera is arranged in an optical path relative to the at least one outcoupling optical element to receive at least a portion of the light from the environment in front of the user, the at least a portion of the light is coupled into the waveguide via a coupling element and guided therein, and is coupled out of the waveguide through the outcoupling coupling element so that an image can be captured by the camera.

[0506] 66. A system according to Example 65, wherein the camera comprises a detector array and an imaging optical device.

[0507] 67. The system of Example 66, wherein the imaging optics are configured to focus collimated light onto the detector array.

[0508] 68. A system according to any of Examples 65 to 67, wherein the at least one waveguide comprises a material transparent to visible light having a refractive index sufficient to guide light in the waveguide by total internal reflection.

[0509] 69. A system according to any of Examples 65 to 68, wherein the at least one waveguide comprises a stack of waveguides.

[0510] 70. A system according to Example 69, wherein different waveguides in the waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eyes.

[0511] 71. A system according to Example 69 or 70, wherein different waveguides in the waveguide stack are configured to output light having different colors.

[0512] 72. A system according to any of Examples 69 to 71, wherein the different waveguides include first, second, and third waveguides, and the system is configured such that the first waveguide is used for red light, the second waveguide is used for green light, and the third is used for blue light.

[0513] 73. A system according to any one of Examples 65 to 72, wherein the coupling optical element includes a diffractive optical element.

[0514] 74. A system according to any one of Examples 65 to 73, wherein the coupling-out optical element comprises a diffractive optical element.

[0515] 75. A system as described in any of Examples 65 to 74, wherein the coupling element is configured to increase the size of the eye box along at least one axis.

[0516] 76. The system of Example 75 further comprises an orthogonal pupil expander comprising at least one light redirecting element positioned in or on the at least one waveguide, the at least one light redirecting element being configured to increase the size of the eye box along an axis orthogonal to the at least one axis.

[0517] 77. A system according to Example 76, wherein the at least one light redirecting element comprises a diffractive optical element.

[0518] Example Part III

[0519] 1. A head mounted display system configured to project light into a user's eyes to display augmented reality image content in the user's field of view, and configured to image at least a portion of an eye of a user wearing the head mounted display system, the head mounted display system comprising:

[0520] a frame configured to be supported on a head of the user;

[0521] an image projector configured to project an image;

[0522] camera;

[0523] an eyepiece disposed on the frame, the eyepiece being configured to direct light into the user's eyes to display augmented reality image content to the user's field of view, at least a portion of the eyepiece being transparent and disposed at a position in front of the user's eyes when the user wears the head mounted display, such that the transparent portion transmits light from an environment in front of the user to the user's eyes to provide a view of the environment in front of the user, the eyepiece comprising:

[0524] (a) at least one waveguide;

[0525] (b) at least one in-coupling optical element configured to couple light from the image projector into the at least one waveguide so as to guide the light from the image projector therein;

[0526] (c) at least one coupling optical element configured to couple light from the image projector guided in the waveguide out of the waveguide and direct the light to an eye of a user; and

[0527] (d) at least one out-coupling element configured to couple the light guided within the waveguide out of the waveguide and guide the light to the camera; and

[0528] a reflective surface having optical power and configured to receive light reflected from an eye of a user that has passed through the eyepiece and direct the light back toward the eyepiece;

[0529] wherein an image projector is arranged in an optical path relative to the at least one coupling optical element to couple light from the image projector into the waveguide to be guided therein so that the light is coupled out of the waveguide to the user's eye by the at least one coupling element so that the image from the projector is within the user's field of view,

[0530] wherein the at least one coupling element is configured such that light from the user's eye that passes through the eyepiece and reflects from the reflective surface back to the eyepiece is coupled into and guided in the waveguide, and

[0531] Wherein, the camera is arranged in the optical path relative to the at least one outcoupling optical element to receive at least a portion of the light from the user's eye, at least a portion of which is reflected from the reflective surface and coupled into the waveguide via the coupling element and guided therein and coupled out of the waveguide through the outcoupling coupling element.

[0532] 2. The system according to Example 1 also includes a light source configured to illuminate the user's eyes.

[0533] 3. The system of Example 2, wherein the light source comprises one or more infrared light sources configured to direct infrared light to the user's eyes.

[0534] 4. A system according to Example 2 or 3, wherein the light source includes one or more infrared light emitting diodes (LEDs).

[0535] 5. The system of any one of Examples 2 to 4, wherein the light source is pulsed.

[0536] 6. The system of any of the above examples further comprises an off-axis reflector configured to receive light from the light source and illuminate the eye with the light.

[0537] 7. A system according to any of the above examples, wherein the reflective surface reflects infrared light but transmits visible light.

[0538] 8. A system according to any of the above examples, wherein the reflective surface is curved.

[0539] 9. A system according to any of the above examples, wherein the reflective surface is provided on a curved optical element.

[0540] 10. A system according to any of the above examples, wherein the reflective surface is arranged on a concave mirror.

[0541] 11. A system according to any of the above examples, wherein the reflective surface has positive optical power with respect to reflection and negligible optical power with respect to transmission.

[0542] 12. A system according to any of the above examples, wherein the reflective surface is configured to collimate light from the user's eyes.

[0543] 13. A system according to any of the above examples, wherein the reflective surface is configured to collimate light from the retina of the user's eye.

[0544] 14. A system according to any of the above examples, wherein the reflective surface is configured to collimate light from a front area of ​​the user's eye.

[0545] 15. A system according to any of the above examples, wherein the reflective surface is configured to collimate light from the cornea of ​​the user's eye.

[0546] 16. A system according to any of the above examples, wherein the reflective surface is formed on a curved optical element, the curved optical element having an infrared reflective coating located on the reflective surface.

[0547] 17. A system according to Example 9 or 16, wherein the curved optical element has negligible focal length with respect to light transmitted therethrough.

[0548] 18. A system according to any of Examples 9 or 16 or 17, wherein the curved optical element has a first curved surface and a second curved surface located on opposite sides of the curved optical element, and the first curved surface and the second curved surface have the same curvature.

[0549] 19. A system according to any of the above examples, further comprising a delay device arranged relative to the reflective surface and the coupling optical element to rotate the polarization of light that passes through the eyepiece and is reflected from the reflective surface back to the eyepiece and the coupling optical element.

[0550] 20. A system according to any of the above examples, wherein at least one coupling element comprises a polarization selective steering element.

[0551] 21. A system according to any of the above examples, wherein at least one coupling element comprises a polarization grating.

[0552] 22. A system according to any of the above examples, wherein at least one coupling element is configured to divert light guided within at least one waveguide out of the waveguide as collimated light directed toward an eye of a user.

[0553] 23. A system according to any of the above examples, wherein at least one coupling element is configured to redirect collimated light from the reflective surface into at least one waveguide.

[0554] 24. The system of any of the above examples, wherein at least one outcoupling element comprises an off-axis reflector.

[0555] 25. A system according to any of the above examples, wherein at least one outcoupling element comprises a polarization selective steering element.

[0556] 26. A system according to any of the above examples, wherein at least one outcoupling element comprises a polarization grating.

[0557] 27. A system according to any of the above examples, further comprising a circular polarizer.

[0558] 28. A system according to any of the above examples, wherein the coupling element includes a polarization selective steering element.

[0559] 29. A system according to any of the above examples, wherein the coupling-in element comprises a polarization grating.

[0560] 30. The system of any of the above examples, wherein the coupling-in element comprises an off-axis reflector.

[0561] 31. A system according to any of the above examples, wherein the reflective surface includes a liquid crystal reflector.

[0562] 32. A system according to any of the above examples, wherein the reflective surface comprises a cholesteric liquid crystal reflective lens.

[0563] 33. A system according to any of the above examples, wherein the image projector includes a light source, a modulator, and a projection optical device.

[0564] 34. A system according to any of the above examples, wherein the image projector includes a scanning optical fiber.

[0565] 35. A system according to any of the above examples, wherein the modulator comprises an optical modulator.

[0566] 36. A system according to Example 34, wherein the light modulator includes a spatial light modulator.

[0567] 37. A system according to any of the above examples, wherein the camera includes a detector array and an imaging optical device.

[0568] 38. A system according to Example 36, wherein the imaging optical device is configured to focus the collimated light onto the detector array.

[0569] 39. A system according to any of the above examples, wherein the at least one waveguide includes a material transparent to visible light, the material having a refractive index sufficient to guide light in the waveguide by total internal reflection.

[0570] 40. A system according to any of the above examples, wherein the at least one waveguide comprises a stack of waveguides.

[0571] 41. A system according to Example 40, wherein different waveguides in the waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eyes.

[0572] 42. A system according to Example 40 or 41, wherein different waveguides in the waveguide stack are configured to output light having different colors.

[0573] 43. A system according to any of Examples 40, 41 or 42, wherein the different waveguides include first, second and third waveguides, and the system is configured such that the first waveguide is used for red light, the second waveguide is used for green light, and the third waveguide is used for blue light.

[0574] 44. A system according to any of the above examples, wherein the coupling-in optical element includes a diffractive optical element or a reflector.

[0575] 45. A system according to any of the above examples, wherein the coupling optical element includes a diffractive optical element.

[0576] 46. ​​A system according to any of the above examples, wherein the outcoupling optical element includes a diffractive optical element.

[0577] 47. A system according to any of the above examples, wherein the coupling element is configured to increase the size of the eye box along at least one axis.

[0578] 48. The system of Example 47 further comprises an orthogonal pupil expander comprising at least one light redirecting element disposed in or on the at least one waveguide, the at least one light redirecting element being configured to increase a size of an eye box along an axis orthogonal to the at least one axis.

[0579] 49. The system of Example 48, wherein the at least one light redirecting element comprises a diffractive optical element.

[0580] 50. A system according to any of the above examples, wherein the same waveguide (a) guides light coupled from a user's eye into the at least one waveguide to be received by the camera so as to capture an image of at least a portion of the user's eye, and (b) guides light coupled from the image projector so that light from the projector can be guided to the user's eye so that the image from the image projector is in the user's field of view.

[0581] 51. A system according to any of the above examples, wherein the same coupling element (a) couples light from the user's eyes into the at least one waveguide to be received by the camera, and (b) couples light from the image projector out of the at least one waveguide to the user's eyes.

[0582] 52. The system of any of the above examples, further comprising an electronic device configured to cause the camera to capture a first image when light reflected from the reflective surface is blocked.

[0583] 53. A system according to Example 52, wherein the electronic device is configured to cause the camera to capture a second image when light reflected from the reflective surface is not blocked.

[0584] 54. A system according to Example 53, wherein the electronic device is configured to use the first image to modify the second image.

[0585] 55. A system according to Example 54, wherein the electronic device is configured to subtract the second image based on the first image.

[0586] 56. A system according to any of the above examples, wherein the system is configured to perform eye tracking based on the image of the eye.

[0587] 57. A system according to Example 56, wherein performing eye tracking based on the image of the eye includes storing an image of the retina of the eye.

[0588] 58. A system according to any of the above examples, wherein the system is configured to:

[0589] storing an image of a retina of said eye;

[0590] capturing an image of a portion of the retina of the eye;

[0591] comparing a stored image of the retina to an image of the portion of the retina; and

[0592] Based on a comparison of the stored image with an image of the portion of the retina, the user's gaze is determined.

[0593] 59. A system according to Example 58, wherein determining the user's gaze includes determining which portion of the retina corresponds to the image of the portion of the retina.

[0594] 60. A system according to any of Examples 58 to 59, wherein determining the gaze of the user includes determining an orientation of the eyes.

[0595] 61. A system according to any of the above examples, wherein the coupling element is configured so that light from the environment in front of a user wearing a head-mounted display is coupled into the waveguide and guided therein.

[0596] 62. A system according to any of the above examples, wherein a camera is arranged in an optical path relative to the at least one out-coupling optical element to receive at least a portion of light from the environment in front of a user, the at least a portion of the light is coupled into the waveguide via a coupling element and guided therein, and is coupled out of the waveguide by the out-coupling coupling element so that an image of the environment can be captured by the camera.

[0597] 63. A system according to any of the above examples, wherein the same waveguide (a) guides light coupled into the waveguide from the environment to be received by the camera so as to capture an image of at least a portion of the environment in front of a user, and (b) guides light coupled from the projector so that the light from the projector is guided to the user's eyes so that the image from the projector is within the user's field of view.

[0598] 64. A system according to any of the above examples, wherein the same coupling element (a) couples light from the environment into the at least one waveguide to be received by the camera, and (b) couples light from the image projector out of the at least one waveguide to the user's eyes.

[0599] 65. A head mounted display system configured to project light into an eye of a user to display augmented reality image content in the user's field of view, and configured to image at least a portion of an eye of a user wearing the head mounted display system, the head mounted display system comprising:

[0600] a frame configured to be supported on a head of the user;

[0601] an image projector configured to project an image;

[0602] camera;

[0603] an eyepiece disposed on the frame, the eyepiece being configured to direct light into the user's eyes to display augmented reality image content to the user's field of view, at least a portion of the eyepiece being transparent and disposed at a position in front of the user's eyes when the user wears the head mounted display, such that the transparent portion transmits light from an environment in front of the user to the user's eyes to provide a view of the environment in front of the user, the eyepiece comprising:

[0604] (a) at least one waveguide;

[0605] (b) at least one in-coupling optical element configured to couple light from the image projector into the at least one waveguide so as to guide the light from the image projector therein;

[0606] (c) at least one coupling optical element configured to couple light from the image projector guided in the waveguide out of the waveguide and direct the light to an eye of a user; and

[0607] (d) at least one out-coupling element configured to couple the light guided within the waveguide out of the waveguide and guide the light to the camera; and

[0608] a positive lens having positive optical power disposed in an optical path between the user's eye and the eyepiece so that light reflected from the user's eye is projected through the lens to the eyepiece; and

[0609] a negative lens having negative optical power, which is disposed on the other side of the eyepiece with the positive lens to cancel the optical power of the positive lens for light from the environment in front of the user,

[0610] wherein an image projector is arranged in an optical path relative to the at least one coupling-in optical element to couple light from the image projector into the waveguide to be guided therein, so that the light is coupled out of the waveguide to the user's eye by the at least one coupling element so that the image from the image projector is within the user's field of view,

[0611] wherein the at least one coupling element is configured such that light from the user's eye that passes through the lens to the eyepiece is coupled into and guided in the waveguide, and

[0612] Wherein, the camera is arranged in the optical path relative to the at least one outcoupling optical element to receive at least a portion of the light from the user's eye, at least a portion of which is reflected from the reflective surface and coupled into the waveguide via the coupling element and guided therein and coupled out of the waveguide through the outcoupling coupling element.

[0613] 66. The system of Example 65, wherein the positive lens comprises a Fresnel lens.

[0614] 67. A system according to Example 65 or 66, wherein the positive lens is configured to collimate light from a front region of the user's eye.

[0615] 68. The system of any of Examples 65, 66, or 67 above, wherein the positive lens is configured to collimate light from the cornea of ​​the user's eye.

[0616] 69. A system according to any of Examples 65 to 68, wherein the system is configured to perform eye tracking based on the image of the eye.

[0617] 70. The system of any one of Examples 65 to 69 further includes a light source configured to illuminate the user's eyes.

[0618] 71. A system according to Example 70, wherein the light source includes one or more infrared light sources configured to direct infrared light to the user's eyes.

[0619] 72. A system according to example 70 or 71, wherein the light source includes one or more infrared light emitting diodes (LEDs).

[0620] 73. A system according to any of the above examples, wherein the system is configured to identify a user via biometric sensing based on the image of the eye.

Claims

1. A head mounted display system configured to project light into a user's eyes to display augmented reality image content in the user's field of view, the head mounted display system comprising: a frame configured to be supported on a head of the user; an image projector configured to project an image into the eyes of the user to display image content in the field of view of the user; camera; at least one waveguide having a first major surface and a second major surface such that the light is guided therein; at least one coupling optical element configured such that light is coupled into and guided in the at least one waveguide; at least one outcoupling element configured to couple light guided in the at least one waveguide out of the at least one waveguide and direct the light to the camera; at least one image content out-coupling optical element configured to couple light from the image projector guided within the at least one waveguide out of the at least one waveguide so that image content can be viewed by an eye of the user; as well as a reflective optical element configured to collimate light from an eye of the user, the reflective optical element not being located within the at least one waveguide between the first major surface and the second major surface of the at least one waveguide, wherein the camera is arranged in an optical path relative to the at least one outcoupling element to receive at least a portion of the light, the at least a portion of the light being coupled into and guided in the at least one waveguide via the coupling optical element and being coupled out of the at least one waveguide by the outcoupling element, so that an image can be captured by the camera, and wherein the at least one coupling optical element is configured such that light is coupled into a first waveguide and guided therein to the camera, and wherein the at least one image content outcoupling optical element is configured to couple light from the image projector guided in a second waveguide out of the second waveguide.

2. The head mounted display system according to claim 1, wherein: The at least one coupling optical element is configured such that light from the environment in front of the user wearing the head mounted display system is coupled into and guided in the at least one waveguide so that an image of the environment can be captured by the camera.

3. The head mounted display system according to claim 1, wherein: The at least one coupling optical element is configured such that light reflected from the eye of the user wearing the head-mounted display system is coupled into and guided in the at least one waveguide so that an image of the eye can be captured by the camera.

4. The head mounted display system according to claim 1, wherein: The at least one coupling optical element is configured such that light reflected from the eye of the user wearing the head-mounted display system is coupled into and guided in the at least one waveguide so that an image of the eye can be captured by the camera, the system being configured to image the front of the eye.

5. The head mounted display system according to claim 1, wherein: The at least one coupling optical element is configured such that light reflected from the eye of the user wearing the head-mounted display system is coupled into and guided within the at least one waveguide so that an image of the eye can be captured by the camera, the system being configured to image the corneal surface of the eye.

6. The head mounted display system according to claim 1, wherein: The at least one coupling optical element is configured such that light reflected from the eye of the user wearing the head-mounted display system is coupled into and guided in the at least one waveguide so that an image of the eye can be captured by the camera, and the system is configured to image the retina of the user's eye.

7. The head-mounted display system according to claim 1 further includes an eyepiece disposed on the frame, the eyepiece being configured to direct light into the user's eyes to display augmented reality image content to the user's field of view, at least a portion of the eyepiece being transparent and disposed at a position in front of the user's eyes when the user wears the head-mounted display system, so that the transparent portion transmits light from the environment in front of the user to the user's eyes to provide a view of the environment in front of the user.

8. The head mounted display system according to claim 7, wherein: The eyepiece is configured to receive light from the image projector and direct the light into the user's eyes to display augmented reality image content to the user's field of view.

9. The head mounted display system according to claim 7, wherein: The eyepiece comprises the at least one waveguide.

10. The head mounted display system according to claim 7, wherein: The image projector is configured to direct light into an edge of the eyepiece.

11. The head mounted display system according to claim 9, wherein: The image projector is configured to direct light into an edge of the at least one waveguide.

12. The head-mounted display system of claim 1 , further comprising at least one in-coupling optical element configured to couple light from the image projector into the at least one waveguide so as to guide the light from the image projector to provide the image content to the user's eyes.

13. The head mounted display system according to claim 1, wherein: The at least one coupling optical element is also configured to couple light from the image projector guided in the first waveguide out of the first waveguide so that image content can be viewed by the user's eyes.

14. The head mounted display system according to claim 1, wherein: The same coupling optical element is configured to couple light from the image projector guided within the first waveguide out of the first waveguide so that image content can be viewed by the user's eyes, and to couple light into the first waveguide to be guided therein to the camera.

15. The head mounted display system according to claim 1, wherein: The at least one coupling optical element faces the eye of the user wearing the head-mounted display system to receive light from the eye.

16. The head mounted display system according to claim 1, wherein: The at least one coupling optical element faces the environment in front of the user wearing the head-mounted display system to receive light from the environment.

17. The head mounted display system according to claim 1, wherein: The at least one image content out-coupling optical element is configured to couple light from the image projector guided in the second waveguide out of the second waveguide, and the at least one coupling optical element is configured such that light is coupled into the first waveguide and guided therein to the camera, wherein the first waveguide and the second waveguide are the same waveguide, and wherein the at least one image content out-coupling optical element and the at least one coupling optical element overlap each other.

18. The head mounted display system according to claim 1, wherein: The at least one image content out-coupling optical element is configured to couple light from the image projector guided within the at least one waveguide out of the at least one waveguide, and the at least one coupling optical element is configured such that light is coupled into the at least one waveguide and guided therein to the camera, wherein the at least one image content out-coupling optical element and the at least one coupling optical element are stacked on each other.

Images