Eyewear with pinhole and slit cameras

Abstract

Providing eyewear with pinhole and slit cameras. [Solution] A head-mounted display system may include a camera, at least one waveguide, at least one coupling optical element configured to couple and guide light within the waveguide, and at least one external coupling element. The at least one external coupling element may be configured to couple the light induced in the waveguide out of the waveguide and direct the light towards the camera. The at least one coupling element may comprise a diffractive optical element having refractive power. In addition, or alternatively, the at least one coupling optical element may have a coupling area for coupling light into the waveguide, having an average thickness in the range of 0.1 to 3 millimeters across it, and / or a slit-shaped coupling area.

Description

[Technical Field]

[0001] (Cross-reference of related applications) This application claims the benefit of priority to U.S. Provisional Application No. 62 / 737,063, filed on 26 September 2018 and titled "EYEWEAR WITH PINHOLE AND SLIT CAMERAS," and U.S. Provisional Application No. 62 / 737,817, filed on 27 September 2018 and titled "EYEWEAR WITH PINHOLE AND SLIT CAMERAS," respectively, which are incorporated herein by reference in their entirety under 35 U.S. SC § 119(e).

[0002] This disclosure relates to optical devices, including augmented reality imaging and visualization systems. [Background technology]

[0003] Modern computing and display technologies are driving the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or parts thereof are presented to the user in a manner that appears, or can be perceived, as real. Virtual reality, or "VR," scenarios typically involve the presentation of digital or virtual imagery without transparency to other real-world visual inputs, while augmented reality, or "AR," scenarios typically involve the presentation of digital or virtual imagery as an extension of the user's visualization of the real world around them. Mixed reality, or "MR," scenarios are a type of AR scenario that typically involves virtual objects integrated into and responding to the natural world. For example, an MR scenario may include AR imagery content that appears to be obscured by, or is perceived to interact with in a different way than, objects in the real world.

[0004] Referring to Figure 1, an augmented reality scene 10 is depicted. The user of the AR technology sees a real-world park-like setting 20 featuring people, trees, buildings in the background, and a concrete platform 30. The user also perceives "virtual content" such as a robot figure 40 standing on the real-world platform 30 and a flying cartoon-like avatar character 50 that looks like an anthropomorphic bumblebee. These elements 50 and 40 are "virtual" in that they do not exist in the real world. The human visual perception system is complex, and it is difficult to produce AR technology that facilitates a comfortable, natural, and rich presentation of virtual image elements among other virtual or real-world image elements.

[0005] The systems and methods disclosed herein address various challenges related to AR or VR technologies.

[0006] A polarizing beam splitter may be used in a display system to direct polarized light to a light modulator and then direct the light to the viewer. Generally, there is a continuing demand to reduce the size of display systems, and consequently, there is also a demand to reduce the size of components of the display system, including components that utilize polarizing beam splitters. [Overview of the project] [Means for solving the problem]

[0007] The various implementations described herein include display systems configured to provide illumination and / or image projection to the eye. In addition, or alternatively, the display system may image the eye and / or the environment.

[0008] In some embodiments, a head-mounted display system is configured to project light into a user's eyes and display augmented reality image content within the user's field of view. The head-mounted display system may include a frame configured to be supported on the user's head. The display system may also include an image projector configured to project images into the user's eyes and display image content within the user's field of view. The display system may include a camera, at least one waveguide, at least one coupled optical element configured so that light is coupled into and guided within the waveguide, and at least one external coupled element. The at least one external coupled element may be configured to couple the light guided within the waveguide out of the waveguide and direct the light towards the camera. The camera may be positioned in an optical path relative to the at least one external coupled optical element to receive at least a portion of the light coupled into and guided within the waveguide via the coupled element and coupled out of the waveguide by the external coupled coupled element, so that an image can be captured by the camera. The present invention provides, for example, the following: (Item 1) A head-mounted display system configured to project light onto a user's eyes and display augmented reality image content within the user's field of view, wherein the head-mounted display system is A frame configured to be supported on the user's head, An image projector configured to project an image into the user's eyes and display the image content within the user's field of view, At least one camera, At least one waveguide, At least one coupling optical element, wherein the at least one coupling optical element is configured such that light is coupled into and guided within the waveguide, and the at least one coupling optical element comprises a diffractive optical element having a coupling region for coupling light into the waveguide, and the coupling region has an average thickness within a range of 0.1 to 3 millimeters across it, at least one coupling optical element; At least one external coupling element configured to couple light guided within the waveguide out of the waveguide and direct the light towards the camera; Comprising; At least one camera is arranged in the optical path with respect to at least one external coupling optical element to receive at least a part of the light coupled out of the waveguide by the at least one external coupling element, wherein an image can be captured by the at least one camera through the coupling region of the at least one coupling optical element, coupled into and guided within the waveguide. A head-mounted display system. (Item 2) The system according to item 1, wherein the average thickness of the coupling region is within a range of 0.5 to 2 millimeters. (Item 3) The system according to item 1, wherein the average thickness of the coupling region is within a range of 1 to 2 millimeters. (Item 4) The system according to item 1, wherein the coupling region has a rectangular shape. (Item 5) The system according to item 1, wherein the coupling region has an arc shape. (Item 6) The system according to item 1, wherein the coupling region has an aspect ratio within a range of 5 to 100. (Item 7) The system according to item 1, wherein the coupling region has an aspect ratio within a range of 10 to 100. (Item 8) The system according to item 1, wherein the external coupling optical element comprises a diffractive optical element. (Item 9) The system according to item 1, wherein the bonding region of the external bonding optical element has an aspect ratio within the range of 1 to 2. (Item 10) The system according to item 1, wherein the at least one bonding optical element is configured such that light reflected from the eye of the user wearing the head-mounted display system can be coupled into and guided through the at least one waveguide so that an image of the eye can be captured by the at least one camera. (Item 11) A head-mounted display system configured to project light onto a user's eye and display augmented reality image content within the user's field of view, the head-mounted display system comprising: A frame configured to be supported on the user's head; An image projector configured to project an image into the user's eye and display image content within the user's field of view; At least one camera; At least one waveguide; At least one coupling optical element, the at least one coupling optical element being configured such that light is coupled into and guided through the waveguide, the at least one coupling optical element comprising a diffractive optical element having a slit-shaped coupling region for coupling light into the waveguide; At least one external coupling element configured to couple light guided within the waveguide out of the waveguide and direct the light towards the camera; Comprising; The camera is disposed in an optical path with respect to the at least one external coupling optical element to receive at least a portion of the light coupled out of the waveguide by the at least one external coupling element, the light being coupled into and guided through the waveguide via the coupling region of the at least one coupling optical element such that an image can be captured by the camera. (Item 12) The system according to item 11, wherein the bonding region has an average thickness in the range of 0.5 to 3 millimeters. (Item 13) The system according to item 11, wherein the coupling region has a rectangular shape. (Item 14) The system according to item 11, wherein the coupling region has an arc shape. (Item 15) The system according to item 11, wherein the bonding region has a non-arc shape. (Item 16) The system according to item 11, wherein the bonding region has a certain length and width, the length being greater than the width, and the bonding region is a straight line along its length. (Item 17) The system according to item 11, wherein the coupling region has an aspect ratio in the range of 5 to 100. (Item 18) The system according to item 11, wherein the external coupling optical element has a coupling region for coupling light having a thickness of 0.5 mm to 3.0 mm across it out of the waveguide. (Item 19) The system according to item 11, wherein the at least one external coupling optical element has a coupling region for coupling light having an aspect ratio in the range of 1 to 2 out of the waveguide. (Item 20) The system according to item 11, wherein the at least one external coupling element is not a slit and has a coupling region for coupling light out of the waveguide. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 illustrates the user's view of augmented reality (AR) through an AR device.

[0010] [Figure 2] Figure 2 illustrates an embodiment of a wearable display system.

[0011] [Figure 3] Figure 3 illustrates a conventional display system for simulating three-dimensional images for the user.

[0012] [Figure 4] Figure 4 illustrates aspects of an approach to simulating a 3D image using multiple depth planes.

[0013] [Figure 5] Figures 5A-5C illustrate the relationship between the radius of curvature and the radius of focus.

[0014] [Figure 6] Figure 6 illustrates an example of a waveguide stack for outputting image information to the user.

[0015] [Figure 7] Figure 7 illustrates an example of an output beam generated by a waveguide.

[0016] [Figure 8] Figure 8 illustrates an embodiment of a stacked waveguide assembly, where each depth plane includes an image formed using multiple different primary colors.

[0017] [Figure 9A] Figure 9A shows a cross-sectional side view of an embodiment of a stacked set of waveguides, each including an internally coupled optical element. As discussed herein, the stack of waveguides may include an eyepiece.

[0018] [Figure 9B] Figure 9B shows a perspective view of an embodiment of the multiple stacked waveguides shown in Figure 9A.

[0019] [Figure 9C] Figure 9C shows top and bottom plan views of the embodiment of the multiple stacked waveguides shown in Figures 9A and 9B.

[0020] [Figure 10] Figure 10 schematically illustrates a cross-sectional side view of an exemplary imaging system comprising an eyepiece, an image projector, a light source for illuminating the eye, and a camera for capturing an image of the eye.

[0021] [Figure 11A] Figure 11A schematically illustrates how both the light source for illuminating the eye and the image projector for projecting an image into the eye emit light toward the internally coupled optical element on the waveguide of the eyepiece.

[0022] [Figure 11B] Figure 11B schematically illustrates how light projected from a light source and an image projector is coupled into a waveguide.

[0023] [Figure 11C] Figure 11C schematically illustrates how internally coupled light can propagate through a waveguide by total internal reflection.

[0024] [Figure 11D] Figure 11D schematically illustrates how light from the light source and image projector is coupled outwards from the eyepiece.

[0025] [Figure 11E] Figure 11E schematically illustrates a waveguide and a coupled optical element configured to propagate internally coupled light along at least all dimensions of the coupled optical element (e.g., along the x-direction). The light incident on the eye is indicated from the expansion source (e.g., the imaging light will capture a region of the retina).

[0026] [Figure 12A] Figure 12A is a schematic cross-sectional view illustrating the light reflected from the retina, which exits the eye and enters the eyepiece.

[0027] [Figure 12B]Figure 12B schematically illustrates exemplary light coupled into the waveguide of the eyepiece.

[0028] [Figure 12C] Figure 12C schematically illustrates how collimated and internally coupled light from the eye propagates through a waveguide towards an imaging device.

[0029] [Figure 12D] Figure 12D schematically illustrates how internally coupled light from the eye propagates to one or more externally coupled optical elements.

[0030] [Figure 12E] Figure 12E schematically illustrates how light from the eye is coupled out of the waveguide by an external coupling optical element and directed towards a camera, so that an image of the eye (e.g., the retina) can be captured by the camera.

[0031] [Figure 13A] Figure 13A schematically illustrates how an imaging system can focus on various parts of the eye, such as the retina, which can enable the determination of eye orientation and tracking of eye position.

[0032] [Figure 13B] Figure 13B illustrates a pattern of sequentially displayed fixed targets used to direct the eye in various different directions, during which the retina is imaged. The resulting images correspond to different parts of the retina. For example, when the eye is directed in various directions and views fixed targets positioned differently on the display, the images captured by the camera include different parts of the retina. These images can be assembled to form a larger map or composite image of the retina.

[0033] [Figure 14A]Figure 14A schematically illustrates a cross-sectional view of an imaging system comprising an eyepiece and a camera for collecting light from the environment in front of the eyepiece. The light from the environment is shown to be reflected or emitted from one or more physical objects in the environment. Collecting light from objects in the environment in front of the eyepiece can enable the capture of an image of the environment.

[0034] [Figure 14B] Figure 14B schematically illustrates how light from the environment is coupled into the waveguide of the eyepiece by a coupling optical element.

[0035] [Figure 14C] Figure 14C schematically illustrates an imaging system for collecting light from the environment using refractive optical elements such as a refractive optical element in front of the eyepiece (e.g., a wide-angle lens).

[0036] [Figure 15A] Figure 15A schematically illustrates an exemplary imaging system comprising a polarization-selective internal coupling optical element for receiving light from an illumination source and coupling the light into a waveguide within an eyepiece. The eyepiece further includes a polarization-selective optical coupling element for coupling the light out of the waveguide. A polarizer may be used to polarize the light from the illumination source, and a half-wave retarder may be used to rotate the orientation of linear polarization so that it is redirected into the waveguide by the polarization-selective internal coupling optical element.

[0037] [Figure 15B] Figure 15B schematically illustrates how light from the eye (for example, from the retina illuminated by infrared light from a light source) is coupled back into the waveguide and directed towards a camera for image acquisition.

[0038] [Figure 16]Figure 16 schematically illustrates an imaging system configured to image the anterior portion of the eye (e.g., the cornea). The imaging system comprises an eyepiece as described above. The imaging system further includes a positive lens for collimating the light collected from the anterior portion of the eye for propagation to a camera for image acquisition, which is coupled to a waveguide via an optical coupling element. The system further includes a negative lens to offset the positive refractive force introduced by the positive lens and to prevent image inversion of the environment in front of the eyepiece that would otherwise occur due to the positive lens.

[0039] [Figure 17] Figure 17 schematically illustrates another exemplary imaging system configured to image the anterior portion of the eye (e.g., the cornea). The imaging system includes a curved wavelength-selective reflector that collimates light from the anterior portion of the eye for propagation to a camera for image acquisition, coupled in a waveguide via an optical coupling element. The wavelength-selective reflector may operate reflectively for infrared light reflected from the eye and transmissively for visible light from the environment in front of the user.

[0040] [Figure 18] Figure 18 schematically illustrates an exemplary imaging system that similarly includes a curved wavelength-selective reflector that collimates light from the anterior portion of the eye for propagation to a camera for image acquisition, coupled into a waveguide via an optical coupling element. Polarization selectivity may be employed to assist in controlling the path of light reflected from the eye. Eye illumination is provided via the waveguide, instead of multiple light sources between the waveguide and the eye, as shown in Figure 18.

[0041] [Figure 19] Figure 19 schematically illustrates an imaging system that includes a shutter to assist in the procedure for removing noise.

[0042] [Figure 20A]Figures 20A-20E schematically illustrate an alternative procedure for removing noise using wavelength modulation in conjunction with a curved wavelength-selective reflector. [Figure 20B] Figures 20A-20E schematically illustrate an alternative procedure for removing noise using wavelength modulation in conjunction with a curved wavelength-selective reflector. [Figure 20C] Figures 20A-20E schematically illustrate an alternative procedure for removing noise using wavelength modulation in conjunction with a curved wavelength-selective reflector. [Figure 20D] Figures 20A-20E schematically illustrate an alternative procedure for removing noise using wavelength modulation in conjunction with a curved wavelength-selective reflector. [Figure 20E] Figures 20A-20E schematically illustrate an alternative procedure for removing noise using wavelength modulation in conjunction with a curved wavelength-selective reflector.

[0043] [Figure 21] Figure 21 shows an exemplary eyepiece that can be used to project light into the user's eye and provide image content thereto, while simultaneously receiving image data of the user's eye or the environment in front of the user.

[0044] [Figure 22] Figure 22 shows a cross-sectional side view of an example of a cholesteric liquid crystal diffraction grating (CLCG) having multiple uniform chiral structures.

[0045] [Figure 23] Figure 23 illustrates an embodiment of an imaging system comprising a forward-facing camera configured to image the wearer's eye using a cholesteric liquid crystal (CLC) off-axis mirror.

[0046] [Figure 24]Figure 24 shows another exemplary eyepiece that can be used to receive image data of the user's eye or the environment in front of the user, while simultaneously projecting light into the user's eye and providing image content thereto. In this embodiment, the coupled optical element, configured to receive light from the user or the environment in front of the user, is displaced laterally from the image content external coupled optical element (e.g., exit pupil expander).

[0047] [Figure 25] Figure 25 shows another exemplary eyepiece, similar to that shown in Figure 24, where the coupled optical element, configured to receive light from the user or the environment in front of the user, is laterally separated from the external coupled optical element (e.g., the exit pupil expander) of the image content. However, in the implementation shown in Figure 25, the space does not laterally separate the coupled optical element 2111 from the external coupled optical element 2110.

[0048] [Figure 26] Figure 26 shows an exemplary eyepiece including first and second coupling optical elements configured to couple light received from the user's eye or the environment into the waveguide for induction therein, and first and second external coupling optical elements configured to couple the light induced in the waveguide out of the waveguide to one or more cameras.

[0049] [Figure 27] Figure 27 is a cross-section of the waveguide, focusing optical element, and external coupling optical element shown in Figure 26, showing a polarizer in the optical path between one of the external coupling optical elements and the camera. The polarizer may be used to eliminate undesirable flash reflections from the cornea when acquiring an image of the retina.

[0050] [Figure 28]Figures 28A and 28B are front and perspective views of a coupling optical element for coupling light into a waveguide and an external coupling optical element for coupling light out of the waveguide to a camera, respectively, where the coupling optical element has a pinhole coupling region. The external coupling optical element is similarly sized and molded in this implementation.

[0051] [Figure 29] Figures 29A and 29B are front and perspective views of a coupling optical element for coupling light into a waveguide and an external coupling optical element for coupling light out of the waveguide to a camera, respectively. The coupling optical element has an arc-shaped slit coupling region. The external coupling optical element has a pinhole-sized coupling region.

[0052] [Figure 30] Figures 30A and 30B are front and perspective views of a coupling optical element for coupling light into a waveguide and an external coupling optical element for coupling light out of the waveguide to a camera, wherein the coupling optical element has a non-arc-shaped (e.g., straight-rectangular) slit-shaped coupling region. The external coupling optical element has a pinhole-sized coupling region.

[0053] [Figure 31] Figure 31 is a front view of an eyepiece, which includes a coupling optical element for coupling light into a waveguide and an external coupling optical element for coupling light out of the waveguide to a camera, wherein the coupling optical element has a non-arc-shaped (e.g., straight-rectangular) slit-shaped coupling region. The eyepiece further includes an image content internal coupling optical element for receiving light from an image projector and a light distribution element for directing light from the internal coupling optical element to the external coupling optical element for coupling light induced in the waveguide to the user for viewing the image content.

[0054] [Figure 32]Figures 32A and 32B are front and perspective views of a pair of coupling optical elements for coupling light into a waveguide and a pair of external coupling optical elements for coupling light out of the waveguide to a camera, wherein the coupling optical elements have a non-arc-shaped (e.g., straight-rectangular) slit-shaped coupling region. Such a configuration may be useful for imaging different parts of the eye, such as the retina and cornea (or flashes on them).

[0055] [Figure 33] Figure 33 is a front view of an eyepiece, which includes a pair of coupling optical elements for coupling light into a waveguide and a pair of external coupling optical elements for coupling light out of the waveguide to a camera, wherein the coupling optical elements have a non-arc-shaped (e.g., straight-rectangular) slit-shaped coupling region. The eyepiece further includes an image content internal coupling optical element for receiving light from an image projector and a light distribution element for directing the light from the internal coupling optical element to the external coupling optical element for coupling the light induced in the waveguide to the user for viewing the image content.

[0056] [Figure 34] Figures 34A and 34B are front and perspective views of a pair of coupling optical elements for coupling light into a waveguide and a pair of external coupling optical elements for coupling light out of the waveguide to a camera, wherein the coupling optical elements have a non-arc-shaped (e.g., straight-rectangular) slit-shaped coupling region. The arrangement differs from that shown in Figures 32A and 32B.

[0057] [Figure 35] Figure 35 is a front view of an eyepiece, similar to those shown in Figures 30A and 30B, which includes a pair of coupling optical elements for coupling light into a waveguide and a pair of external coupling optical elements for coupling light out of the waveguide to a camera. The eyepiece further includes an image content internal coupling optical element for receiving light from an image projector and a light distribution element for directing the light from the internal coupling optical element to the external coupling optical element for coupling the light induced in the waveguide to the user for viewing the image content.

[0058] The drawings are provided to illustrate exemplary embodiments and are not intended to limit the scope of this disclosure. Similar reference numbers refer to similar parts throughout. [Modes for carrying out the invention]

[0059] Here, the same reference number refers to the same part of the figure throughout.

[0060] Figure 2 illustrates an embodiment of a wearable display system 60. The display system 60 includes a display 70 and various mechanical and electronic modules and systems to support the functions of the display 70. The display 70 may be coupled to a frame 80, which is wearable by a display system user or viewer 90 and is configured to position the display 70 in front of the user 90's eyes. The display 70 may be considered eyewear in some embodiments. In some embodiments, a speaker 100 is coupled to the frame 80 and is configured to be positioned adjacent to the user 90's ear canal (in some embodiments, another speaker, not shown, may optionally be positioned adjacent to the user's other ear canal to provide stereo / shapeable sound control). The display system may also include one or more microphones 110 or other devices to detect sound. In some embodiments, the microphones may be configured to allow the user to provide input or commands to the system 60 (e.g., selection of voice menu commands, natural language questions, etc.) and / or enable audio communication with other persons (e.g., other users of a similar display system). The microphone may also be configured as a peripheral sensor to collect audio data (e.g., sounds from the user and / or the environment). In some embodiments, the display system may also include a peripheral sensor 120a, which is separate from the frame 80 and may be mounted on the user 90's body (e.g., on the user 90's head, torso, limbs, etc.). In some embodiments, the peripheral sensor 120a may be configured to acquire data characterizing the user 90's physiological state. For example, the sensor 120a may be an electrode.

[0061] Continuing to refer to Figure 2, the display 70 is operably coupled to the local data processing module 140 by a communication link 130, such as a wired connection or wireless connectivity, which may be mounted in various configurations, such as being fixedly attached to the frame 80, fixedly attached to a helmet or hat worn by the user, built into headphones, or otherwise detachably attached to the user 90 (e.g., in a backpack configuration, in a belt-mounted configuration). Similarly, the sensor 120a may be operably coupled to the local data processing module 140 by a communication link 120b, such as a wired connection or wireless connectivity. The local processing and data module 140 may include a hardware processor and digital memory such as non-volatile memory (e.g., flash memory or a hard disk drive), both of which may be used to assist in data processing, caching, and storage. The data includes a) data captured from an image capture device (such as a camera), a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a wireless device, a gyroscope, and / or other sensors disclosed herein (e.g., operably coupled to frame 80 or otherwise attached to user 90), and / or b) possibly data acquired and / or processed using a remote processing module 150 and / or a remote data repository 160 (including data related to virtual content) for passage to display 70 after processing or reading. The local processing and data module 140 may be operably coupled to the remote processing module 150 and the remote data repository 160 by communication links 170, 180, such as via wired or wireless links, so that these remote modules 150, 160 are operably coupled to each other and available as resources to the local processing and data module 140. In some embodiments, the local processing and data module 140 may include one or more of the following: an image acquisition device, a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a wireless device, and / or a gyroscope.In some other embodiments, one or more of these sensors may be mounted on the frame 80, or they may be independent structures that communicate with the local processing and data module 140 via a wired or wireless communication path.

[0062] Continuing to refer to Figure 2, in some embodiments, the remote processing module 150 may comprise one or more processors configured to analyze and process data and / or image information. In some embodiments, the remote data repository 160 may comprise a digital data storage facility, which may be available through the internet or other networking configurations in a “cloud” resource configuration. In some embodiments, the remote data repository 160 may comprise one or more remote servers, which provide information, for example, augmented reality content, for generating to the local processing and data module 140 and / or the remote processing module 150. In some embodiments, all data is stored, and all calculations are performed in the local processing and data module, enabling fully autonomous use from the remote module.

[0063] Referring here to Figure 3, the perception of an image as "three-dimensional" or "3-D" can be achieved by providing slightly different presentations of the image to each eye of the viewer. Figure 3 illustrates a conventional display system for simulating a three-dimensional image with respect to a user. Two distinctly different images 190, 200, one for each eye 210, 220, are output to the user. Images 190, 200 are spaced only 230 units apart from eyes 210, 220 along an optical axis or z-axis parallel to the viewer's line of sight. Images 190, 200 are flat, and eyes 210, 220 can focus on the image by taking a single perspective-accommodated state. Such a 3-D display system relies on the human visual system, combines images 190, 200, and provides a perception of depth and / or scale of the combined image.

[0064] However, it should be understood that the human visual system is more complex and providing a realistic perception of depth is more difficult. For example, many viewers of conventional "3-D" display systems may find such systems uncomfortable or may not perceive any sense of depth at all. While not limited by theory, it is thought that viewers of objects may perceive them as "three-dimensional" due to a combination of convergence and divergence movements and accommodation. The convergence and divergence movements of two eyes relative to each other (i.e., the rotation of the eyes, in which the pupils move toward or away from each other to converge the lines of sight of the eyes and fixate on an object) are closely related to the focusing (or "accommodation") of the eye's lens and pupil. Under normal conditions, changing the focus of the eye's lens, or adjusting the eye to shift focus from one object to another at a different distance, will automatically produce a consistent change in convergence-divergence movement up to the same distance, under the relationship known as the "accommodation-convergence-divergence reflex" and pupillary dilation or constriction. Similarly, changes in convergence-divergence movement will, under normal conditions, induce a consistent change in accommodative lens shape and pupil size. As described herein, many stereoscopic or "3-D" display systems display a scene using slightly different presentations (and therefore slightly different images) to each eye so that the three-dimensional viewpoint is perceived by the human visual system. However, such systems, above all, simply provide different presentations of the scene, but are uncomfortable for many viewers because they act against the "accommodation-convergence-divergence reflex" when the eye views all image information in a single accommodated state. A display system that provides a better match between accommodation and convergence-divergence movement can form a more realistic and comfortable simulation of a three-dimensional image.

[0065] Figure 4 illustrates aspects of an approach to simulating a three-dimensional image using multiple depth planes. Referring to Figure 4, objects at various distances from eyes 210, 220 on the z-axis are accommodated by eyes 210, 220 so that those objects are in focus. Eyes 210, 220 take on specific accommodated states, focusing objects at different distances along the z-axis. As a result, a specific accommodated state can be associated with one of the depth planes 240 having an associated focal length such that an object or part of an object in a particular depth plane is in focus when the eye is accommodated to that depth plane. In some embodiments, the three-dimensional image may be simulated by providing different presentations of the image for each eye 210, 220, and by providing different presentations of the image corresponding to each depth plane. For the sake of clarity in the illustration, they are shown as separate, but it should be understood that the fields of view of eyes 210, 220 may overlap, for example, as the distance along the z-axis increases. Furthermore, although shown as flat for the sake of illustration, it should be understood that the contour of the depth plane can be curved in physical space so that all features within the depth plane are in focus with the eye in a particular state of perspective adjustment.

[0066] The distance between an object and the eye 210 or 220 can also change the amount of light diverging from that object as visible to that eye. Figures 5A-5C illustrate the relationship between distance and ray divergence. The distance between the object and the eye 210 is expressed in the order of decreasing distances R1, R2, and R3. As shown in Figures 5A-5C, the rays diverge more as the distance to the object decreases. As the distance increases, the rays become more collimated. In other words, the light field generated by a point (object or part of an object) can be said to have a spherical wavefront curvature, which is a function of the distance the point is from the user's eye. The curvature increases with decreasing distance between the object and the eye 210. Consequently, in different depth planes, the ray divergence is also different, and the divergence increases with decreasing distance between the depth plane and the viewer's eye 210. Only a single eye 210 is illustrated in Figures 5A–5C and various other figures herein for the sake of clarity in the illustration; however, it should be understood that the discussion relating to eye 210 may apply to both eyes 210 and 220 of the viewer.

[0067] While not limited by theory, the human eye is typically thought to be capable of interpreting a finite number of depth planes and providing depth perception. Consequently, a highly realistic simulation of perceived depth can be achieved by providing the eye with different presentations of images corresponding to each of these limited number of depth planes. These different presentations may be used to provide depth cues to the user based on the eye's accommodation required to focus on different image features for scenes located on different depth planes, and / or based on the observation of different image features on different depth planes that are out of focus.

[0068] Figure 6 illustrates an embodiment of a waveguide stack for outputting image information to a user. The display system 250 includes a waveguide stack or stacked waveguide assembly 260, which may be used to provide three-dimensional perception to the eye / brain using a plurality of waveguides 270, 280, 290, 300, 310. In some embodiments, the display system 250 is the system 60 in Figure 2, and Figure 6 shows some parts of that system 60 in more detail. For example, the waveguide assembly 260 may be part of the display 70 in Figure 2. It should be understood that the display system 250 may be considered a light field display in some embodiments. In addition, the waveguide assembly 260 may also be referred to as an eyepiece.

[0069] Continuing with Figure 6, the waveguide assembly 260 may also include several features 320, 330, 340, and 350 between the waveguides. In some embodiments, features 320, 330, 340, and 350 may be one or more lenses. Waveguides 270, 280, 290, 300, and 310 and / or several lenses 320, 330, 340, and 350 may be configured to transmit image information to the eye using various levels of wavefront curvature or ray divergence. Each waveguide level may be associated with a specific depth plane and may be configured to output image information corresponding to that depth plane. Image input devices 360, 370, 380, 390, and 400 may function as light sources for the waveguides and may be used to input image information into waveguides 270, 280, 290, 300, and 310, each of which may be configured to disperse incident light across each individual waveguide for output toward the eye 210, as described herein. The light exits from the output surfaces 410, 420, 430, 440, and 450 of image input devices 360, 370, 380, 390, and 400 and is input into the corresponding input surfaces 460, 470, 480, 490, and 500 of waveguides 270, 280, 290, 300, and 310. In some embodiments, the input surfaces 460, 470, 480, 490, and 500 may each be the edge of the corresponding waveguide or a portion of the main surface of the corresponding waveguide (i.e., one of the waveguide surfaces that directly faces the world 510 or the viewer's eye 210). In some embodiments, a single beam of light (e.g., a collimated beam) may be injected into each waveguide and output a whole field of cloned collimated beams directed toward the eye 210 at a specific angle (and divergence) corresponding to the depth plane associated with the particular waveguide. In some embodiments, one of the image input devices 360, 370, 380, 390, and 400 may be associated with a plurality (e.g., three) of waveguides 270, 280, 290, 300, and 310, into which light may be injected.

[0070] In some embodiments, the image input devices 360, 370, 380, 390, and 400 are discrete displays that each generate image information for input into the corresponding waveguides 270, 280, 290, 300, and 310, respectively. In some other embodiments, the image input devices 360, 370, 380, 390, and 400 are output terminals of a single multiplexed display that can send image information to each of the image input devices 360, 370, 380, 390, and 400, for example, via one or more optical conduits (such as optical fiber cables). It should be understood that the image information provided by the image input devices 360, 370, 380, 390, and 400 may include light of different wavelengths or colors (e.g., different primary colors, as discussed herein).

[0071] In some embodiments, the light introduced into waveguides 270, 280, 290, 300, and 310 is provided by an optical projector system 520, which comprises an optical module 540, which may include an optical emitter such as a light-emitting diode (LED). The light from the optical module 540 may be directed and modified via a beam splitter 550 by an optical modulator 530, such as a spatial light modulator. The optical modulator 530 may be configured to change the perceived intensity of the light introduced into waveguides 270, 280, 290, 300, and 310. Embodiments of the spatial light modulator include liquid crystal displays (LCDs) and liquid crystal on silicon (LCOS) displays. Image input devices 360, 370, 380, 390, and 400 are graphically illustrated, and it should be understood that in some embodiments, these image input devices may represent different optical paths and locations within a common projection system, configured to output light into associated waveguides 270, 280, 290, 300, and 310.

[0072] In some embodiments, the display system 250 may be a scanning fiber display comprising one or more scanning fibers configured to project light in various patterns (e.g., raster scanning, helical scanning, Lissajous patterns, etc.) into one or more waveguides 270, 280, 290, 300, 310, and ultimately to the viewer's eye 210. In some embodiments, the illustrated image input devices 360, 370, 380, 390, 400 may schematically represent a single scanning fiber or a bundle of scanning fibers configured to input light into one or more waveguides 270, 280, 290, 300, 310. In some other embodiments, the illustrated image input devices 360, 370, 380, 390, 400 may schematically represent a plurality of scanning fibers or a plurality of bundles of scanning fibers, each configured to input light into one of the associated waveguides 270, 280, 290, 300, 310. It should be understood that one or more optical fibers may be configured to transmit light from the optical module 540 to one or more waveguides 270, 280, 290, 300, and 310. It should be understood that one or more intervening optical structures may be provided between the scanning fiber or multiple fibers and one or more waveguides 270, 280, 290, 300, and 310, for example, to redirect light emitted from the scanning fiber into one or more waveguides 270, 280, 290, 300, and 310.

[0073] The controller 560 controls the operation of one or more of the stacked waveguide assemblies 260, including the operation of the image input devices 360, 370, 380, 390, 400, the light source 540, and the optical module 530. In some embodiments, the controller 560 is part of the local data processing module 140. The controller 560 includes programming (e.g., instructions in a non-transient medium) to coordinate the timing and provisioning of image information to the waveguides 270, 280, 290, 300, 310, for example, according to any of the various schemes disclosed herein. In some embodiments, the controller may be a single integrated device or a distributed system connected by wired or wireless communication channels. In some embodiments, the controller 560 may be part of the processing module 140 or 150 (Figure 2).

[0074] Continuing with Figure 6, waveguides 270, 280, 290, 300, and 310 may be configured to propagate light within each individual waveguide by total internal reflection (TIR). Waveguides 270, 280, 290, 300, and 310 may each be planar or have another shape (e.g., curved), with a main upper surface and a main bottom surface and edges extending between their main upper and main bottom surfaces. In the illustrated configuration, waveguides 270, 280, 290, 300, and 310 may each include external coupling optical elements 570, 580, 590, 600, and 610, respectively, configured to extract light from the waveguide by redirecting the light propagating within each individual waveguide out of the waveguide and outputting image information to the eye 210. The extracted light may also be referred to as externally coupled light, and the external coupling optical elements may also be referred to as light extraction optical elements. The extracted beam of light can be output by the waveguide at the point where light propagating within the waveguide strikes the light extraction optical element. The external coupling optical elements 570, 580, 590, 600, 610 may be gratings, for example, including diffractive optical features as further discussed herein. For the sake of clarity and to illustrate the drawings, they are shown positioned on the bottom main surfaces of the waveguides 270, 280, 290, 300, 310, but in some embodiments, the external coupling optical elements 570, 580, 590, 600, 610 may be positioned on the upper main surfaces and / or the bottom main surfaces, and / or directly within the volume of the waveguides 270, 280, 290, 300, 310, as further discussed herein. In some embodiments, the external coupling optical elements 570, 580, 590, 600, 610 may be mounted on a transparent substrate and formed within a layer of material that forms the waveguides 270, 280, 290, 300, 310. In some other embodiments, the waveguides 270, 280, 290, 300, 310 may be a monolithic piece of material, and the external coupling optical elements 570, 580, 590, 600, 610 may be formed on and / or inside the material piece.

[0075] Continuing with reference to Figure 6, as discussed herein, each waveguide 270, 280, 290, 300, 310 is configured to emit light and form an image corresponding to a specific depth plane. For example, the waveguide 270 closest to the eye may be configured to deliver collimated light (injected into such waveguide 270) to the eye 210. The collimated light may represent the optical infinity focal plane. The next upper waveguide 280 may be configured to emit collimated light that passes through a first lens 350 (e.g., a negative lens) before reaching the eye 210. Such a first lens 350 may generate some convex wavefront curvature so that the eye / brain interprets the light originating from the next upper waveguide 280 as originating from a first focal plane closer inward from optical infinity toward the eye 210. Similarly, the third upper waveguide 290 passes its output light through both the first lens 350 and the second lens 340 before reaching the eye 210. The combined refractive power of the first lens 350 and the second lens 340 may be configured to produce a different, gradually increasing wavefront curvature so that the eye / brain interprets the light emanating from the third waveguide 290 as originating from a second focal plane that is even closer inward toward the person from optical infinity than the light from the next upper waveguide 280.

[0076] Other waveguide layers 300, 310 and lenses 330, 320 are configured similarly, with the highest waveguide 310 in the stack emitting its output through all the lenses between it and the eye for a convergent focusing force representing the focal plane closest to the person. When viewing / interpreting light originating from the other side world 510 of the stacked waveguide assembly 260, a compensating lens layer 620 may be positioned on top of the stack to compensate for the convergent forces of the lower lens stacks 320, 330, 340, 350 to compensate for the stack of lenses 320, 330, 340, 350. Such a configuration provides the same number of perceived focal planes as there are available waveguide / lens pairs. Both the external coupling optical elements of the waveguides and the focusing sides of the lenses may be static (i.e., not dynamic or electroactive). In some alternative embodiments, one or both may be dynamic using electroactive features.

[0077] In some embodiments, two or more of the waveguides 270, 280, 290, 300, and 310 may have the same associated depth plane. For example, multiple waveguides 270, 280, 290, 300, and 310 may be configured to output images set in the same depth plane, or multiple subsets of waveguides 270, 280, 290, 300, and 310 may be configured to output images set in the same multiple depth planes, with one set for each depth plane. This may offer the advantage of forming tiled images that provide an extended field of view in those depth planes.

[0078] Continuing with Figure 6, the external coupling optical elements 570, 580, 590, 600, and 610 may be configured to redirect light from their individual waveguides for specific depth planes associated with the waveguides and to output the light with an appropriate amount of divergence or collimation. As a result, waveguides having different associated depth planes may have external coupling optical elements 570, 580, 590, 600, and 610 with different configurations, outputting light with different amounts of divergence depending on the associated depth plane. In some embodiments, the light extraction optical elements 570, 580, 590, 600, and 610 may be stereoscopic or surface features that can be configured to output light at specific angles. For example, the light extraction optical elements 570, 580, 590, 600, and 610 may be stereoscopic holograms, surface holograms, and / or diffraction gratings. In some embodiments, the features 320, 330, 340, and 350 may not be lenses. Rather, they may simply be spacers (e.g., cladding layers and / or structures for forming voids).

[0079] In some embodiments, the external coupling optical elements 570, 580, 590, 600, 610 are diffraction features or “diffractive optical elements” (also referred to herein as “DOEs”) that form a diffraction pattern. Preferably, the DOEs have relatively low diffraction efficiency such that only a portion of the beam light is deflected toward the eye 210 at each intersection of the DOEs, while the remainder continues to travel through the waveguide via the TIR. The light carrying the image information is therefore split into several associated emission beams that exit the waveguide at multiple locations, resulting in a very uniform pattern of emission toward the eye 210 with respect to this particular collimated beam bouncing within the waveguide.

[0080] In some embodiments, one or more DOEs may be switchable between an "on" state in which they actively diffract and an "off" state in which they do not significantly diffract. For example, a switchable DOE may comprise a layer of polymer-dispersed liquid crystal where microdroplets have a diffraction pattern within the host medium, and the refractive index of the microdroplets may be switched to substantially match the refractive index of the host material (in which case the pattern does not significantly diffract incident light), or the microdroplets may be switched to a refractive index that does not match that of the host medium (in which case the pattern actively diffracts incident light).

[0081] In some embodiments, a camera assembly 630 (e.g., a digital camera including visible light and infrared light cameras) may be provided to capture images of the eye 210 and / or the surrounding tissues, for example, to detect user input and / or monitor the user's physiological state. As used herein, the camera may be any image-capturing device. In some embodiments, the camera assembly 630 may include the image-capturing device and a light source that projects light (e.g., infrared light) onto the eye, and the light is then reflected by the eye and can be detected by the image-capturing device. In some embodiments, the camera assembly 630 may be mounted on a frame 80 (Figure 2) and may communicate with processing modules 140 and / or 150 that can process image information from the camera assembly 630. In some embodiments, one camera assembly 630 may be used per eye to monitor each eye separately.

[0082] Referring here to Figure 7, an embodiment of an outgoing beam output by a waveguide is shown. Although one waveguide is illustrated, other waveguides within the waveguide assembly 260 (Figure 6) may function similarly, and it should be understood that the waveguide assembly 260 includes multiple waveguides. Light 640 is introduced into the waveguide 270 at the input surface 460 of the waveguide 270 and propagates through the waveguide 270 by TIR. At the point where the light 640 collides on the DOE 570, a portion of the light exits the waveguide as an outgoing beam 650. The outgoing beam 650 is illustrated as substantially parallel, but may be redirected to propagate to the eye 210 at a certain angle (e.g., forming a divergent outgoing beam), depending on the depth plane associated with the waveguide 270, as discussed herein. It should be understood that a nearly parallel emitted beam may represent a waveguide with an externally coupled optical element that externally couples the light to form an image that appears to be set in the depth plane at a distance from the eye 210 (e.g., optical infinity). Other waveguides or other sets of externally coupled optical elements may output a more divergent emitted beam pattern, which would require the eye 210 to adjust to a closer distance and focus onto the retina, and would be interpreted by the brain as light from a distance closer to the eye 210 than optical infinity.

[0083] In some embodiments, a full-color image may be formed in each depth plane by overlaying an image onto each of primary colors, for example, three or more primary colors. Figure 8 illustrates an embodiment of a stacked waveguide assembly, where each depth plane includes an image formed using several different primary colors. The illustrated embodiment shows depth planes 240a–240f, but more or fewer depths may also be considered. Each depth plane may have three or more associated primary color images, including a first image of a first color G, a second image of a second color R, and a third image of a third color B. Different depth planes are indicated in the figure by different numbers relating to diopters (dpt) following the letters G, R, and B. As merely an embodiment, the numbers following each of these letters indicate diopters (1 / m), i.e., the inverse distance of the depth plane from the viewer, and each box in the figure represents an individual primary color image. In some embodiments, the precise location of the depth plane for different primary colors may vary to account for differences in the focusing of light of different wavelengths in the eye. For example, different primary color images for a given depth plane may be placed on depth planes corresponding to different distances from the user. Such arrangements may increase visual acuity and user comfort and / or reduce chromatic aberration.

[0084] In some embodiments, each primary color light may be output by a single dedicated waveguide, and as a result, each depth plane may have multiple waveguides associated with it. In such embodiments, each box in the figure, including the letters G, R, or B, can be understood to represent an individual waveguide, and three waveguides may be provided for each depth plane, and three primary color images are provided for each depth plane. The waveguides associated with each depth plane are shown adjacent to each other in this drawing for ease of explanation, but it should be understood that in a physical device, all waveguides may be arranged in a stack with one waveguide per level. In some other embodiments, multiple primary colors may be output by the same waveguide, for example, so that only a single waveguide may be provided for each depth plane.

[0085] Continuing to refer to Figure 8, in some embodiments, G is green, R is red, and B is blue. In some other embodiments, other colors associated with other wavelengths of light, including magenta and cyan, may be used in addition to or replace one or more of red, green, or blue.

[0086] Throughout this disclosure, any reference to a given color of light should be understood as encompassing one or more wavelengths of light within a range of wavelengths that are perceived by the viewer as that given color. For example, red light may include one or more wavelengths of light in the range of approximately 620–780 nm, green light may include one or more wavelengths of light in the range of approximately 492–577 nm, and blue light may include one or more wavelengths of light in the range of approximately 435–493 nm.

[0087] In some embodiments, the light source 540 (Figure 6) may be configured to emit light of one or more wavelengths outside the viewer's visual perception range, such as infrared and / or ultraviolet wavelengths. In addition, internal coupling, external coupling, and other light redirection structures of the waveguide of the display 250 may be configured to direct and emit this light from the display towards the user's eye 210, for example, for imaging and / or user stimulation applications.

[0088] Referring here to Figure 9A, in some embodiments, light impacting a waveguide may need to be redirected to internally couple that light into the waveguide. Internal coupling optical elements may be used to redirect and internally couple the light into its corresponding waveguide. Figure 9A illustrates cross-sectional side views of embodiments of multiple or set 660 stacked waveguides, each including an internal coupling optical element. Each waveguide may be configured to output light of one or more different wavelengths or one or more different wavelength ranges. Stack 660 may correspond to stack 260 (Figure 6), and the illustrated waveguides of stack 660 may correspond to a portion of multiple waveguides 270, 280, 290, 300, 310, but it should be understood that light from one or more of the image input devices 360, 370, 380, 390, 400 is input into the waveguide from a position where the light is required to be redirected for internal coupling.

[0089] The illustrated set of stacked waveguides 660 includes waveguides 670, 680, and 690. Each waveguide includes associated internal coupling optical elements (which may also be referred to as optical input areas on the waveguide), for example, internal coupling optical element 700 is located on the main surface of waveguide 670 (e.g., the upper main surface), internal coupling optical element 710 is located on the main surface of waveguide 680 (e.g., the upper main surface), and internal coupling optical element 720 is located on the main surface of waveguide 690 (e.g., the upper main surface). In some embodiments, one or more of the internal coupling optical elements 700, 710, and 720 may be located on the bottom main surfaces of individual waveguides 670, 680, and 690 (in particular, one or more internal coupling optical elements are reflective deflection optical elements). As illustrated, the internally coupled optical elements 700, 710, and 720 may be located on the upper main surface of their respective waveguides 670, 680, and 690 (or on the upper part of the following lower waveguide), and in particular, these internally coupled optical elements are transmissive deflection optical elements. In some embodiments, the internally coupled optical elements 700, 710, and 720 may be located within the body of the respective waveguides 670, 680, and 690. In some embodiments, as discussed herein, the internally coupled optical elements 700, 710, and 720 are wavelength-selective, selectively redirecting one or more wavelengths of light while transmitting other wavelengths of light. Although illustrated on one side or corner of their respective waveguides 670, 680, and 690, it should be understood that in some embodiments, the internally coupled optical elements 700, 710, and 720 may be located within other areas of their respective waveguides 670, 680, and 690.

[0090] As illustrated, the internally coupled optical elements 700, 710, and 720 may be offset laterally from one another. In some embodiments, each internally coupled optical element may be offset so that its light does not pass through another internally coupled optical element before receiving light. For example, each internally coupled optical element 700, 710, and 720 may be configured to receive light from different image input devices 360, 370, 380, 390, and 400, as shown in Figure 6, and may be separated from the other internally coupled optical elements 700, 710, and 720 (e.g., separated laterally) so that it does not substantially receive light from the other internally coupled optical elements 700, 710, and 720.

[0091] Each waveguide also includes associated optical dispersion elements, for example, optical dispersion element 730 is located on the main surface (e.g., upper main surface) of waveguide 670, optical dispersion element 740 is located on the main surface (e.g., upper main surface) of waveguide 680, and optical dispersion element 750 is located on the main surface (e.g., upper main surface) of waveguide 690. In some other embodiments, optical dispersion elements 730, 740, and 750 may be located on the bottom main surfaces of the associated waveguides 670, 680, and 690, respectively. In some other embodiments, optical dispersion elements 730, 740, and 750 may be located on both the top and bottom main surfaces of the associated waveguides 670, 680, and 690, respectively, or optical dispersion elements 730, 740, and 750 may be located on different top and bottom main surfaces within different associated waveguides 670, 680, and 690, respectively.

[0092] Waveguides 670, 680, and 690 may be separated and isolated by, for example, gaseous, liquid, and / or solid layers of material. For example, as shown, layer 760a may separate waveguides 670 and 680, and layer 760b may separate waveguides 680 and 690. In some embodiments, layers 760a and 760b are formed from a low refractive index material (i.e., a material having a lower refractive index than the material forming the immediate vicinity of waveguides 670, 680, and 690). Preferably, the refractive index of the material forming layers 760a and 760b is 0.05 or greater, or 0.10 or less, than the refractive index of the material forming waveguides 670, 680, and 690. Advantageously, the lower refractive index layers 760a, 760b may function as cladding layers that promote total internal reflection (TIR) ​​of light through the waveguides 670, 680, 690 (e.g., TIR between the upper and bottom primary surfaces of each waveguide). In some embodiments, layers 760a, 760b are formed from air. It should be understood that, although not shown, the upper and bottom of the illustrated set 660 waveguides may also include an immediate cladding layer.

[0093] Preferably, to facilitate manufacturing and other considerations, the materials forming waveguides 670, 680, and 690 are similar or identical, and the materials forming layers 760a and 760b are similar or identical. In some embodiments, the materials forming waveguides 670, 680, and 690 may differ between one or more waveguides, and / or the materials forming layers 760a and 760b may differ, while still maintaining the various refractive index relationships described above.

[0094] Continuing to refer to Figure 9A, rays 770, 780, and 790 are incident on the waveguide set 660. It should be understood that rays 770, 780, and 790 may also be introduced into waveguides 670, 680, and 690 by one or more image input devices 360, 370, 380, 390, and 400 (Figure 6).

[0095] In some embodiments, the rays 770, 780, and 790 may have different properties, such as different wavelengths or different wavelength ranges, which may correspond to different colors. The internal coupling optical elements 700, 710, and 720 each deflect the incident light so that the light propagates through one of the waveguides 670, 680, and 690 by TIR. In some embodiments, the internal coupling optical elements 700, 710, and 720 each selectively deflect one or more specific wavelengths of light while allowing other wavelengths to pass through the lower waveguide and associated internal coupling optical elements.

[0096] For example, the internally coupled optical element 700 may be configured to transmit rays 780 and 790 having different second and third wavelengths or wavelength ranges, respectively, while deflecting a ray 770 having a first wavelength or wavelength range. The transmitted ray 780 collides with an internally coupled optical element 710 configured to selectively deflect light of the second wavelength or wavelength range, and is thereby deflected. The ray 790 is deflected by an internally coupled optical element 720 configured to selectively deflect light of the third wavelength or wavelength range.

[0097] Continuing with Figure 9A, the deflected rays 770, 780, and 790 are deflected so that they propagate through the corresponding waveguides 670, 680, and 690. That is, the internal coupling optical elements 700, 710, and 720 of each waveguide deflect the light into its corresponding waveguide 670, 680, and 690, and internally couple the light into the corresponding waveguide. The rays 770, 780, and 790 are deflected at an angle that causes the light to propagate through the individual waveguides 670, 680, and 690 by TIR. The rays 770, 780, and 790 propagate through the individual waveguides 670, 680, and 690 by TIR until they collide with the corresponding optical dispersion elements 730, 740, and 750 of the waveguide.

[0098] Referring now to Figure 9B, a perspective view of an embodiment of the multiple stacked waveguides shown in Figure 9A is illustrated. As described above, the internally coupled rays 770, 780, and 790 are deflected by the internally coupled optical elements 700, 710, and 720, respectively, and then propagate by TIR within waveguides 670, 680, and 690, respectively. The rays 770, 780, and 790 then collide with the optical dispersion elements 730, 740, and 750, respectively. The optical dispersion elements 730, 740, and 750 deflect the rays 770, 780, and 790 so that they propagate toward the externally coupled optical elements 800, 810, and 820, respectively.

[0099] In some embodiments, the light dispersion elements 730, 740, and 750 are orthogonal pupil expanders (OPEs). In some embodiments, the OPEs deflect or disperse light to the external coupling optical elements 800, 810, and 820, and in some embodiments, they can also increase the beam or spot size of the light as it propagates to the external coupling optical elements. In some embodiments, the light dispersion elements 730, 740, and 750 may be omitted, and the internal coupling optical elements 700, 710, and 720 may be configured to deflect light directly to the external coupling optical elements 800, 810, and 820. For example, referring to Figure 9A, the light dispersion elements 730, 740, and 750 may be replaced by the external coupling optical elements 800, 810, and 820, respectively. In some embodiments, the external coupling optical elements 800, 810, 820 are exit pupils (EPs) or exit pupil expanders (EPEs) that direct light towards the viewer's eye 210 (Figure 7). It should be understood that the OPEs may be configured to increase the dimensions of the eyebox along at least one axis, and the EPEs may increase the eyebox along axes that intersect with the axes of the OPEs, for example, orthogonal axes. For example, each OPE may be configured to redirect a portion of the light impacting the OPE to an EPE in the same waveguide, while allowing the rest of the light to continue propagating along the waveguide. In response to the impact on the OPE, another portion of the remaining light is again redirected to the EPE, and the rest of that portion continues to propagate further along the waveguide, etc. Similarly, in response to the impact on the EPE, a portion of the impacting light is directed out of the waveguide towards the user, and the rest of that light continues to propagate through the waveguide until it impacts the EP again, at which point another portion of the impacting light is directed out of the waveguide, and so on. As a result, the internally coupled single beam of light is "duplicated" each time a portion of its light is redirected by the OPE or EPE, thereby forming a cloned beam field of light, as shown in Figure 6. In some embodiments, the OPE and / or EPE may be configured to modify the size of the beam of light.

[0100] Therefore, referring to Figures 9A and 9B, in some embodiments, the set of waveguides 660 includes, for each primary color, waveguides 670, 680, 690, internally coupled optical elements 700, 710, 720, optical dispersion elements (e.g., OPE) 730, 740, 750, and externally coupled optical elements (e.g., EP) 800, 810, 820. Waveguides 670, 680, 690 may be stacked with air gaps / cladding layers between each one. The internally coupled optical elements 700, 710, 720 redirect or deflect the incident light into their waveguides (using different internally coupled optical elements that receive light of different wavelengths). The light then propagates within the individual waveguides 670, 680, 690 at angles that will result in a TIR. In the embodiment shown, a ray 770 (e.g., blue light) is polarized by the first internal coupling optical element 700 in the manner described above, and then continues to bounce along the waveguide, interacting with the optical dispersion element (e.g., OPE) 730 and then the external coupling optical element (e.g., EP) 800. Rays 780 and 790 (e.g., green and red light, respectively) pass through waveguide 670, with ray 780 colliding with the internal coupling optical element 710, thereby being deflected. Ray 780 will then bounce along waveguide 680 via TIR, proceeding to its optical dispersion element (e.g., OPE) 740 and then the external coupling optical element (e.g., EP) 810. Finally, ray 790 (e.g., red light) passes through waveguide 690 and colliding with the optical internal coupling optical element 720 of waveguide 690. The internal optical coupling element 720 deflects the ray 790 so that it propagates by TIR to the optical dispersion element (e.g., OPE) 750, and then by TIR to the external coupling optical element (e.g., EP) 820. The external coupling optical element 820 then finally externally couples the ray 790 to the viewer, who also receives externally coupled light from the other waveguides 670, 680.

[0101] Figure 9C illustrates upper and lower plan views of embodiments of the multiple stacked waveguides shown in Figures 9A and 9B. As shown, waveguides 670, 680, and 690 may be vertically aligned with their associated optical dispersion elements 730, 740, and 750 and associated external coupling optical elements 800, 810, and 820. However, as discussed herein, the internal coupling optical elements 700, 710, and 720 are not vertically aligned. Rather, the internal coupling optical elements are preferably non-overlapping (e.g., laterally spaced, as seen in the upper and lower figures). As further discussed herein, this non-overlapping spatial arrangement facilitates the ingress of light from different resources into different waveguides on a one-to-one basis, thereby enabling a specific light source to be uniquely coupled to a specific waveguide. In some embodiments, arrangements including non-overlapping, spatially separated internal coupling optical elements may be referred to as pupil-shifting systems, where the internal coupling optical elements in these arrangements may correspond to subpupils. Eye image formation and environmental image formation

[0102] As discussed above, head-mounted displays can be used to provide a user with image content that is integrated with, combined with, and / or superimposed over, the wearer's view of the world in front of them. Such a head-mounted display system can be configured to project light into the user's eyes to form augmented reality image content and to transmit light from the environment in front of the user to the user. The head-mounted display system may include one or more cameras to image the environment and / or the user's eyes. Outward-facing cameras may be used to directly image the environment and, for example, to determine where augmented reality image content should be placed relative to objects in the environment. For example, imaging the environment may provide the location of a table so that the head-mounted display can render an image of a person standing next to a table instead of on or inside the table. Inward-facing cameras may be used to directly image the eyes, for example, for eye tracking. Disclosed herein are embodiments of head-mounted display systems and / or imaging systems that can similarly be configured to image the eyes and / or the environment. In some designs, the system does not require inward-facing and / or outward-facing cameras to directly image the eye and / or environment, respectively. Such a system may employ one or more cameras configured to receive light from the eye / environment via an eyepiece, such as one or more waveguides in an eyepiece, which are in optical communication with one or more cameras. Once the light is collected by the waveguide, the one or more cameras can generate an image of the eye and / or the environment in front of the user. Using waveguides to collect light for imaging the eye and / or environment can potentially reduce the shape factor of the head-mounted display, making the head-mounted display potentially more compact and / or aesthetically desirable.

[0103] Figure 10 illustrates an exemplary imaging system 900, which may be used on a head-mounted display, and is integrated with an eyepiece 950, configured to image the eye. The eyepiece 950, which may be positioned in front of the user's eye 210, can be used both to load image content into the eye and to image the eye. Figure 10 shows one eyepiece 950 in front of one eye 210. Various head-mounted display systems, such as those shown in Figure 2, may include a pair of eyepieces 950 and associated components, positioned in front of separate left and right eyes 210. A single waveguide 940 is shown in Figure 10, but the waveguide 940 may include one, two, three, four, six, seven, eight, or more waveguides (e.g., a stack of one or more waveguides).

[0104] The imaging system 900 may include an eyepiece 950 comprising a light source or illumination source 960 for illuminating the eye and facilitating image acquisition, a waveguide 940 configured to propagate light therein, and an imaging device 920 such as a camera for image acquisition. An image projector 930 for producing an image, which can be projected into the eye via the eyepiece 950, is also shown. The eyepiece 950 may include one or more waveguides 940 configured to carry light from the illumination source 960 and / or the image projector 930 to the eye and light from the eye to the camera 920. The eyepiece 950 may further include one or more coupling optical elements 944 for coupling light out of the waveguide 940 to the eye for illuminating the eye and for image acquisition, and / or from the eye into the waveguide for image acquisition. The eyepiece 950 may also include one or more internal coupling optical elements 942 for coupling light from the illumination source 960 and / or the image projector 930 into the waveguide 940, and one or more external coupling optical elements 952 for coupling light from the waveguide to the camera 920.

[0105] The eyepiece 950 may be mounted on a frame that is attached to the head. The eyepiece 950 may be positioned in front of the eye 210. The eyepiece 950 may have an inner or nasal side, closer to the wearer's nose, and an opposite outer or temple side, closer to the wearer's temple and further from the nose. In Figure 10, the combined optical element 944 is inner or nasal (outer or temple) relative to the internal combined optical element 942 and the external combined optical element 952 (outer or temple) relative to the combined optical element 944. The illumination source 960 is also inner or nasal relative to the image projector 930 (or the image projector is outer or temple) relative to the illumination source. However, the relative positions may differ. For example, in some designs, the illumination source 960 may be outer or temple relative to the image projector 930.

[0106] The waveguide 940 may comprise a sheet or layer having two main surfaces (front and rear surfaces) having the maximum surface area, arranged opposite to each other. The front surface may be further from the user's eyes 210 (closer to the environment in front of the wearer) when the user wears the head-mounted display, and the rear surface may be closer to the user's eyes (and further from the environment in front of the wearer). The waveguide 940 may include a transparent material (e.g., glass, plastic) with a refractive index greater than 1.0 so that light can be guided within it between the main surfaces by total internal reflection. Elements with the same number may have the same functionality for one or more of the embodiments described herein.

[0107] A coupling optical element 944 for coupling light from the waveguide 940 to the eye 210 and / or from the waveguide to the eye may be located on or inside the waveguide 940. As shown in Figure 10, the coupling optical element 944 may be located in the optical path between the user's eye 210 and the waveguide 940 so that light coupled from the waveguide 940 via the coupling optical element 944 can be incident on the user's eye 210 (e.g., for illuminating the eye and / or for image loading). The coupling optical element 944 may have a plurality of redirection features configured to redirect light incident on the coupling optical element 944 into the waveguide at a certain angle so that light induced in the waveguide is redirected out of the waveguide or is induced therein by total internal reflection. The coupling optical element 944 and the redirection features may be physically engaged with the waveguide 940. For example, the coupled optical element 944 may comprise a patterned (e.g., etched) holographic or diffractive optical element (e.g., a surface relief grating) within or on the waveguide 940. The coupled optical element 944 may comprise a layer disposed on or formed within the waveguide 940. For example, a volume holographic or other diffractive optical element may be formed by varying the refractive index of the material constituting the waveguide or a layer disposed on or on it. Thus, the coupled optical element 944 may be disposed within the volume of the waveguide 940 or as a layer disposed on or on it.

[0108] Depending on the design, the coupled optical element 944 may be transmissive or reflective, and may operate transmissively or reflectively. For example, the coupled optical element 944 may each operate transmissively or reflectively and include, for example, a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that redirects light transmitted through or reflected from it. The coupled optical element 944 may include a polarization optical element such as a polarization-selective redirection element (e.g., a polarizer). The polarization-selective redirection element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may comprise a liquid crystal structure such as a liquid crystal polarization grating. The coupled optical element 944 may be configured to direct light from the image projector 930 and / or light source 960, which is induced in the waveguide 940 by total internal reflection (TIR), towards the user's eye 210 at an angle less than the critical angle (e.g., a more perpendicular angle) so that it is emitted out of the waveguide to the eye. In addition, or alternatively, the coupling optical element 944 may be configured to couple light from the eye 210 into the waveguide 940 at an angle above the critical angle (e.g., an angle that is not very perpendicular) so that it is guided in the camera 920 by total internal reflection.

[0109] As shown in Figure 10, an internal coupling optical element 942 for coupling light from the illumination source 960 and / or the image projector 930 into the waveguide 940 may be located on or inside the waveguide 940. The internal coupling optical element 942 may be located in the optical path between the light source 960 and the waveguide 940 so that the light coupled from the light source 960 via the internal coupling optical element 942 is guided into the waveguide 940. The internal coupling optical element 942 may have a plurality of redirection features configured to redirect light incident on it into the waveguide at a certain angle, for example, so that it is guided therein by total internal reflection. The internal coupling optical element 942 may include a liquid crystal structure such as a liquid crystal polarizing grating. In addition, or alternatively, the internal coupling optical element 942 may include a blazed grating. The internal coupling optical element 942 may comprise a layer located on the waveguide 940, or may be formed on or within the waveguide 940 (e.g., patterned), or otherwise manufactured therein. For example, a surface holographic or diffractive optical element (e.g., a surface relief grating) may be fabricated by patterning (e.g., etching) the surface of the waveguide or a layer located thereon. A volumetric holographic or diffractive optical element may also be formed by changing the refractive index of the material constituting the waveguide or a layer located thereon. Thus, the internal coupling optical element 942 may be located within the volume of the waveguide 940 or a layer located thereon. Depending on the design, the internal coupling optical element 942 may be transmissive or reflective, and may operate transmissively or reflectively. For example, the internal coupling optical element 942 may include a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates transmissively or reflectively, respectively, and, for example, redirects light transmitted through or reflected from it.

[0110] The internally coupled optical element 942 may include a reflective optical element (e.g., a mirror). For example, the internally coupled optical element 942 may include an off-axis reflector. In addition, or alternatively, the internally coupled optical element 942 and / or the coupled optical element 944 may include a polarization optical element such as a polarization-selective diversion element (e.g., a polarizer). The polarization-selective diversion element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may include a liquid crystal structure such as a liquid crystal polarization grating. For example, one or both of the internally coupled optical element 942 and / or the coupled optical element 944 may include a liquid crystal polarization grating (LCPG). An LCPG can potentially provide high-efficiency diffraction over a wide wavelength range. Therefore, an LCPG may be useful for the internally coupled optical element 942 and / or the coupled optical element 944. The LCPG may be polarization-dependent. LCPG or other types of liquid crystal gratings, diffractive optical elements, or optical elements may include patterns or arrangements of liquid crystal molecules configured to provide one or more functions, such as redirecting light into or out of a waveguide. Thus, the internally coupled optical element 942 and / or coupled optical element 944 may comprise a polarizing grating. In addition, or alternatively, the internally coupled optical element 942 and / or coupled optical element 944 may comprise liquid crystals, and therefore, in some implementations, one or both may be a liquid crystal grating or a liquid crystal diffractive optical element. In addition, or alternatively, one or both of the internally coupled optical element 942 and / or coupled optical element 944 may include a blazed grating. In some designs, the internally coupled optical element 942 comprises a liquid crystal reflector, such as a cholesteric liquid crystal reflective lens (e.g., a reflective liquid crystal diffractive lens, a Bragg reflective structure, a reflective liquid crystal diffractive grating, etc.). Several non-limiting embodiments of liquid crystal gratings, liquid crystal polarizing gratings, and other liquid crystal optical elements are incorporated herein by reference, respectively, in whole and for any purpose, to the following published application, namely, "Multilayer Liquid Crystal Diffractive Gratings for Redielectric Light" U.S. Patent Publication No. 2018 / 0143438, filed November 16, 2017, titled "Of Wide Incident Angle Ranges"; U.S. Patent Publication No. 2018 / 0143485, filed November 16, 2017, titled "Spatially Variable Liquid Crystal Differtion Grattings"; U.S. Patent Publication No. 2018 / 0143509, filed November 16, 2017, titled "Waveguide Light Multiplexer Using Crossed Grattings"; U.S. Patent Publication No. 2018 / 0143509, filed February 22, 2018, titled "Display System with Variable Power Reflector"; U.S. Patent Publication No. 2018 / 0239147, filed February 22, 2018, titled "Variable-Focus Virtual Image Devices" This is discussed in U.S. Patent Publication No. 2018 / 0239177, filed on February 22, 2018, titled “BASED ON POLARIZATION CONVERSION”, and in U.S. Patent Publication No. 2018 / 0164627, filed on December 7, 2017, titled “DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL”. However, the design of the internally coupled optical element 942 and / or coupled optical element 944 is not limited to these and may include other types of optical elements, diffractive optical elements, liquid crystal optical elements, liquid crystal gratings, and liquid crystal polarizing gratings. Further information regarding embodiments of cholesteric liquid crystal structures such as reflectors can also be found in the section titled “Cholesteric Liquid Crystal Mirrors” below. Other liquid crystal optical elements and other non-liquid crystal optical elements may be used as discussed above. Therefore, many types of coupled optical elements (e.g., internal coupled optical element 942 and / or coupled optical element 944), diffractive optical elements, gratings, polarizing gratings, etc., which are both those described herein and other types of gratings, diffractive optical elements, liquid crystal elements, and optical elements in general, may be used.In various implementations, the internal coupling optical element 942 may be configured to couple light from the image projector 930 and / or light source 960 into the waveguide at an angle above the critical angle so that it is guided into the user's eye 210 within the waveguide 940 by total internal reflection to the eye.

[0111] Waveguide 940 may comprise one or more waveguides. In some implementations, one or more waveguides 940 comprise a stack of waveguides. In some designs, for example, different waveguides in a stack of waveguides are configured to output light with different wavefront divergences, as if projected from different distances from the user's eye. For example, a first waveguide or group of waveguides may be configured to output light that is collimated or has a first divergence, as if projected from a first depth, while a second waveguide or group of waveguides may be configured to output light that is diverging (not collimated) or has a second divergence (greater than the first divergence), as if projected from a second depth closer than the first depth. In some designs, different waveguides may be configured to output light having different associated colors. For example, the first waveguide may be configured to output red light, the second waveguide may be configured to output green light, and the third waveguide may be configured to output blue light. The fourth waveguide may be configured to output and / or input infrared light.

[0112] An external coupling optical element 952 for coupling light from a waveguide 940 to a camera 920, as shown in Figure 10, may have multiple redirection features configured to redirect light at certain angles such that light incident on it is not guided within the waveguide but redirected outwards towards the camera. The external coupling optical element 952 may be located inside the waveguide 940, or it may be patterned (e.g., etched) on or within the surface (e.g., the main surface) of the waveguide 940. For example, a surface holographic or diffractive optical element (e.g., a surface relief grating) may be fabricated by patterning (e.g., etching) the surface of the waveguide or a layer on top of it. A volumetric holographic or diffractive optical element may also be formed by changing the refractive index of the material constituting the waveguide or a layer placed on top of it. Depending on the design, the external coupling optical element 952 may be transmissive or reflective, and may operate transmissively or reflectively. For example, the external coupling optical elements 952 may each include a transmitting or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates transmitted or reflected, for example, by redirecting light transmitted through or reflected from it.

[0113] The external coupling optical element 942 may include a reflective optical element (e.g., a mirror). For example, the external coupling optical element 952 may include an off-axis reflector. In some designs, the external coupling optical element 952 may include a polarization optical element such as a polarization-selective redirection element (e.g., a polarizer). Thus, the polarization-selective redirection element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may include a liquid crystal structure such as a liquid crystal polarizing grating. In some implementations, for example, the external coupling optical element 952 may include a liquid crystal polarizing grating (LCPG). An LCPG can potentially provide high-efficiency diffraction over a wide wavelength range. Similarly, an LCPG may be useful for the external coupling optical element 952. An LCPG may be polarization-dependent. An LCPG or other type of liquid crystal grating may include a pattern or arrangement of liquid crystal molecules configured to provide one or more functions, such as redirecting light into or out of a waveguide. Thus, the external coupling optical element 952 may include a polarization grating. In addition, or alternatively, the external coupling optical element 952 may comprise a liquid crystal, and therefore, in some implementations, may be other liquid crystal optical elements such as a liquid crystal grating or a liquid crystal diffraction optical element. In addition, or alternatively, the external coupling optical element 952 may include a blazed grating. In some designs, the external coupling optical element 952 comprises a liquid crystal reflector such as a cholesteric liquid crystal reflective lens (e.g., a reflective liquid crystal diffraction lens, a Bragg reflective structure, a reflective liquid crystal diffraction grating, etc.).Several non-limiting embodiments of liquid crystal gratings, liquid crystal polarizing gratings, and other liquid crystal optical elements are incorporated herein by reference, respectively, in whole and for any purpose, by the following published applications: U.S. Patent Publication No. 2018 / 0143438, titled "MULTILAYER LIQUID CRYSTAL DIFFRACTIVE GRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES," filed November 16, 2017; U.S. Patent Publication No. 2018 / 0143485, titled "SPATIALLY VARIABLE LIQUID CRYSTAL DIFFRACTION GRATINGS," filed November 16, 2017; and "WAVEGUIDE LIGHT MULTIPLEXER USING CROSSED This is discussed in U.S. Patent Publication No. 2018 / 0143509, filed on November 16, 2017, titled "GRATINGS," U.S. Patent Publication No. 2018 / 0239147, filed on February 22, 2018, titled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR," U.S. Patent Publication No. 2018 / 0239177, filed on February 22, 2018, titled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION," and U.S. Patent Publication No. 2018 / 0164627, filed on December 7, 2017, titled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL." However, the design of the external coupling optical element 952 is not limited to these and may include other types of optical elements, diffractive optical elements, liquid crystal optical elements, liquid crystal gratings, and liquid crystal polarizing gratings. Further information regarding embodiments of cholesteric liquid crystal structures such as reflectors can also be found in the section titled “Cholesteric Liquid Crystal Mirrors” below. As discussed above, other liquid crystal optical elements and other non-liquid crystal optical elements may be used.Therefore, many types of coupled optical elements (e.g., external coupled optical element 952), diffractive optical elements, gratings, polarizing gratings, etc., which are both those described herein and other types of gratings, diffractive optical elements, liquid crystal elements, or optical elements in general, may be used. As referenced above, the external coupled optical element 952 may be configured to redirect light induced in the waveguide 940 at an angle less than the critical angle so that it is not induced in the waveguide by total internal reflection but is emitted to the camera 920.

[0114] In various designs, the coupled optical element 944 may be transmissive in the visible spectrum so that the user can see the environment in front of the user through the coupled optical element 944 and the eyepiece 950. The internal coupled optical element 942 may also redirect light in the visible spectrum, for example, when the internal coupled optical element is used to receive light from an image projector 930 and / or when the illumination source 960 is configured to output visible light and illuminate the eye 210 with visible light. In some embodiments, the internal coupled optical element 942 is configured to redirect infrared light, for example, when the illumination source 960 is configured to output infrared light and illuminate the eye 210 with infrared light. In some designs, such as those shown in Figure 10, the internal coupled optical element 942 may be located inside or towards the nose of the external coupled optical element 952. However, in other designs, the internal coupled optical element 942 may be located outside or towards the temple of the external coupled optical element 952. In one implementation, as shown in Figure 10, the external coupling optical element 952 may be adjacent to the internal coupling optical element 942, but non-adjacent positioning is also a possibility.

[0115] The illumination source 960 may be positioned on the same side (e.g., posterior or proximal) of the eyepiece 950 as the eye 210 (proximal can refer to the side closest to the eye 210), as shown in Figure 10. Alternatively, the illumination source 960 may be positioned on the opposite side (e.g., anterior or distal) of the eye 210. The illumination source 960 may be configured to direct light into at least one of the main surfaces of the waveguide 940 via an internal coupling optical element 942. The light source 960 may be configured to emit invisible light (e.g., infrared light). The light source 960 may include one or more LEDs. The LEDs may include infrared LEDs. The light source 960 may be configured to emit coherent light. In some designs, the light source 960 includes a laser (e.g., an infrared laser). In some designs, the light source 960 emits pulsed light. For example, the camera 920 may be configured to periodically capture images. Therefore, the illumination source 960 can be pulsed to match the period during which the camera acquires an image. The intensity output from the illumination source 960 can be reduced when the camera is not acquiring an image. By concentrating the total illumination energy for a short period of time, an increased signal-to-noise ratio can be acquired without exposing the eye 210 to an unsafe intensity level. In some cases, for example, the camera 920 acquires one image every 30 milliseconds, and the camera exposure time is a few milliseconds. The illumination source 960 can be configured to output pulses having a similar period and duration to match that of the camera 920.

[0116] In some implementations, different light sources with different wavelengths are pulsed to provide illumination of different wavelengths at different times, as discussed below.

[0117] The internal coupling optical element 942 may be in direct optical communication with the illumination source 960 and / or the image projector 930, for example, to guide light from the image projector 930 and / or the light source 960 into it. For example, light emitted by the light source 960 may be incident on the internal coupling optical element 942 before optically interacting with either the coupling optical element 944 and / or the external coupling optical element 952.

[0118] As shown in Figures 11A-11E, the light 902 projected from the image projector 930 can form an image on the retina. The image projector 930 may include a light source, a modulator, and / or a projection optical system. The light source for the image projector 930 may comprise one or more LEDs, lasers, or other light sources, or one or more visible light sources. The modulator may comprise a spatial light modulator, such as a liquid crystal spatial light modulator. Such a spatial light modulator may be configured, for example, to modulate the intensity of light at different spatial locations. The projection optical system may comprise one or more lenses. Other types of image projectors 930 capable of projecting and / or forming images may also be employed. For example, the image projector 930 may comprise a scanning optical fiber.

[0119] The image projector 930 and the internal coupling optical element 942 may be in direct optical communication with each other. The image projector 930 may be matched with the internal coupling optical element 942, for example, into which light from the image projector 930 is directed. In some cases, the image projector 930 is positioned adjacent to the corresponding internal coupling optical element 942 and / or waveguide 940. The image projector 930 may also be positioned in an optical path including the internal coupling optical element 942, the coupling optical element 944, and the eye 210.

[0120] The image projector 930 may be a separate element from the illumination source 960, as shown in Figures 10 and 11A-11E. However, in some cases, the image projector 930 may be used as an illumination source. For example, in addition to projecting an image into the eye 210, the image projector 930 may be used to illuminate the eye by directing visible and / or infrared light into the eye for image acquisition. However, alternatively, one or more separate light sources 960 may be used to illuminate the eye 210 for image acquisition.

[0121] The light emitted by the illumination source 960 may comprise a specific wavelength range, such as invisible light. The illumination source 960 may be configured to project invisible (e.g., infrared) light onto / into the eye 210 to image one or more parts of the eye 210 (e.g., cornea, retina). In one exemplary implementation, the light source 960 may be configured to emit light in the range of about 850 nm to 940 nm. The light source 960 may be configured to emit light extending over a wavelength range of at least about 20 nm. Other ranges are also possible. The emitted wavelength range may be 5 nm, 10 nm, 15 nm, 50 nm, 75 nm, 100 nm, 150 nm, 200 nm, or any range between these values. The light source 960 may be configured to emit light across a broadband of wavelengths, such as any range in the infrared spectrum.

[0122] The imaging device 920, which may include a camera, may also include a detector array and, possibly, an imaging optical system. The detector array may include, for example, a CCD or CMOS detector array, and the imaging optical system may include one or more lenses. The one or more lenses may have a positive refractive power and an associated focal length. In one design, the camera 920 is focused to infinity. For example, the optical system may have a focal length f, and the detector array may be positioned at a distance from the optical system corresponding to a focal length such that objects at long distances are imaged onto the detector array. Similarly, light from an eye or an object in the environment, which is collimated, will also be focused onto the detector array, forming an image of the eye or object on it.

[0123] The imaging device 920 may be located on the opposite side of the waveguide 940 from the illumination source 960 and / or the eye 210. In some designs, the imaging device 920 may be located on the same side of the waveguide 940 as the light source 960 and / or the eye 210. As shown in Figure 10, the imaging device 920 may be located outside the eyepiece 950 or near the temple edge, although other locations are also possible.

[0124] Figures 11A–11E illustrate the operation of the exemplary imaging system 900 of Figure 10. Figure 11A shows how the illumination source 960 emits light 902 toward the internal coupling optical element 942 on the waveguide 940. As shown, the light 902 can be directed to incident on the eyepiece 950 at approximately perpendicular, although other angles are also possible. In some designs, the light source 960 is configured to emit collimated light into the eyepiece 950. As shown in Figure 11B, the illumination light 902 can be coupled into the waveguide 940 via the internal coupling optical element 942. In some designs, where the internal coupling optical element 942 comprises a diffractive optical element (e.g., a grating, a holographic element), the light incident on it is diffracted at an angle above the critical angle of the waveguide, and the internally coupled light 904 is guided into the eyepiece 950 by total internal reflection (TIR). In some designs, the internal coupling optical element 942 may be configured to direct light toward the coupling optical element 944. The internal coupling optical element 942 may be polarization-selective. For example, the internal coupling optical element 942 may include a polarization-selective diversion element such as a polarization grating, such as a liquid crystal polarization grating. Figure 11C shows how the internally coupled light 904 propagates through the waveguide 940 by TIR.

[0125] Figure 11D illustrates an exemplary imaging system 900 that couples light outward from the eyepiece 950. As the internally coupled light 904 propagates through the waveguide 940, some of the light may be incident on the coupling optical element 944. The coupling optical element 944 may be configured to couple the internally coupled light 904 outward from the eyepiece 950 toward the user's eye 210. The coupling optical element 944 may be configured to couple the light toward the eye 210 as collimated light. The coupling optical element 944 may be tuned to light in a specific wavelength range. For example, the coupling optical element 944 may be configured to couple infrared light (e.g., about 700 nm to 15,000 nm) outward from the waveguide 940. In some designs, the coupling optical element 944 may be configured to couple multiple wavelengths of light outward from the eyepiece 950. For example, the coupling optical element 944 may be tuned for both infrared and visible light. The coupling optical element 944 can also be configured to couple light into the waveguide 940, as fully described below.

[0126] The coupled optical element 944 can be configured to increase the dimensions of one or more eyeboxes for the user. For example, one or more dimensions may be measured along a first axis (e.g., the x-axis). The eyepiece 950 may further include an orthogonal pupil expander (OPE). The OPE may have at least one optical redirection element located on or inside the waveguide (e.g., on one of the main surfaces), or the OPE may be located inside the waveguide 940. The OPE may include features similar to or the same as those described above with respect to the optical dispersion elements 730, 740, and 750. In some implementations, the optical redirection element may comprise a diffractive optical element. The OPE may be configured to increase the dimensions of the eyebox along a second axis (e.g., the y-axis) orthogonal to the first axis.

[0127] Figure 11D shows how some of the light exits the eyepiece 950 toward the user's eye 210. In some designs, the coupled optical element 944 is configured such that internally coupled light 904, incident on the coupled optical element 944 at various parts of the coupled optical element 944 along a first axis (e.g., parallel to the x-axis), exits from the eyepiece 950 at each part of the coupled optical element 944 along the first axis. This can provide the user with light to project an image or to illuminate the eye for different eye positions or locations.

[0128] As shown in Figures 11D-11E, the coupling optical element 944 may be configured to couple the internally coupled light 904 outward from the eyepiece 950 as collimated light. This light may also generally be directed substantially perpendicular to the main surface of the eyepiece 950 and / or waveguide 940. The collimated light may be directed into the eye and focused onto the retina by the eye (e.g., the cornea and natural lens of the eye). The light 908 incident on the retina may provide illumination to image the retina and / or provide image content to the eye. A portion of the light 908 may, for example, be reflected or scattered from the retina, exit the eye, and provide an image of the retina to be captured. The light source 960 may be an extended light source so that the light will illuminate an area of ​​the retina.

[0129] Figures 12A–12E illustrate how the imaging system 900 of Figures 11A–11E may be used in addition to, or as an alternative to, image acquisition of the eye 210. Figure 12A shows how light 910 reflected from the retina exits the eye 210. As shown, the light 910 scattered or reflected from the retina, passing through the eye's natural lens, intraocular pupil, and cornea, can be collimated. This light may also be incident on the eyepiece 950 with normal incidence (e.g., perpendicular to the main surface of the waveguide 940 and / or coupling optical element 944). The coupling optical element 944 may be configured to couple the light 910 reflected from the retina into the waveguide 940.

[0130] Figure 12B illustrates an exemplary imaging system 900 as light is coupled into the eyepiece 950. The coupling optical element 944 may include redirection features, such as a diffractive optical element or other structure, that redirect the light at an angle above the critical angle so that it is induced within the waveguide 940. The coupling optical element 944 may be configured to generally direct the internally coupled light 914 toward the light source 960 and / or imaging device 920. The coupling optical element 944 may be configured to couple less than a certain percentage of the original light propagating toward the camera 920 toward the waveguide 940 toward the outside. For example, a partially reflective element (e.g., a translucent mirror) may be placed on or within the waveguide 940 so that a portion of the internally coupled light 914 continues to propagate within the waveguide 940 by total internal reflection, while reducing leakage of the internally coupled light 914 out of the waveguide 940 along the portion of the waveguide 940 in which the coupling optical element 944 is located. The portion of light that does not leak out may be any proportion between 0 and 1. For example, this portion may be 0.90, in which case 90% of the light rays propagating through the waveguide 940 along the coupled optical element 944 are retained within the waveguide 940 at each reflection of the rays. Other portions are also possible (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or any range between these values). Such partially reflective elements can also be used in the implementations described below.

[0131] As shown in Figure 12C, the collimated internally coupled light 914 can continue to propagate through the waveguide 940 toward the imaging device 920. Figure 12D shows how a portion of the internally coupled light 914 can continue to propagate until it is incident on one or more externally coupled optical elements 952. To reduce the amount of internally coupled light 914 leakage out of the internally coupled optical element 942, the internally coupled optical element 942 can be configured to hardly couple the light propagating toward the camera 920 back out of the waveguide. For example, a partially reflective element (e.g., a translucent mirror) may be placed on or inside the waveguide 940 so as to reduce leakage of internally coupled light 914 out of the waveguide 940 along the portion of the waveguide 940 where the internally coupled optical element 942 is located, while allowing a portion of the internally coupled light 914 to continue propagating within the waveguide 940 by total internal reflection. The portion of light that does not leak may be any proportion between 0 and 1. For example, the portion may be 0.90, so that 90% of the light rays propagating along the coupled optical element 944 through the waveguide 940 are retained within the waveguide 940 at each reflection of the rays. Other portions may also be considered (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or any range between any of these values). Such partially reflective elements can also be used in the implementations described below.

[0132] As shown in Figure 12E, the external coupling optical element 952 can be configured to couple light induced in the waveguide 940 out of the waveguide 940 to the imaging device 920. As a result, light propagating in the waveguide 940 and incident on the external coupling element 952 can be redirected so that it is emitted out of the waveguide 940, for example, from the main surface of the waveguide 940 (e.g., forward or backward side of the waveguide 940), and directed onto the imaging device 920. The external coupling optical element 952 may be configured to direct the light 926 so that it exits the waveguide 940 perpendicular to the main surface of the waveguide 940 (e.g., perpendicular). In some designs, the external coupling optical element 952 is configured to direct collimated light 924 onto the imaging device 920 with normal incidence to the photosensitive portion of the imaging device 920. As discussed above, the camera 920 may be focused to infinity, and for example, the imaging optical system may be configured to focus collimated light onto a detector array.

[0133] Therefore, the waveguide 940 may be configured to guide light coupled from the user's eye 210 into the waveguide 940 so that it is received by the imaging device 920 (e.g., a camera) to capture an image of at least a portion of the user's eye 210. The same waveguide 940 may also be configured to guide light coupled from the image projector 930 so that the light from the image projector 930 can be directed towards the user's eye 210 so that an image from the image projector 930 is within the user's field of view. In some implementations, the same waveguide is configured to guide light coupled from the illumination source 960 so that the light from the illumination source can be directed towards the user's eye 210 and illuminate the eye so that an image of the eye can be captured by the camera 920.

[0134] In some implementations, the same coupling optical element 944 can be configured to (i) couple light from the user's eye 210 into the waveguide 940 so that it is received by the imaging device 920, and (ii) couple light from the image projector 930 out of the waveguide 940 to the user's eye 210, projecting image content into the user's field of view. In some implementations, the same coupling optical element 944 can be configured to couple light from the illumination source 960 out of the waveguide to the user's eye 210 so that light from the illumination source can illuminate the eye.

[0135] In other designs, different waveguides and / or different coupling optical elements 944 may be used. In some designs, for example, the first waveguide 940 may be configured to guide light coupled from the user's eye 210 so that it is received by the camera 920 to capture an image of at least a portion of the user's eye 210, and the second waveguide may be configured to guide light coupled from the image projector 930 so that the light from the image projector 930 is directed towards the user's eye 210. The first and second waveguides may be stacked on top of each other. Another waveguide may be configured, in addition or alternatively, to guide light coupled from the illumination source 960 so that the light from the illumination source is directed towards the user's eye 210 and can illuminate the eye.

[0136] In some implementations, the first coupling optical element 944 may be configured to (i) couple light from the user's eye 210 into the waveguide 940 so that it is received by the imaging device 920, and (ii) couple light from the image projector 930 out of the waveguide 940 to the user's eye 210, projecting image content into the user's field of view. Another coupling optical element may, in addition or alternatively, be configured to couple light from the illumination source 960 out of the waveguide to the user's eye 210 so that light from the illumination source can illuminate the eye.

[0137] In some designs, the coupled optical element 944 may include multiple diffractive optical elements (DOEs). For example, a first DOE may be configured to couple light from the user's eye 210 into the waveguide 940 so that it is received by the imaging device 920. A second DOE may be configured to couple light from an image projector 930 out of the waveguide 940 to the user's eye 210, projecting image content into the user's field of view. Optionally, a third DOE may be configured to couple light from a light source 960 out of the waveguide 940 to the user's eye 210, illuminating the eye. The first and second (and possibly third) DOEs may be stacked, for example, in some implementations, so that light from the user's frontal environment passes through the first DOE, then onto the second DOE, then onto the third DOE, and then onto the user's eye. However, the order may differ.

[0138] In some designs, the first and second DOEs are integrated within a single element or volume of the waveguide 940. In some implementations, for example, both the first and second DOEs are superimposed on each other within the waveguide 2102 (e.g., occupying the same or nearly the same volume). For example, the first and second DOEs may be recorded in the same medium.

[0139] As described above, image acquisition of the eye, for example, the retina, can facilitate eye tracking. Figure 13A illustrates an imaging system 900 configured to image various parts of the eye 210 (e.g., the retina) at different times when the eye is in different positions. Stages A and B may refer to images of the eye 210 between different eye orientations. Figure 13A shows images of the eye 210 during both imaging stages A and B and the results.

[0140] In some implementations, light emission 928 (e.g., from an illumination source 960 as described above, or from one or more illumination sources configured and / or located differently) can be used to acquire one or more images of the retina 962, as shown in Figure 13A. The image of the retina 962 may comprise one or more regions 964, 966 that are imaged between different orientations of the eye 210. Figure 13A shows two regions 964, 966 of the image of the retina 962. For example, region 964 of the retina to be imaged in stage A may be imaged while the eye 210 is directed at an angle perpendicular to the waveguide 940. Image data relating to region 966 to be imaged in stage B may be acquired while the eye 210 is directed at a certain acute angle with respect to the waveguide 940. A composite image or map of the retina 962 may be acquired using one or more orientations of the eye 210 during one or more stages of imaging. Processing electronic equipment or a processor, such as a data module 140 (see Figure 2), may be used to find overlapping image data between two neighboring regions. Using the image data of the overlapping regions, a composite image of the retina 962 can be determined. A larger size (e.g., full size) composite image or map of the user's retina can be stored.

[0141] As described herein, a head-mounted display can be used to map the retina of the user's eye based on the direction in which the user's eye is directed. To provide a realistic and intuitive interaction with objects in the user's environment and / or to identify the wearer of the head-mounted display device using eye line of sight, the head-mounted display system can use retinal mapping to incorporate the uniqueness of the user's eye features and other conditions that may influence ocular measurements. For example, an image may be identified based on the location of blood vessels in the corresponding retinal image.

[0142] Retinal mapping may involve a process that allows a computing device to learn how to associate a user's line of sight (e.g., as identified in a retinal image) with a point of view in 2D or 3D space. The line of sight may be associated with a single point in 2D or 3D space. The line of sight may also be associated with multiple points in space, which can describe the movement of virtual objects (e.g., a series of points, the location of a moving image).

[0143] A head-mounted display system can determine the user's line of sight based on retinal images. The head-mounted display system can acquire retinal images using sensors (e.g., an eye camera such as imaging device 920). The head-mounted display system can image one or both of the user's eyes while the user changes their line of sight (e.g., when the user is looking around to track a moving or shifting calibration target or a fixed target). To map the user's retina, the head-mounted display system can present a virtual target, such as a fixed target, for the user to view. The virtual target may be associated with one or more known points in the line of sight in 2D or 3D space. While the user is viewing the target, the head-mounted display system can acquire retinal images and associate the images with the line of sight points. Based on the association between individual retinal images and the line of sight points associated with the targets, the head-mounted display system can calculate and / or generate a mapping matrix.

[0144] Retinal mapping results can reflect the uniqueness within each individual's eye. For example, a head-mounted display system can generate mapping matrices customized for a specific individual's single or double eye. For instance, a user may have different amounts of eye movement or line of sight in response to specific targets. In addition, or alternatively, a user may have different locations, sizes, shapes, and / or orientations of blood vessels within their retina. As a result, by generating calibration results specific to each individual user, a head-mounted display system can enable more accurate user interaction with line of sight and / or enable the identification of a particular user.

[0145] Therefore, when a user wears a head-mounted display device, the system can detect whether the user is a previous user or a new user. A confusion matrix can be calculated, in which a score for a particular eye gaze image stored in system memory is compared with a corresponding image of the current user. The confusion matrix may include comparison scores for multiple eye gazes and associated retinal images. Based on the comparison scores, the system may be able to make a decision regarding the confidence level for user identification (e.g., whether the user is identical to the individual associated with the stored retinal image or composite map) and / or determination. The confidence level may include, for example, an identification coefficient. A stored image, e.g., a composite image or map, may be compared with a later acquired image, referred to as an instantaneous or real-time image acquired for the current user. If the system detects that the user is a new user, it may provide an alert or take other action.

[0146] The system may apply filtering, such as digital filtering or image processing, to the retinal image captured by the camera. Such filtering or imaging processing can enhance features, which can be used, for example, for identification, stitching, assembling composite images, eye tracking, etc. Such filtering or imaging processing may include edge enhancement. Such filters may include, for example, a Frangi filter, but other types of filters may be used. Such filters or processing (e.g., edge enhancement or Frangi filter) can be used to enhance and / or detect image features such as blood vessels or tubular structures or fibers in the retinal image.

[0147] Figure 13B illustrates a pattern of sequentially displayed fixed targets that may be used in a retinal mapping process. These virtual targets, upon which the user's eye will direct its gaze, can be redirected to various different directions, during which the retina can be imaged. The resulting images associated with different gaze directions correspond to different parts of the retina. As discussed above, when the eye gazes in different directions and views fixed targets positioned differently on the display, the images captured by the camera include different parts of the retina. These images can be assembled to form a larger map or composite image of the retina.

[0148] Figure 13B shows virtual targets at 16 different locations within the user's field of view (FOV) 1200. In various implementations, the virtual targets will be presented at a given location at a given time. One or more retinal images will be acquired during the time the virtual target is presented to the user at that particular location. These images may be associated with the target location and / or the corresponding line of sight direction. More or fewer target locations may be used. In the embodiment shown in Figure 13B, 16 target locations 1202a-1202p are shown. More or fewer target locations may be used. The target locations may also vary. The order in which the targets are presented at different locations may vary. For example, the targets may move in a raster pattern from left to right in the user's field of view, then conversely, from right to left, and again from left to right, descending the position of the targets within the field of view, with each lateral pass traversing the field of view. However, other patterns and approaches are also possible. Similarly, targets can be rendered in the same or different ways at different locations. For example, rendered targets may differ in size, shape, color, etc. Targets can be rendered sequentially to the user during the eye-tracking calibration process. For example, as discussed above, the head-mounted display system may render targets in a meandering pattern. For example, target 1202a may be followed by 1202b, then 1202c, then 1202d, then 1202h, then 1202g, and so on. Other patterns are also possible. For example, targets may be displayed in a more random or non-sequential pattern. In some embodiments, a single target is displayed to the user, and the target moves around the user's field of view (e.g., passing through or temporarily stopping at positions 1202a-1202p during the target's movement). The head-mounted display system can obtain images of the user's retina while the user is viewing these targets.For example, a head-mounted display system may obtain a first image when the user is looking at a target at a first location 1202a, a second image when the user is looking at a target at a second location 1202b, a third image when the user is looking at a target at a third location 1202c, and so on. The wearable system may associate the first image with the first location 1202a, the second image with the second location 1202b, the third image with the third location 1202c, and so on. Neighboring images may be stitched together in a database to create a complete or partial retinal map. For example, two images may be stitched together in appropriate alignment using a common feature or part of a feature (e.g., blood vessels or part of one) in multiple images. In various implementations, adjacent target locations will produce overlapping images that can be aligned and stitched together. For example, target positions 1202a and 1202b, and target positions 1202b and 1202c, can produce overlapping and adjacent retinal images that can be stitched together. Thus, several different retinal images can be acquired using different eye lines to assemble a larger image of the retina (e.g., a composite image or map).

[0149] As discussed above, eye tracking can be performed using a synthetic retinal image or map. For example, after a target is no longer visible, the user may shift their gaze as they view different real objects or augmented reality (virtual) image content displayed by the head-mounted display in front of the user and the head-mounted display. One or more retinal images may be acquired during these times. The terms “instantaneous” or “real-time” images may be used herein to describe these images acquired following calibration, which may be used for eye tracking (or other purposes such as acquiring biometric data). These “instantaneous” or “real-time” images are likely to correspond to a portion of a synthetic retinal image or map. The system may be configured to closely match the “instantaneous” or “real-time” retinal image with a portion of the synthetic retinal image or retinal map. Such a match may be based on features or parts of features (blood vessels or parts thereof) common to both the “instantaneous” or “real-time” retinal image and the portion of the synthetic retinal image or map. Based on the location of the portion of the synthetic retinal image or map that matches the “instantaneous” or “real-time” retinal image, the gaze direction may be inferred. Different gaze directions will result in retinal images corresponding to different parts of the retinal map. Therefore, identifying the location of a "momentary" or "real-time" retinal image on a composite retinal image or map will provide information about the user's gaze direction. Eye tracking, e.g., tracking eye movement and changes in eye gaze, may be performed using such or similar methods. As discussed above, edge enhancement, edge detection, or other digital filtering and / or processing may be used to highlight different image features and / or correlate them with the composite retinal image or retinal map.

[0150] In various implementations, after the completion of the initial calibration process in which virtual or fixed targets are displayed (e.g., at multiple locations) and a composite retinal image or map is assembled, the composite retinal image or map can still be refined. For example, as additional retinal images are acquired, the composite retinal image or map can be further refined or improved using the additional images. Thus, as additional "instantaneous" or "real-time" retinal images are acquired, for example, for the purpose of providing eye tracking, the composite retinal image or map can be further refined or improved using the "instantaneous" or "real-time" images. As the user continues to look at various positions within the display (with or without the assistance of calibration targets), the retinal composite image or map may be further refined using additional images acquired following the initial calibration in which the virtual or fixed targets were displayed. The quality of the composite retinal image or map can therefore be increased.

[0151] Additional, non-limiting embodiments of methods in which eye tracking may be performed and / or a synthetic retinal image or map may be produced are described in U.S. Patent Publication No. 2017 / 0205875, filed January 17, 2017, entitled “EYE IMAGE COLLECTION” (this disclosure is incorporated herein by reference in its entirety).

[0152] Therefore, as discussed above, larger portions of the retina may be recorded and mapped by acquiring retinal images and / or other images of the eye using an imaging system such as those described herein, and such images may facilitate eye tracking. For example, the image of eye 210 shown in Figure 13A may be captured when the eye is in an arbitrary position. Processing electronic equipment or a processor (such as the same or different ones described above as forming a composite image) may then track eye movement by comparing a real-time captured image of the user's retina with a stored composite or larger-size (e.g., full-size) image of the user's retina. A given real-time captured image of the user's retina may show a specific portion of the user's retina. As described above, by comparing such a captured image with a stored image of the user's mapping of larger portions of the user's retina, the system can determine the portion of the user's retina shown in the captured image, and thereby determine the position / orientation of the eye that will produce such an image. See, for example, Figure 13A, which shows two different images of a portion of the retina produced when the eye is in two different positions and / or orientations. Therefore, the position and / or orientation of the eye can be determined by capturing different images of the retina and determining the visible portion of the retina. Such a determination can be made even if a composite image is not formed; rather, multiple images of the retina with respect to different eye positions / orientations are recorded and stored in a database. When a future image of the retina is acquired, it is compared to an image in the database of stored images, and an image in the database that is similar to the most recently acquired image of the eye can be determined. Matching recent images to one or more images in the database that have associated positions and / or orientations can enable the determination of the orientation and / or position of more recent images. Other approaches to eye tracking may also be used based on images captured using the designs described herein.

[0153] As described herein, retinal images may also be employed for other purposes. For example, retinal images may be used to verify that a user is the same user from whom a composite retinal image or map was acquired. Images of the retina acquired while a user is wearing a head-mounted display system (e.g., during the calibration process and / or during subsequent use) may be compared to a previously acquired composite retinal image or map (e.g., created the previous day or when the head-mounted display was previously booted up) that is stored. If the most recently acquired retinal image does not adequately match any part of the composite retinal image or map, it may be concluded that the current user is different from the previous user (e.g., from whom the composite virtual image or map was created). Such methods may be used for security purposes, for example, to verify that the current user of a head-mounted display device is the owner or a typical user of the device. Thus, biometric data acquired via retinal imaging may be used for security purposes.

[0154] Retinal imaging may also be used to collect biometric data for monitoring the user's health. Medical-related data may be obtained from retinal images. Such medical data may be useful for monitoring the user's health.

[0155] While various applications of ocular imaging, such as eye tracking, health monitoring, and the collection of biometric data for security, are discussed herein in the context of retinal imaging, imaging of other parts of the user, such as the user's eyes, may also be employed for these and other purposes.

[0156] While the eyepiece 950 is described above as being usable to facilitate imaging of the eye, the eyepiece can also be used to image the world in front of the user. Figures 14A–14B illustrate an exemplary imaging system 900 that may be used, for example, to image a portion of the environment in front of the user and / or objects within a portion of the environment. The imaging system 900 used may be a similar system to those described with respect to Figures 11A–11E and / or Figures 12A–12E, except that light is collected by the eyepiece 950 from the environment in front of the eyepiece and the user. Figure 14A illustrates, for example, light 970 from the environment reflected and / or emitted by one or more physical objects 972 in the environment in front of the user and the eyepiece 950. As shown, light 970 from the environment can be nearly collimated (e.g., to infinity) because, for example, a physical object 972 in the environment may be located at a sufficiently long distance from the imaging system 900 for the light rays reaching the imaging system 900 to be collimated or nearly collimated. In some implementations, the imaging system 900 may be configured to image the environment and / or objects in the environment without using any optical elements (e.g., lenses, mirrors) that have refractive power within the imaging system 900.

[0157] The imaging system 900 shown in Figures 14A and 14B is similar to the imaging system described above. The imaging system includes an eyepiece 950 comprising one or more waveguides 940, each containing a coupling optical element 944 configured to direct light from an image projector 930 (not shown) into the eye 210 and form an image therein. The one or more waveguides may include multiple waveguides (e.g., a stack of waveguides) configured to internally / externally couple multiple corresponding colors / wavelengths. Each waveguide in the stack of waveguides may be configured to direct light of a specific color (e.g., red, green, blue). For example, the furthest waveguide (e.g., a stack of waveguides) may be configured for visible light (e.g., red, blue, green) such that the waveguide is configured to internally and externally couple the same wavelengths of visible light. In addition, or alternatively, waveguides configured to internally and externally couple invisible (e.g., infrared) light may be located proximal to the eye 210. Multiple such waveguides corresponding to waveguide 940 may be used in any other implementation described herein. The imaging system 900 may also include an imaging device (e.g., a camera) 920 and an external coupling optical element 952 configured to redirect light reflected from the eye 210, which is propagated to the camera within waveguide 940. In Figures 14A and 14B, the illumination source 960 is excluded because the illumination source may not be required to image the environment in front of the user. However, an illumination source (e.g., the light source 960 described above) may be used in some designs.

[0158] The eyepiece 950, waveguide 940, coupling optical element 944, external coupling optical element 952, and camera 920 may be identical or similar to those described above. For example, the coupling optical element 944 may be physically engaged with the waveguide 940. For example, the coupling optical element 944 and / or the external coupling optical element 952 may be positioned in the optical path between the environment in front of the eyepiece 950 and the camera 920 such that light from the environment is coupled through the coupling optical element 944 into the waveguide 940 and incident onto the camera 210 (for example, to form an image of at least a portion of the environment), or coupled out of the waveguide via the external coupling optical element. The coupling optical element 944 may have a plurality of redirection features configured to redirect light induced in the waveguide out of the waveguide, or to redirect light incident on the coupling optical element 944 into the waveguide at a certain angle so that it is induced therein by total internal reflection. The external coupling optical element 952 may comprise a plurality of redirection features configured to redirect light (from the environment) that is induced at a certain angle within the waveguide so that it is directed outward toward the camera, but is not induced within the waveguide by total internal reflection. The coupling optical element 944, the external coupling optical element 952, and the redirection features associated with each may be physically engaged with the waveguide 940. For example, the coupling optical element 944 and / or the external coupling optical element 952 may comprise one or more holographic or diffractive optical elements (e.g., surface relief gratings) patterned (e.g., etched) within or on the waveguide 940. The coupling optical element 944 and / or the external coupling optical element 952 may comprise a layer placed on or within the waveguide 940. For example, volumetric holographic or diffractive optical elements may be formed by varying the refractive index of the material constituting the waveguide or the layer placed on or on it. Therefore, the coupling optical element 944 and / or the external coupling optical element 952 may be located within the volume of the waveguide 940 or a layer placed on top of it. Depending on the design, the coupling optical element 944 and / or the external coupling optical element 952 may be transmissive or reflective, and may operate transmissively or reflectively.For example, the coupled optical element 944 and / or the external coupled optical element 952 may each include a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates transmissively or reflectively, for example, to redirect light transmitted through or reflected from it. The coupled optical element 944 and / or the external coupled optical element 952 may include a polarization optical element such as a polarization-selective redirection element (e.g., a polarizer). The polarization-selective redirection element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may comprise a liquid crystal structure such as a liquid crystal polarization grating. In some implementations, the reflective optical element may include a reflector (e.g., a mirror). Other elements, such as the waveguide 940, may similarly be analogous to those described above.

[0159] Figure 14B illustrates the operation of the imaging system 900 shown in Figure 14A. Light 970 from the environment is coupled into the waveguide 940 by a coupling optical element 944. The coupling optical element 944 may be configured to redirect the collimated light at an angle exceeding the critical angle of the waveguide 940 so that at least a portion of the collimated light is guided toward the camera 920 by total internal reflection within the waveguide. An external coupling optical element 952 may be configured to receive at least a portion of the light from the environment in front of the user that is coupled into and guided within the waveguide 940 via the coupling optical element 944. The external coupling optical element 952 may be configured to couple the internally coupled light out of the waveguide 940 toward the camera 920 so that an image of the environment can be captured by the camera 920. The image of the environment may pass through processing electronics (e.g., one or more processors), such as a data module 140 (see Figure 2). The data module 140 may be configured to reproduce a modified image of the environment within the augmented reality context. The processing electronics may communicate with the camera 920 via wired or wireless electronic signals. In addition, or alternatively, the processing electronics may communicate with the camera 920 using one or more remote receivers. The processing electronics may reside remotely (e.g., a cloud computing device, a remote server, etc.).

[0160] The imaging system 900 may therefore be used directly to image an environment, which may be useful for a variety of reasons. For example, imaging an environment can be used to determine where augmented reality image content should be placed relative to objects in the environment. For example, imaging an environment may provide the location of a table so that a head-mounted display can render an image of a person standing next to a table, instead of on or inside the table. The imaging system 900 described in relation to imaging an environment may also be used to image an eye 210, as described with reference to Figures 10, 11A-11E, and / or 12A-12E.

[0161] It may be desirable to image a wide field of view of the environment using the imaging system 900. Figure 14C schematically illustrates the imaging system 900 for collecting light from the environment using a refractive power optical element or lens, such as a refractive optical element 980 (e.g., a wide-angle lens) in front of the eyepiece. The refractive optical element 980 may have a positive refractive power. The refractive optical element 980 (e.g., a positive lens) focuses the collimated light 970 from the environment toward the waveguide 940. Lenses of a type other than those shown in Figure 14C may be employed. Transmitted light (not shown) may pass through a refractive power optical element or lens, such as a refractive optical element 990 (e.g., a negative lens), configured for the same but opposite negative refractive power as the refractive optical element 980. The negative lens 990 has a refractive power similar to or identical to that of the positive lens 980 and may offset or cancel out the refractive power or part of the positive lens. Thus, light from the environment (e.g., distal to the waveguide 940) can pass through the negative lens 990, the eyepiece 950, and the positive lens 980, substantially without affecting the net change in the refractive power introduced to the eye by these two lenses. The negative lens 990 may be configured to offset or cancel out the refractive power of the positive lens 980 so that the user will not suffer the refractive power of the positive lens when viewing the environment in front of the eyepiece 950. The negative lens 990 will also cancel out the effect of the positive lens 980, inverting the image of objects in the environment in front of the wearer. Some light 970 from the environment can be internally coupled into the waveguide 940 by the coupling optical element 944, despite some of the rays converging. The internally coupled light incident on the external coupling optical element 952 can be emitted out of the waveguide 940.

[0162] Implementations (e.g., those described by Figures 14A-14C) may be used outside of an augmented reality context. For example, an imaging system 900 configured to image the environment is intended to be implemented in a wearable device such as eyeglasses (including non-refractive eyeglasses) or bifocal eyeglasses. Such an imaging system 900 may not require an image projector 930 and / or light source 960. In addition, or alternatively, such an imaging system 900 may not require an internally coupled optical element configured for the corresponding image projector 930 and / or light source 960.

[0163] It may be advantageous to implement such an imaging system 900 to project the environment onto a viewing screen (e.g., a television screen, computer screen) of a handheld device (e.g., a mobile phone, a tablet). The imaging system 900 can improve video chat capabilities. For example, when a viewer who can see their chat partner looks at the screen, it may appear as if the chat partner is looking directly at the viewer. This would be possible because the rays captured by the imaging system 900 would be captured within the same area that the user is viewing (in contrast to viewing a screen, which has rays captured by, for example, a separate outward-facing camera positioned in a different location).

[0164] In an implementation where the imaging system 900 in Figure 14C is also used to image the eye 210, the light source 960 and / or image projector 930 may be configured to inject light into the waveguide 940. Since the light reflected from the eye, which is internally coupled into the waveguide, will pass through a refractive optical element 990 (e.g., a negative lens), a positive refractive power type refractive optical element may be placed between the light source 960 and / or image projector 930 and the waveguide 940. The positive lens may be configured to offset or cancel out any refractive power provided by the refractive optical element 990 before the internally coupled light from the light source and / or light projector is incident on the eye 210. Lenses of types other than those shown in Figure 14C may also be used as the optical element 990. Alternatively, or in addition, processing electronics communicating with the light source and / or image projector may be configured to modify the image sufficiently after the light has passed through the refractive optical element 990 to present an undistorted image to the user. The corresponding internal coupling optical elements, external coupling optical elements, and / or coupling optical elements may, in some designs, be configured to act on uncollimated light (e.g., divergent, converging light).

[0165] In various implementations, the same waveguide 940 may be used to (i) propagate light from the eyepiece 950 and the environment in front of the user to the camera 940, and (ii) propagate light from the image projector 930 to the eye 210 to form image content within it. Using the same waveguide 940 may simplify the system and / or eyepiece, making the system and / or eyepiece more compact and potentially providing a reduced shape factor. Reducing the thickness of the eyepiece 950 by reducing the number of waveguides 940 may also be advantageous for other reasons. Lower cost and a simpler manufacturing process may be some of these advantages.

[0166] Furthermore, in various designs, the same or different imaging systems may be used within the same head-mounted display, for example, to image the eye by propagating light from the eye to the camera 940 via a waveguide in the eyepiece 950, as described above. Such a system may also use the eyepiece to transfer light from an illumination source to the eye 210 to illuminate the eye. In some designs, the eyepiece may also be used to propagate light from an image projector 930 to the eye 210 to form image content within it. Using the eyepiece to assist in imaging the environment and imaging the eye (and possibly illuminating the eye) may simplify the system and / or make the system more compact and potentially provide a reduced shape factor.

[0167] Furthermore, in some implementations, the same waveguide 940 may be used to (i) propagate light from the environment in front of the eyepiece 950 to the camera 940, and (ii) propagate light from the eye 210 to the camera to capture an image of the eye. The same waveguide may also be used to propagate light from the image projector 930 to the eye 210 to form image content therein and / or for image capture, and to propagate light from the illumination source 960 to the eye 210 to illuminate the eye. Using the same waveguide 940 may simplify the system and / or eyepiece, make the system and / or eyepiece more compact, and potentially provide a reduced shape factor. Reducing the thickness of the eyepiece 950 by reducing the number of waveguides 940 may also be advantageous for other reasons. Lower cost and a simpler manufacturing process may be some such advantages.

[0168] Similarly, in addition to coupling light from the environment into the waveguide 940, the same coupling optical element 944 may be configured to direct light from the eye into the waveguide 940 so that light from the image projector 930 is directed towards the eye 210, forming image content within it and / or being guided to the camera 920 within it. In addition, or alternatively, the same coupling optical element 944 may be configured to couple light from the illumination source 960, which is guided within the waveguide 940, out of the waveguide to the user's eye 210.

[0169] As discussed above, one or more of the coupled optical elements 944, the internal coupled optical element 942, or the external coupled optical element 952 may be equipped with a polarization-selective coupling element. Thus, in various designs, the light input into the eyepiece 950 or waveguide 940 is polarized so as to be appropriately acted upon by the polarization-selective diversion element.

[0170] Therefore, in some embodiments, the illumination source 960 is equipped with a polarization source of suitable polarization so as to be appropriately acted upon by a polarization-selective coupling / direction-changing element.

[0171] One or more polarization-specific optical filters and polarization correction elements may be included in various imaging systems 900, such as those in which an image projector 930 and / or a light source 960 are positioned facing each other through a waveguide 940. Polarization sensing elements may be useful in reducing directional light emission into the imaging device 920 and / or, for example, in a configuration where these elements are aligned on opposite sides of the waveguide 940 at the same lateral position, in order to reduce saturation of the imaging device 920. Figures 15A-15B illustrate such a configuration. A light source 960, as shown in Figure 15A, can be configured to direct light through a polarization-specific optical filter 982, such as a polarizer (e.g., a linear polarizer), and / or through a polarization correction element 986, such as a polarization rotor, which is configured to modify the polarization state of the incident light. A retarder, such as a half-wave retarder, can rotate linear polarization, for example. Therefore, a properly oriented half-wave retarder or half-wave plate can rotate s-polarization to p-polarization or vice versa. Thus, in various implementations, a polarization-specific optical filter 982 and / or polarization correction element 986 are placed in the optical path between the light source 960 and the internal coupling optical element 942 so as to provide properly oriented polarization to the internal coupling optical element. In some implementations, the imaging system 900 does not include a polarization correction element but includes a properly oriented polarizing optical filter such as a polarizer.

[0172] Light emitted by the light source 960 can pass through an array of optical elements in a specific order. For example, as shown in Figure 15A, light can first pass from the light source 960 through a polarization-specific optical filter 982 (e.g., a polarizer), and then through a polarization correction element 986 (e.g., a rotor). After passing through the polarization correction element 986, the light may be incident on an internal coupling optical element 942, which can direct the light into the waveguide 940 as induced therein.

[0173] For example, the light source 960 may be configured to emit light of mixed polarization (e.g., s-polarized and p-polarized). The polarization-specific optical filter 982 may be configured to transmit only light of a first polarization state (e.g., p-polarized). As the light continues, the polarization correction element 986 may be configured to change the polarization state of the light (e.g., from p-polarized to s-polarized). The internal coupling optical element may be configured to redirect the s-polarized light to an angle above the critical angle of the waveguide so that s-polarization is induced within the waveguide. The internally coupled light 904 may be substantially polarized in a second polarization (s-polarized) as it propagates through the waveguide 940. The coupling optical element 944 may be configured to redirect only light of the second polarization state (s-polarized). The coupling optical element 944 may be configured to couple the internally coupled light 904 out of the waveguide 940 to the eye 210 to provide illumination for image acquisition.

[0174] To prevent direct illumination (e.g., saturation) of the imaging device 920, polarization correction elements 958 and / or polarization-specific optical filters 984 may be placed inside or on the waveguide 940 so that only light of a certain polarization state (e.g., p-polarization) can pass through the polarization-specific optical filter 984 to the imaging device 920. The polarization correction element 958 (e.g., a half-wave plate) may be configured to change the polarization state (e.g., from s-polarization to p-polarization). The polarization-specific optical filter 984 may be configured to transmit only light of a certain polarization (e.g., p-polarization) through it. Thus, light passing through the polarization-specific optical filter 982 would not be configured to be transmitted directly through the polarization-specific optical filter 984. In any of the above implementations (for example, the image projector 930 and / or light source 960 are on the same optical axis as shown in Figure 15A), the configuration of the polarization-specific optical filter 982, polarization correction element 986, internal coupling optical element 942, polarization correction element 958, and / or polarization-specific optical filter 984 may be implemented according to the design in Figure 15A, as shown in Figures 10, 11A-11E, and 12A-12E. The polarization-specific optical filter 984 may be a transmissive-reflective polarizer (e.g., a polarizer beam splitter) configured to transmit light of a first polarization and redirect or reflect light of a second polarization different from the first.

[0175] A partially reflective element (e.g., a translucent mirror) may be included to redirect the internally coupled light 904 towards the imaging device 920. The partially reflective element may be positioned between the internally coupled optical element 942 and the polarization correction element 986 such that a portion of the internally coupled light 914 is reflected towards the imaging device 920 while reducing the leakage of the internally coupled light 914 out of the waveguide 940. The portion of light that does not leak may be any percentage between 0 and 1. For example, the portion may be 0.90, so that 90% of the light rays propagating through the waveguide 940 along the coupled optical element 944 are retained within the waveguide 940 at each reflection of the rays. Other portions are also possible (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or any value within this range).

[0176] Figure 15B illustrates the propagation of light reflected or scattered from the retina. A portion of the light 910 reflected from the retina, having a second polarization (s-polarization), incident on the coupling optical element 944, can be redirected by the coupling optical element 944 at an angle exceeding the critical angle of the waveguide 940, and thus induced within it. A portion of the light cannot be coupled into the waveguide 940 and will pass through it as uncoupled light 912. The internally coupled light 904 can propagate through the waveguide 940 toward the camera.

[0177] Other implementations may also benefit from the use of polarization-selective elements near the light source and camera. For example, various systems can be configured to provide illumination with a first polarization and use light with a different polarization to capture an image with a camera. For example, such a configuration may be used to reduce undesirable reflections from the cornea, etc., when imaging the retina. Reflections from the cornea will be specular. Therefore, if light of the first polarization is incident on the cornea, the light reflected from the cornea will retain its first polarization. In contrast, the retina is diffuse. If light of the first polarization is incident on the retina, the light reflected from the retina will not retain only the first polarization. Diffuse reflection is more likely to result in unpolarization. Therefore, a second polarization different from the first polarization will be present in the reflected light. Similarly, by illuminating with a first polarization and imaging with a second different polarization, the retina can be imaged with reduced glare from the cornea.

[0178] Therefore, in various implementations, both polarization-specific optical filters 982 and 984 may be used to reduce undesirable light reflected from the eye 210 (e.g., from the cornea). For example, undesirable light, glare, or flash that can saturate the image captured by the imaging device 920 may be reflected from the cornea. Light reflected from the cornea is specular and can maintain its polarization. In contrast, light reflected from the retina may be more diffusely reflected and may not be as uniformly polarized. Similarly, a combination of polarizers may be used to remove some or most of the undesirable reflected light. Initially, polarization can be used to illuminate the eye 210. In some designs, a polarized light source (e.g., light source 960) may be used. In addition, or alternatively, a first polarizer (e.g., polarization-specific optical filter 982) may be positioned at the beginning of the optical path of the light source to provide initial polarization of the light. A second polarizer (e.g., a polarization-specific optical filter 984) may be positioned in the optical path before the light enters the imaging device 920. The second polarizer is positioned from the first polarizer 90 oThey may be rotated (for example, polarizers 982 and 984 may "cross"). As a result, the eye will be illuminated with first polarization, and some of the light of first polarization will be reflected from the cornea. This light will not pass through the polarizer 984 proximal to the camera. However, the light reflected from the retina will contain second polarization. Similarly, the light diffusely reflected from the retina will pass through the polarizer 984 proximal to the camera, allowing the camera to capture an image of the retina. Thus, in such a configuration, undesirable light received from the eye (e.g., from the cornea) that may be incident on the imaging device 920 can be reduced or eliminated. Other configurations are also possible. For example, a polarization-selective internal coupling optical element 942 for coupling light from the light source 960 into the waveguide 940 and a polarization-selective external coupling optical element for coupling light out of the waveguide to the camera 920 may be employed so as to have different polarization-selective properties. For example, a polarization-selective internal coupling optical element can selectively redirect light from an illumination source with a first polarization into the waveguide, while an external coupling optical element can selectively redirect light with a second, different polarization out of the waveguide towards the camera. The effect can again be to reduce or remove undesirable light received from the eye (e.g., from the cornea) before it enters the imaging device 920.

[0179] Various imaging systems 900 capable of collecting light and imaging the retina using an eyepiece 950 are discussed herein. However, the imaging system 900 may also be configured to image other parts of the eye, such as the anterior portion of the eye. Figure 16 illustrates how the imaging system 900 may be used to image the anterior portion of the eye 210 (e.g., the cornea). The imaging system 900 may include one or more elements of the exemplary imaging system 900 described above. In addition, the exemplary imaging system 900 may include one or more refractive power optical elements or lenses having refractive power, such as refractive power optical elements 980, 990. For example, a positive refractive power lens or positive lens 980 may be positioned on the proximal side (e.g., closer to the eye 210) of the eyepiece 950 between the eye 210 and the eyepiece. A negative refractive power lens or negative lens 990 may be positioned on the distal side of the eyepiece 950 between the eyepiece and the environment in front of the user. One or both of the lenses 980, 990 may be a variable focus element (e.g., a variable focus lens) and / or may include a liquid crystal element. In some designs, one or both of the lenses 980, 990 include a Fresnel lens. The lenses 980, 990 may incorporate liquid crystals to produce Fresnel lens functionality. Such functionality may enable variable focus of one or both of the lenses 980, 990. In some designs, one or more of the lenses 980, 990 may be integrated with the eyepiece 950 and / or manufactured on or within it (e.g., formed).

[0180] In various embodiments, the coupled optical element 944 is configured to redirect collimated light reflected from the eye 210 into the optical waveguide, as induced therein. Therefore, the positive lens 980 may be configured to collimate light reflected from the eye 210, such as from the anterior portion of the eye (e.g., the cornea). The positive lens 980 may therefore have a focal length equal to or substantially equal to the distance of the lens to the portion of the eye 210 to be imaged, such as the cornea.

[0181] The negative lens 990 may have a refractive power similar to or identical to that of the positive lens 980, and may offset or cancel out the refractive power of the positive lens. In this way, light from the environment (e.g., distal to the waveguide 940) can pass through the negative lens 990, the eyepiece 950, and the positive lens 980, and substantially have no net change in the refractive power introduced by these two lenses. Therefore, the negative lens 990 may be configured to offset or cancel out the refractive power of the positive lens 980 so that the user will not suffer the refractive power of the positive lens when viewing the environment in front of the eyepiece 950. The negative lens 990 will also cancel out the effect of the positive lens 980, inverting the image of objects in the environment in front of the wearer.

[0182] Figure 16 illustrates light 928 incident on the cornea and scattered therefrom. The imaging system 900 may be configured to capture the in-beam light 988 reflected from the cornea. For example, a positive lens 980 may collect a portion of the light 988 scattered from the cornea and collimate the in-beam light 988. The in-beam light 988 collimated by the positive lens 980 is incident on a coupling optical element 944, which is configured to redirect the collimated light into the waveguide 940 at an angle greater than the critical angle of the waveguide, so that the light is guided in it by TIR. The coupling optical element 944, the external coupling optical element 952, and / or the waveguide 940 may be as described above. The resulting externally coupled light 906 can be directed out of the waveguide 940 to a camera (not shown) by the external coupling optical element 952.

[0183] Figure 16 shows light 928, such as collimated light, which may originate from the eyepiece 950 as described above. The illumination source 960 may couple the light into the waveguide 940, and the coupling element 944 may couple the light from the illumination source 960 out of the waveguide. The coupling element 944 may be configured to couple the light out of the waveguide 940 as collimated light. The light illuminates the anterior part of the eye (e.g., the cornea) and scatters from there. As discussed above, the scattered light 988 can be collected by the positive lens 980 and the imaging system 900 to form an image of the anterior part of the eye 210. Also, as discussed above, the illumination 928 directed onto the eye 210 may be invisible (e.g., infrared) light.

[0184] Figure 16 also shows alternative arrangements for illuminating the eye 210. In some designs, one or more light sources 934, such as LEDs or emitters, are positioned relative to the eye 210 and directed through a waveguide 940 by TIR, directing the light onto the eye 210 without being directed onto the eye itself. In some implementations, the eyepiece 950 or waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210. In some designs, multiple such light sources 934 may be arranged in a pattern (e.g., circular or ring-shaped pattern) near and / or around the eye. In some designs, the pattern of light sources 934 may define illumination axes parallel (e.g., coaxial) to the optical axes of one or more lenses 980, 990. The one or more light sources 934 may be similar to the one or more light sources 960 described above, and may, for example, be pulsed. Similarly, the one or more light sources 934 may comprise infrared light sources such as infrared LEDs or other types of invisible light. Alternatively, one or more light sources may be visible light sources that emit visible light. Or, one or more light sources may emit both visible and invisible (e.g., infrared) light.

[0185] Figure 17 illustrates another exemplary imaging system 900 configured to image a portion of the eye 210, such as the anterior portion of the eye (e.g., the cornea). The imaging system 900 shown in Figure 17 employs a reflective optical element 996 configured to collimate light from the eye, in contrast to the transmissive optical element (lens) 980 shown in Figure 16. Reflective optical elements will have less aberration than transmissive optical elements because chromatic aberration is generally not applicable to reflective optical elements such as the reflector 996 shown in Figure 17. Therefore, by using a reflective surface when collecting light from the eye 210, less (e.g., chromatic) aberration is introduced into the captured image of the eye.

[0186] Figure 17 illustrates an imaging system 900, which includes, for example, a curved transmissive optical element 996 having a wavelength-dependent reflective coating 998. The curved transmissive optical element 996 may be positioned distal to the waveguide 940 (on the environment side of the eyepiece 950). Thus, the curved transmissive optical element 996 may be positioned between the environment in front of the wearer and the waveguide 940 and / or the coupled optical element 944. Similarly, the waveguide 940 and / or the coupled optical element 944 may be positioned between the curved transmissive optical element 996 and the eye 210.

[0187] The wavelength-dependent reflective coating 998 may be configured to reflect light of a certain wavelength or range of wavelengths. In some implementations, for example, the wavelength-dependent reflective coating 998 may be configured to reflect invisible light (e.g., infrared light) within a certain range of wavelengths, while the wavelength-dependent reflective coating 998 may be configured to transmit visible light. In some cases, the wavelength-dependent reflective coating 998 may be placed on the surface of a curved transmissive optical element 996.

[0188] As discussed above, in various designs, the coupled optical element 944 is configured to redirect collimated light reflected from the eye 210 into the waveguide 940, as induced within it. Therefore, the reflective optical element 996 may be configured to collimate light reflected from the eye 210, such as from the anterior portion of the eye (e.g., the cornea). The curved reflective optical element 996 may therefore have a positive refractive power with respect to light incident on its proximal side reflected from the wavelength-dependent reflective coating 998. In particular, in various designs, the reflective optical element 994 may have a focal length equal to or substantially equal to the distance from the reflective optical element 996 to the portion of the eye 210 to be imaged, such as the cornea, iris, etc. Exemplary values ​​for the focal length may be, for example, 2 cm to 8 cm. In some implementations, the focal length is 4 cm to 6 cm. In some designs, the focal length is approximately 5 cm. The focal length may be within any range formed by any of these values, or, in different designs, may be outside such a range.

[0189] In various implementations, the reflective optical element 996 is positioned on the distal side of the eyepiece 950, in front of the eyepiece. Therefore, the reflective optical element 996 is positioned between the eyepiece 950 and the environment in front of the user. Similarly, the eyepiece 950 is positioned between the reflective optical element 996 and the eye 210.

[0190] The curved transmissive optical element 996 may have a curved reflective surface having a curvature of any shape. In some implementations, the surface is rotationally symmetric. In some implementations, the surface may be spherical or aspherical (e.g., parabolic). Non-rotationally symmetric shapes are also possible. However, in various designs, the reflective surface has a positive refractive power. The reflective optical element 996 may, for example, include a concave mirror for at least a certain wavelength and / or polarization.

[0191] The curved transmissive optical element 996 may be configured to have a negligible refractive power in transmission. Similarly, the curved transmissive optical element 996 may be configured to transmit light without introducing convergence or divergence. In one embodiment, the curved transmissive optical element 996 may have an inner radius curvature that is substantially the same as the outer radius curvature. The thin optical element 996 may reduce optical aberrations with respect to light transmitted through it, may be lighter, and / or more compact.

[0192] In various designs, the reflective optical element 996 includes a material that is transparent to visible light so that the user can see the environment in front of the wearer. In some cases, to improve transmission, the curved transparent optical element 996 may be coated with an anti-reflective coating on its outer surface (e.g., the distal surface). The anti-reflective coating may be configured to reduce the reflection of visible light, such as red, green, and / or blue light. However, the reflective optical element 996 may be configured to reflect some of the light scattered from the eye 210 to form an image of the eye. Therefore, the reflective optical element 996 may act differently with different light. For example, the reflective optical element 996 may act differently with different wavelengths. The reflective optical element 996 may be configured to reflect infrared light and transmit visible light.

[0193] As discussed above, one or more light sources 934 may be configured to illuminate the eye 210 with infrared light. The resulting light 988 reflected from the eye 210 (e.g., the cornea) may diverge, as schematically illustrated in Figure 17. A curved transmissive optical element 996 may be positioned to receive this light 988 reflected from the eye (e.g., the cornea, iris). A wavelength-dependent reflective coating 998 may be configured to reflect the light 988 reflected from the eye, since the wavelength illumination used to illuminate the eye is the same wavelength (e.g., 850 nm) reflected by the reflective coating on the curved transmissive optical element 996. For example, the eye may be illuminated with infrared light (e.g., 850 nm), and the curved transmissive optical element 996 may be configured to reflect infrared light (e.g., 850 nm) and allow visible light to pass through. The shape of the curved transmissive optical element 996 may also be configured to reflect the light to the coupling optical element 944, which collimates the light 988 reflected from the eye and redirects the collimated light into the waveguide 940 so that it is guided therein by TIR.

[0194] In Figure 17, as in some other designs, one or more light sources 934, such as LEDs or emitters, are positioned relative to the eye 210 and guided by TIR through a waveguide 940, directing the light onto the eye 210 without being directed onto it. In some implementations, the eyepiece 950 or waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210. In some designs, multiple such light sources 934 may be arranged in a pattern (e.g., circular or ring-shaped pattern) near and / or around the eye. In some designs, the pattern of light sources 934 may define illumination axes parallel (e.g., coaxial) to the optical axes of one or more lenses 980, 990. The one or more light sources 934 may be similar to the one or more light sources 960 described above, and may, for example, pulse oscillate. Similarly, the one or more light sources 934 may comprise infrared light sources such as infrared LEDs or other types of invisible light. However, other types of light sources can also be used.

[0195] Figure 18 illustrates another exemplary imaging system 900 configured to image a portion of the eye 210, such as the anterior portion of the eye (e.g., the cornea). In the implementation shown in Figure 18, polarization selectivity is employed to assist in controlling the path of light reflected from the eye. In particular, in various designs, the coupling optical element 944 is polarization selective. For example, light having a first polarization is transmitted through the coupling optical element 944, while light of a second different polarization is redirected into the waveguide 940 by the coupling optical element 944 so that it is coupled into it by TIR. Thus, in various implementations, a polarizer (not shown) is placed between the eye and the waveguide 940 so that the eye 210 is illuminated with polarization, or so that light from the eye incident on the waveguide is polarized. For example, the emitter 934 may emit polarization, or the polarizer may be placed in front of the emitter 934 so that the eye 210 is illuminated with polarization. Therefore, in various designs, the polarization of the polarized light that is received by the optical coupling element 944, incident on and / or reflected from the eye 210, may be a first polarization such that the light is directed toward the reflector 996.

[0196] Similarly, in various implementations, the coupled optical element 944 (and / or the external coupled optical element 952) is configured to transmit light in a first polarization state, such as a first linear, circular, or elliptic polarization state (e.g., p-polarization, left-hand circular polarization, or elliptic polarization), and to redirect light in a second polarization state, such as a second linear, circular, or elliptic polarization state (e.g., s-polarization, right-hand circular polarization, or elliptic polarization), into and out of the waveguide. In some implementations, the eye illuminator 934 may further include a polarization correction element (e.g., a polarizer) which may emit only or primarily emit the first polarization (e.g., p-polarization), or is configured to transmit only light in the first polarization state (e.g., p-polarization). In addition, the coupled optical element 944 and / or the external coupled optical element 952 may each be configured to redirect light in a second polarization state (e.g., s-polarization) into and out of the waveguide.

[0197] Similar to the imaging system 900 shown in Figure 17, the curved reflector 998 shown in Figure 17 comprises a curved transmissive optical element 996 having a wavelength-dependent reflective coating 998. The wavelength-dependent reflective coating 998 may be configured to reflect light of a certain wavelength or range of wavelengths. In some implementations, for example, the wavelength-dependent reflective coating 998 may be configured to reflect invisible light (e.g., infrared light) within a certain range of wavelengths, while the wavelength-dependent reflective coating 998 may be configured to transmit visible light. The wavelength-dependent reflective coating 998 may, in some cases, be located on the surface of the curved transmissive optical element 996.

[0198] In various implementations, the curved transmissive optical element 996 is positioned on the distal side of the eyepiece 950, in front of the eyepiece. Therefore, the reflective optical element 996 is positioned between the eyepiece 950 and the environment in front of the user. Similarly, the eyepiece 950 is positioned between the reflective optical element 996 and the eye 210.

[0199] Therefore, light having a first polarization (e.g., p-polarization) from the eye 210 is incident on the coupled optical element 944 and passes through it to the curved transmissive optical element 996. The imaging system 900 further includes a polarization correction optical element 978, such as a retarder (e.g., a quarter-wave retarder). The retarder 978 is transmissive and imparts a quarter-wave phase difference to the light transmitted through it. This light is incident on the curved transmissive optical element 996 and reflected from it. A wavelength-dependent reflective coating 998 may be configured to reflect light of a certain wavelength reflected from the eye. As a result, this light is reflected from the curved surface of the curved transmissive optical element 996 and collimated. This collimated light then passes again through the retarder 978, thereby imparting another quarter-wave phase difference to the light transmitted through it. The phase difference introduced in these two passes through the retarder (e.g., a full-wave phase difference) rotates the polarization. Therefore, the first polarization (e.g., p-polarization) transmitted through the polarization-selective coupling optical element 944 in the first pass is converted to a second polarization (s-polarization) and redirected into the waveguide 940 so that it is guided to the camera 920 by TIR. As discussed above, in various designs, the coupling optical element 944 is configured to redirect the collimated light reflected from the eye 210 into the waveguide 940 so that it is guided therein. Therefore, the reflective optical element 996 may be configured to collimate the light reflected from the eye 210, such as from the anterior portion of the eye (e.g., the cornea). The curved reflective optical element 996 may therefore have a positive refractive power. In particular, in various designs, the reflective optical element 994 may have a focal length equal to or substantially equal to the distance from the reflective optical element 996 to the part of the eye 210 to be imaged, such as the cornea or iris. Exemplary values ​​for the focal length may be, for example, 2 cm to 8 cm. In some implementations, the focal length is 4 cm to 6 cm. In some designs, the focal length is approximately 5 cm.

[0200] In various designs, the reflective optical element 996 may include a curved surface configured to reflect light. The curved surface may, in some cases, be spherical or rotationally symmetric. The reflective optical element 996 may include, for example, a concave mirror for at least a certain wavelength and / or polarization.

[0201] In various designs, the reflective optical element 996 includes a material that is transparent to visible light so that the user can see the environment in front of the wearer. A wavelength-dependent reflective coating 998 placed on the surface of the curved transmissive optical element 996 may therefore be transparent to visible light or at least some wavelengths of visible light. The curved transmissive optical element 996 may also be coated with an anti-reflective coating on its outer surface (e.g., the distal surface). The anti-reflective coating may be configured to reduce the reflection of red, green, and / or blue light. However, the reflective optical element 994 may be configured to reflect some of the light scattered from the eye 210 to form an image of the eye. Thus, the reflective optical element 996 may act differently with different light. For example, the reflective optical element 996 may act differently with light of different polarization states (and / or wavelengths). The reflective optical element 996 may be configured to transmit visible light and reflect infrared light.

[0202] As shown in Figure 17, one or more light sources 934, such as LEDs or emitters in Figure 18, are positioned relative to the eye 210 and may be directed upwards without being directed directly onto the eye 210, but rather guided through the waveguide 940 by TIR. Therefore, in some implementations, the eyepiece 950 or waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210. In some designs, multiple such light sources 934 may be arranged in a pattern (e.g., circular or ring-shaped pattern) near and / or around the eye. One or more light sources 934 may be similar to the one or more light sources 960 described above, and may, for example, pulse-oscillate. Similarly, one or more light sources 934 may comprise infrared light sources such as infrared LEDs or other types of invisible light. In particular, in various implementations, the light sources 934 may emit light reflected by wavelength-dependent reflective coatings 998 and / or curved transmissive optical elements 996. However, other types of light sources can also be used.

[0203] The polarization-selective coupling optical element 944 is configured to be polarization-selective depending on the type of linear polarization incident upon it, but other polarization-selective coupling optical elements may be polarization-selective to other types of polarization states, such as different types of circular or elliptically polarized light. The polarization-selective coupling optical element 944 may be configured such that, for example, a first polarization, such as a first circular or elliptically polarized light (e.g., left-hand circular polarization or LHP-polarization), is transmitted through the polarization-selective coupling optical element 944, and a second polarization, such as a second circular or elliptically polarized light (e.g., right-hand circular polarization or RHP), is redirected into the optical waveguide, or vice versa. Such a polarization-selective coupling optical element 944 may include a liquid crystal, such as a cholesteric liquid crystal. Examples of several liquid crystal optical elements are discussed in the section titled “Cholesteric Liquid Crystal Mirrors,” U.S. Patent Publication No. 2018 / 0164627, filed December 7, 2017, titled “Diffractive Devices Based on Cholesteric Liquid Crystal,” U.S. Patent Publication No. 2018 / 0239147, filed February 22, 2018, titled “Display System with Variable Power Reflector,” and U.S. Patent Publication No. 2018 / 0239177, filed February 22, 2018, titled “Variable-Focus Virtual Image Devices Based on Polarization Conversion,” each incorporated herein by reference, both in whole and for any purpose.

[0204] A polarization correction element or retarder, such as a circular polarizer, may be placed between the eye and the polarization-selective coupling optical element 944 to convert the light reflected from the eye to a first polarization (e.g., LHP). The LHP light will pass through the polarization-selective coupling optical element 944, be reflected by the reflector 998, change its polarization to RHP, and be redirected by the polarization-selective coupling optical element 944 towards the camera in the waveguide.

[0205] In some implementations, the reflector 996 may be polarization-selective in its reflectivity such that only light in a certain polarization state is reflected, and / or light in a different polarization state is transmitted. Such an optical element may comprise a liquid crystal, such as a cholesteric liquid crystal. Examples of such optical elements are discussed in the section titled “Cholesteric Liquid Crystal Mirrors,” U.S. Patent Publication No. 2018 / 0164627, filed December 7, 2017, titled “Diffractive Devices Based on Cholesteric Liquid Crystal,” U.S. Patent Publication No. 2018 / 0239147, filed February 22, 2018, titled “Display System with Variable Power Reflector,” and U.S. Patent Publication No. 2018 / 0239177, filed February 22, 2018, titled “Variable-Focus Virtual Image Devices Based on Polarization Conversion,” each incorporated herein by reference, both in whole and for any purpose. Such an optical element may reflect light in a first polarization state, such as a first circular or elliptically polarized state (left-hand circular or elliptically polarized), and transmit light in a second polarization state, such as a second circular or elliptically polarized state (e.g., right-hand circular or elliptically polarized), or vice versa. In some embodiments, the liquid crystal is positioned on the curved surface of the reflector 996 such that, in reflection, the reflector has a refractive power such as a positive refractive power. In various other implementations, the liquid crystal optical element may be flat or planar. For example, the liquid crystal may be positioned on a flat or planar substrate or layer. Despite being flat, refractive power may be contained within the liquid crystal optical element. Such an element may be called a cholesteric liquid crystal reflective lens. Thus, light from the eye can be collimated and reflected by the combined optical element 998. The reflector may, for example, reflect light in a first polarization state (e.g., left-hand circular or elliptically polarized) and transmit light in a second polarization state (e.g., right-hand circular or elliptically polarized).Therefore, the eye 210 is illuminated with left-hand circular polarization, or the light reflected from the eye is transmitted through a polarizer (e.g., a circular or elliptic polarizer) that transmits light having a first polarization (e.g., left-hand circular or elliptic polarization). The coupling optical element 944 may also be polarization-selective, transmitting LHP light and redirecting RHP light into the waveguide. LHP light from the eye passes through the coupling optical element 944. This transmitted LHP light is also incident on a wavelength-selective liquid crystal reflector 996 and reflected from there. In one design, the wavelength-selective liquid crystal reflector 996 converts the first polarization state (e.g., LHP) to a second polarization state (e.g., RHP) in response to the reflection. This light in the second polarization state (e.g., RHP light) is directed to the coupling optical element 944, which redirects the light in the second polarization state (RHP) into the waveguide 940 towards the camera 920.

[0206] In some designs, the coupled optical element 944 does not have a liquid crystal grating, but instead has, for example, a surface relief diffraction grating or a holographic grating. As discussed above, these coupled optical elements 944 that do not have cholesteric liquid crystals may also have a volume diffraction or holographic optical element or grating.

[0207] Therefore, the light scattered from the eye is reflected back to the waveguide 940 by the reflective optical element 996 so that it is coupled into the waveguide by the coupling element 944. However, in contrast, some of the unpolarized light from the environment in front of the wearer, corresponding to a second polarization state (e.g., RHP), will be transmitted through the reflective optical element 996. Thus, the wearer may see objects through the reflective optical element 996.

[0208] However, in various designs, the reflective optical element 996 will have a negligible refractive power in transmission. For example, the reflective optical element 996 may have curved surfaces on both sides of the optical element having the same curvature such that the total refractive power of the optical element for light transmitted through it will be negligible.

[0209] As discussed above, in various implementations, the reflective optical element 996 is described in the following sections: "Cholesteric Liquid Crystal Mirror," "Diffractive Devices Based on Cholesteric Liquid Crystal," U.S. Patent Publication No. 2018 / 0164627, filed December 7, 2017; and "Display System with Variable Power Reflector," U.S. Patent Publication No. 2018 / 0239147, filed February 22, 2018. The optical element comprises a cholesteric liquid crystal reflective lens, which is a cholesteric liquid crystal reflective element as discussed in U.S. Patent Publication No. 2018 / 0239177, filed on February 22, 2018, titled “CONVERSION” (each incorporated herein by reference, both in whole and for any purpose). Such an optical element can act on a specific wavelength or wavelength range. Thus, light such as infrared light reflected from the eye can be acted upon by the cholesteric liquid crystal reflective element. However, light outside its wavelength range, such as visible light from the environment, can pass through the cholesteric liquid crystal reflective element without being acted upon. Thus, the cholesteric liquid crystal reflective element can have a negligible refractive power for visible light from the environment passing through it.

[0210] As discussed above, in one implementation, the illumination source 960 couples light into the waveguide 940, redirecting it out of the waveguide, to illuminate the eye 210. In such an embodiment, the coupling optical element 944 may be polarization-selective. For example, the coupling optical element 944 may transmit a first polarization (p-polarization) and a second polarization (s-polarization).

[0211] Therefore, if light from the illumination source 906 propagates through the waveguide 940 and is redirected by the coupling optical element 944, the illumination will be s-polarized. A polarization correction optical element (e.g., a quarter-wave retarder) may be placed between the waveguide 940 and the eye 210 to cause a rotation of the polarization reflected from the eye. The light from the light source 960 reflected from the eye 210 passes twice through the quarter-wave retarder, and as a result, the s-polarized light emitted from the waveguide by the coupling element 944 to illuminate the eye will be converted to p-polarized light.

[0212] This p-polarized light will be transmitted through the coupled optical element 944 and the waveguide and incident on the reflective optical element 996.

[0213] The imaging system 900 may further include a second polarization correction element 978, which may include, for example, a retarder or a waveplate, as discussed above. The retarder may include, for example, a quarter-wave retarder. The second polarization correction element 978 may be located distal to the waveguide 940, i.e., between the waveguide and the reflector 996. The second polarization correction element 978 may also be located between the coupled element light 944 and the reflector 996. Light from the eye 210 (p-polarized) transmitted through the coupled element light 944 passes through the second polarization correction element 978 and is converted to circularly polarized light. If the reflector 996 reflects circularly polarized light, this light will again pass through the polarization correction element 978 and be reflected back to the waveguide 940. The two passes through this polarization correction element (e.g., a quarter-wave retarder) 978 will convert the light to s-polarized and redirect it into the waveguide by the coupling element 944 so that it is guided thereto to a camera (not shown).

[0214] As shown in Figure 18, the light 988 reflected from the eye 210 diverges. This light is incident on a reflector 996 that is curved or otherwise has positive refractive power, and can be collimated thereby. A coupling optical element 944, configured to redirect the collimated light into a waveguide 940, will therefore direct the collimated light from the curved reflective optical element 996 toward an imaging device 920 (not shown). Thus, the light reflected from the eye 210, collimated by the curved reflective optical element 996, is coupled into the waveguide 940 and guided toward an external coupling optical element 952. The external coupling optical element 952 may be configured to direct the light out of the eyepiece 950 toward a camera (not shown).

[0215] Various modifications are possible in the configuration of the imaging system. Different types of reflectors 996 and coupling elements 944 may be employed. The reflectors 996 and coupling elements 944 may be configured to act, for example, on linearly polarized or circularly or elliptically polarized light. As discussed, the reflector 996 has refractive power. The reflector 996 and coupling elements 944 may comprise a cholesteric liquid crystal grating reflector and / or lens, with or without refractive power. Polarization correction elements 978, such as retarders, may be included between the coupling element 944 and the reflector and / or between the coupling element 944 and the eye. In some embodiments, a polarizer, such as a circular polarizer or a linear polarizer, may be placed between the eye and the coupling element 944. For example, if unpolarized light is reflected from the eye, a polarizer (e.g., a circular polarizer or a linear polarizer) may be placed between the eye and the coupling element 944. In some such cases, the coupling element 944 is polarization selective.

[0216] In a configuration as shown in Figures 17 and 18, where light reflected from the eye passes through the waveguide 940 to the curved reflective optical element 996, is collimated, and is redirected back to the waveguide, background noise is introduced. This background noise originates from the light that initially passes from the eye through the coupled optical element 944. As discussed above, the coupled optical element 944 may be configured to redirect the collimated light into the waveguide 940 so that it is directed to the camera 920 in which an image is formed. However, the coupled optical element 944 will redirect some of the uncollimated light incident on it. Therefore, in the initial passage of light from the eye through the coupled optical element 944 and the waveguide 940 to the curved reflective optical element 996, some of the uncollimated (diverging) light reflected from the eye will be coupled into the waveguide by the coupled optical element 944 and contribute to the background noise of the image of the eye formed by the camera 920. This noise will be superimposed on an image formed by collimated light that is back-reflected by a curved reflective optical element 996, which is coupled into the waveguide by a coupling optical element 944 so as to be guided to the camera 920.

[0217] In one design, this noise can be removed from the image. The process for removing the noise from the signal may involve (a) measuring the amount of light (denoted as N) coupled by the coupled optical element 944 in its initial passage through the coupled optical element 944 to the curved reflective optical element 996, which is redirected and reaches the camera 920, and (b) measuring the total signal at the camera 920 as the light passes through the coupled optical element 944 and the waveguide 940 to the curved reflective optical element 996, is collimated, reflected back to the coupled optical element, and redirected to the camera. This total signal will also contain some noise N because uncollimated light reflected from the eye will pass through the coupled optical element 944 and reach the curved reflective optical element 996, and therefore some of the uncollimated light will be redirected by the coupled optical element 944 to the camera 920. If noise N can be measured separately from the total signal T, which includes noise superimposed across the eye image, then noise N can be removed from the total signal T, as expressed by the following equation: I = TN In the formula, I represents the image from which the noise component N has been removed.

[0218] The two measurements (a) and (b) described above can be obtained by various methods. For example, as shown in Figure 19, a shutter 936 can be placed between the curved reflective optical element 996 and the waveguide 940 and the coupled optical element 944. The shutter 936 may be configured to block light when the shutter is in a first state and to transmit light when the shutter is in a second state. The shutter 936 may include, for example, a liquid crystal shutter.

[0219] Therefore, the noise component N can be measured when the shutter 936 is in a first state in which light reflected from the eye 210 is incident on the coupled optical element 944 and passes through it toward the curved reflective optical element 996, however, the closed shutter prevents it from reaching the curved reflective optical element. As discussed above, a portion of the light reflected from the eye 210, though mostly uncollimated, is coupled into the coupled optical element 944, redirected into the waveguide, and guided there toward the camera 920. As referenced above, this light does not contribute to image formation but will become background noise. The camera 920 may record this noise N when the shutter 936 is closed.

[0220] The total signal T, including both noise N and the image, can be measured when the shutter 936 is in a second state, when the shutter is open. The light reflected from the eye 210 is again incident on the coupling optical element 944. A portion of the in-kind light reflected from the eye 210, though mostly uncollimated, is coupled into the coupling optical element 944, redirected into a waveguide, and guided thereto to the camera 920. However, the majority of the in-kind light reflected from the eye 210 passes through the coupling optical element 944, through the open shutter 936, to the curved reflective optical element 996. The curved reflective optical element 996 collimates the in-kind light, reflecting at least a portion of it back to the coupling optical element 944, which redirects the collimated in-kind light into the waveguide 920 so that it is guided to the camera 920 to form an image of the eye 210. The camera 920 can capture the in-kind image of the eye 210.

[0221] Processing electronic equipment (such as processing electronic equipment 140) communicating with camera 920 can receive a noise component N measured when shutter 936 is in a first closed state and a total signal T measured when shutter is in a second open state, and can subtract the two (TN). In this way, the noise N contributed by the uncollimated light reflected from eye 210 coupled into the coupling optical element 944 during the initial pass through it can be subtracted from the total image signal T. Processing electronic equipment may communicate with camera 920 via wired electronic signals. In addition, or alternatively, processing electronic equipment may communicate with camera 920 using one or more remote receivers. Processing electronic equipment may reside remotely (e.g., a cloud computing device, a remote server, etc.).

[0222] Other methods may be employed to perform measurements (a) and (b), obtain N and T, and subtract N from T. For example, if the curved reflective optical element 996 is wavelength-selective as shown in Figure 18, the eye can be illuminated with light of different wavelengths at different times. For example, to perform measurement (a) and quantify the noise N, the eye can be illuminated with wavelengths not reflected by the curved reflective optical element 996. However, to perform measurement (b) and quantify the total signal T, the eye can be illuminated with wavelengths reflected by the curved reflective optical element 996. The noise N can then be subtracted from the total T (e.g., TN) as discussed above.

[0223] FIG. 20-20E illustrates an exemplary imaging system 900 configured to perform measurements using wavelength modulation to remove noise components N as discussed above. The imaging system 900 in FIGS. 20A-20E includes a curved transmissive optical element 996 that is wavelength selective (such as those described in reference to FIGS. 17 and 18 above). For example, the curved transmissive optical element 996 has a wavelength-dependent reflective coating 998 on its curved surface. The imaging system 900 may also include one or more light sources or illumination sources (not shown) configured to illuminate the eye 210. The one or more light sources may be configured to emit infrared light. However, the one or more light sources can be configured to emit different colors or wavelengths of light at different times. Such wavelength modulation can enable separate measurement of N so that it can be removed from the total signal T.

[0224] In various implementations, for example, one or more illumination sources 960, 934, in a first state, one or more wavelengths λ Reflect that are reflected by the curved reflective optical element, and in a second state, one or more wavelengths λ Not Reflect that are not reflected. In the second state, a negligible amount or less of the wavelength λ Reflect that is reflected by the curved reflective optical element is emitted. Similarly, in the first state, a negligible amount or less of the wavelength λ Not Reflect that is not reflected is emitted.

[0225] In some embodiments, the wavelength λ Reflect that is reflected may be about 800 nm to 950 nm. The wavelength λ Reflect that is reflected may be about 835 nm to 915 nm. The wavelength λ Reflect that is reflected may be about 840 nm to 870 nm. In some designs, the wavelength λ Reflect that is reflected is about 850 nm. The light emission 928 from the one or more light sources 960 can illuminate the eye.

[0226] As shown in FIG. 20B, wavelengths λ that are not reflected by the curved reflective optical element 944Not Reflect Light 988 (and a negligible amount of light λ reflected by the curved reflective optical element 944) has the light λ Reflect ) is reflected from a part of the eye 210 (e.g., the cornea). This light is not reflected by the curved reflective optical element 944 at wavelength λ Not Reflect Therefore, the ray 916 is directed to propagate through the curved reflective optical element 996 into the environment in front of the user.

[0227] Light 988 incident on the coupled optical element 944 is not collimated, but the coupled optical element nevertheless couples at least some of the light 914 into the waveguide 940 so that it is guided to the camera 920. Thus, the camera 920 can capture an image (image #1) corresponding to the noise component N resulting from the uncollimated light redirected by the coupled optical element 944 during its initial passage through the curved reflective optical element 996. This image (image #1) is background noise and is not an image recognizable by the eye. Processing electronic equipment 140 is shown to receive this first image (image #1).

[0228] In Figures 20C-20E, an illumination source (not shown) is reflected by a curved reflective optical element at one or more wavelengths λ. Reflect and wavelengths λ less than or equal to a negligible amount that are not reflected Not Reflect It emits at this wavelength λ Reflect This could be, for example, 850 nm.

[0229] As shown in Figure 20C, during the first pass through the coupled optical element 944, a portion of the light 988 reflected from the eye 210, incident on the coupled optical element 944, is coupled into the waveguide 940 by the coupled optical element 944 (as shown in Figure 20B) and directed towards the camera 920. In addition, the wavelength λ ReflectThe curved transmissive optical element 996, which selectively reflects light, reflects and collimates internally uncoupled light 918 reflected from the eye 210 that is incident on the curved transmissive optical element. As shown in FIG. 20E, the coupling optical element 944 redirects the collimated reflected light and couples it into the waveguide 940 toward the camera 920. FIG. 20E shows both components reaching the camera 920, namely, light 988 that is incident on the coupling optical element 944 and reflected from the eye 210 during a first pass through the coupling optical element 944 and is coupled into the waveguide 940 by the coupling optical element, and light that is reflected and collimated by the curved transmissive optical element 996 and is coupled into the waveguide by the coupling optical element. The camera 920 may capture an image (Image #2) corresponding to the total image component T. The processing electronics 140 is shown to receive the second image (Image #2).

[0230] As discussed above, the processing electronics may subtract noise from the image T - N. In this example, Image #1 can be subtracted from Image #2. Thus, the processing electronics 140 may be configured to modify the second image based on the first image. However, other approaches are also possible. For example, the processing electronics 140 may be configured to create a new image representing a version of the second image with reduced optical noise. Implementations for subtracting noise from an image may be used in the implementations described above. For example, the implementations shown in FIGS. 10, 11A - 11E, and / or FIGS. 12A - 12E may include a shutter 936 and / or a curved transmissive optical element 996 having a wavelength - dependent reflective coating 998 configured to selectively reflect internally uncoupled light 912 and direct the light toward the imaging device 920.

[0231] As discussed above, Image #1 is at one or more wavelengths λ at which light is not reflected by the curved reflective optical element Not Reflect and at wavelengths λ at which the amount reflected is below an ignorable amount Reflectacquired for the case when illuminated. Image #2 is for one or more wavelengths λ at which light is reflected by the curved reflective optical element Reflect and wavelengths λ below a negligible amount that are not reflected Not Reflect acquired for the case when illuminated. Thus, one or more illumination sources 960, 934 may be configured to modulate the wavelength. For example, in one design, one or more illumination sources 960, 934 are one or more wavelengths λ that are not reflected by the curved reflective optical element Not Reflect and wavelengths λ below a negligible amount that are reflected Reflect and may comprise a first illumination source configured to output. One or more illumination sources may further comprise a second illumination source configured to output one or more wavelengths λ that are reflected by the curved reflective optical element Reflect and wavelengths λ below a negligible amount that are not reflected Not Reflect The intensities of the first and second illumination sources may alternatively be increased and decreased, turned on and off, attenuated and unattenuated, passed and blocked, to provide modulation of the wavelength of the light and illuminate the eye. For example, during a first time interval, the first illumination source may be blocked while the second illumination source is not blocked. During a subsequent second time interval, the second illumination source may be blocked while the first illumination source is not blocked. This process can be repeated to provide modulation of the wavelength of the light and illuminate the eye. In other designs, the wavelength of the light source may be adjusted and de - adjusted to shift the wavelength back and forth between λ Reflect and λ Not Reflect Other arrangements are also possible as considered

[0232] As described above, the imaging system 900 may also be incorporated within a head-mounted display, such as an augmented reality display, which provides the ability to image the eye by collecting light with an eyepiece 950. Such an imaging system 900 may be used for eye tracking. Multiple images of the retina or anterior portion of the eye may be acquired. Eye movement and / or repositioning can be confirmed from these images to track eye position and / or orientation. These imaging systems may also be used for biometric imaging and / or user identification. For example, imaging of the user's eye, such as the retina or iris, may be acquired and recorded. Subsequent images of the wearer's eye (e.g., retina or iris) may be acquired at a later time. The two images may be compared to determine whether the wearer in the subsequent instance is the same wearer as in the first instance. However, other uses for imaging systems are also possible.

[0233] The illumination system may be described above as waveguide-based and comprising one or more waveguides, but other types of optical redirection optical elements may be employed instead of waveguides. Such optical redirection optical elements include redirection features and may emit light from the optical redirection optical element onto, for example, a spatial light modulator. Thus, in any of the embodiments described herein and any of the embodiments below, any reference to waveguides may be replaced with optical redirection optical elements instead of waveguides. Such optical redirection optical elements may comprise, for example, a polarizing beam splitter such as a polarizing beam splitting prism.

[0234] As discussed above, the systems described herein can enable the collection of biometric data and / or biometric identification. For example, the eye or a part thereof (e.g., the retina) can be imaged to provide such biometric data and / or biometric identification. Images of the eye, such as the retina, may be acquired at various times when the head-mounted display system is worn by a user, possibly the same user. The collection of such images can be recorded, for example, in a database. These images may be analyzed to collect biometric data. Such biometric data may be useful for monitoring the user's health or medical status. Different medical parameters can be monitored by imaging the patient, for example, the patient's eye (e.g., the retina). The medical parameters can be recorded and compared together with subsequent measurements taken when the user is wearing the head-mounted display system.

[0235] In addition, if a person begins to wear a head-mounted display system and an image of the user's eyes is captured that does not match any images stored in the database, it may be concluded that the person currently wearing the head-mounted display system is different from the previous user. This can be useful in determining whether the intended user is wearing the headset or whether a new user is wearing it. Such features may enable certain medical, security, and / or user-friendly applications or functionalities. For example, a head-mounted display may be configured to identify the wearer based on the characteristics of the wearer's eyes. For example, the system may be configured to determine an individual based on the wearer's retina (e.g., blood vessels), corneal features, or other ocular features. In some implementations, for example, a set of markers may be determined for a particular wearer. Based on the set of markers, the system may be able to determine whether a previous user is wearing the headset or, as a replacement, another user is wearing the headset. The markers may include the shape or center of the user's cornea, the composition of blood vessels in the user's retina, the intensity and / or location of light reflection from the cornea, the shape of the side of the eye, and / or any other biometric markers. In some implementations, a confusion matrix can be determined. As discussed above, for example, in the discussion of developing a retinal map using virtual / fixed targets at various locations (see, for example, Figure 13B), the system may have the user look at a given set of directions or eye poses and develop a matrix of the properties of the eye or part of the eye (e.g., cornea, retina, etc.) associated with each direction or eye pose. Using such a matrix, the system can determine the identification of an individual. Other methods are also possible.

[0236] Similarly, as discussed above, various configurations of the system are also possible. For example, Figure 21 shows an exemplary eyepiece 900 that can be used to project light into the user's eye while simultaneously imaging the user's eye. The shown eyepiece 900 includes an internal coupled optical element 2104, a light-dispersing element 2108, a light-gathering element 2116, and an external coupled optical element 2120 on the opposite side of the coupled optical element 2112. Each of these optical elements may be located inside or on a waveguide 2102. The waveguide 2102 may correspond to, for example, one of the waveguides 670, 680, 690 described herein (see, for example, Figures 9A–9C). The internal coupling optical element 2104 may correspond to one of the internal coupling optical elements 700, 710, 720 and / or the internal coupling optical element 942 (see, for example, Figure 10) described herein, and may be configured to load image content from the projector into the waveguide and / or load illumination from the light source 960. The light dispersion element 2108 may correspond to one of the light dispersion elements 730, 740, 750 (see, for example, Figures 9A-9C) described herein, and may be used to diffuse light in a given direction and to redirect light from the internal coupling optical element 2104 to the coupling optical element 2112. The coupling optical element 2112 may correspond to the coupling optical element 944 (see, for example, Figure 10) described herein. In some designs, the coupling optical element 2112 includes the functionality described herein for the external coupling optical elements 800, 810, 820 (see Figures 9A-9C). The focusing element 2116 may be configured to reduce the lateral spatial range of the light received from the coupling optical element 2112 and to redirect the light toward the external coupling optical element 2120. The external coupling optical element 2120 may correspond to the external coupling optical element 952 described herein (see, for example, Figure 10).

[0237] The internal coupling optical element 2104 may be positioned inside or on the waveguide 2102 to receive light from a projector (e.g., image projector 930) and / or an illuminator (e.g., light source 960), etc. The light may pass through the waveguide 2102 to the associated light-dispersing optical element 2108. The internal coupling optical element 2104, the light-dispersing optical element 2108, or the coupling optical element 2112 may be positioned on the main surface of the waveguide (e.g., on the top surface or bottom surface) or inside the waveguide. Similarly, any one or combination of the focusing element 2116 and / or the external coupling optical element 2120 may also be positioned on the main surface of the waveguide 2102 (e.g., on the top or both main surfaces) or inside the waveguide.

[0238] The combined optical element 2112 can receive light from the light-dispersing element 2108 (e.g., via TIR), expand the light, and direct it into the user's eye. Thus, the combined optical element 2112 can be positioned in front of the user's eye and project image content onto it. In addition, or alternatively, the combined optical element 2112 may be configured to provide illumination light above and / or into the user's eye.

[0239] Light reflected from the eye (e.g., illumination light from an illumination source) can be reflected and captured by the coupling optical element 2112. Thus, in some embodiments, the coupling optical element 2112 can play both roles: externally coupling the light received from the optical dispersion element 2108 and internally coupling the light received from the eye into the waveguide 2102.

[0240] In some embodiments, the coupling optical element 2112 may include one or more diffractive optical elements (DOEs) such that the coupling optical element 2112 has dual functionality. The first DOE (e.g., a grating, a holographic region) may also be configured to externally couple light, and the second DOE may be configured to internally couple light reflected from the eye into the waveguide 2102. In some embodiments, both the first and second DOEs are superimposed within the waveguide 2102 (e.g., occupying the same or substantially the same volume).

[0241] Alternatively, in some embodiments, the coupled optical element 2112 includes at least two DOEs stacked across or in front of the other. For example, referring to Figure 21, the first DOE of the coupled optical element 2112 may be positioned across the second diffraction element, while the second diffraction element is positioned below the first DOE. The order of each DOE may be reversed in other implementations. Cholesteric LCD Mirror

[0242] Some liquid crystals exist in a phase referred to as the chiral phase or cholesteric phase. In the cholesteric phase, the liquid crystal may exhibit molecular torsion along an axis perpendicular to the oriented element, while the molecular axis is parallel to the oriented element. As described herein, a cholesteric liquid crystal (CLC) layer comprises a plurality of liquid crystal molecules in the cholesteric phase that extend in a direction perpendicular to the oriented element, such as the layer depth direction, and are continuously rotated or twisted in a rotational direction, such as clockwise or counterclockwise. The oriented element of the liquid crystal molecules in the chiral structure may be characterized as a spiral having a helical pitch (p), which corresponds to a length in the layer depth direction that corresponds to the net rotation angle of the liquid crystal molecules in the chiral structure by one rotation in a first rotational direction. In other words, the helical pitch refers to the distance over which the liquid crystal molecules undergo a complete 360° twist. A chiral liquid crystal can also be described as having a torsion angle or rotation angle (φ) which may refer to, for example, the relative azimuthal rotation between continuous liquid crystal molecules in the layer normal direction, or a net torsion angle or rotation angle which may refer to, for example, the relative azimuthal rotation between the uppermost and lowermost liquid crystal molecules across a defined length, e.g., the length of the chiral structure or the thickness of the liquid crystal layer. As described herein, a chiral structure refers to a plurality of liquid crystal molecules in a cholesteric phase that extend in a direction perpendicular to the orientor, e.g., the layer depth direction, and are continuously rotated or torsioned in a rotational direction, e.g., clockwise or counterclockwise. In one aspect, the orientor of the liquid crystal molecules in a chiral structure can be characterized as a spiral with a helical pitch.

[0243] Figure 22 shows a cross-sectional side view of a cholesteric liquid crystal (CLC) layer 1004 comprising a plurality of uniform chiral structures according to an embodiment. In the CLC layer 1004, adjacent chiral structures in the lateral direction, for example, in the x-direction, have similarly arranged liquid crystal molecules. In the illustrated embodiment, the chiral structures 1012-1, 1012-2, ... 1012-i are similarly configured such that the liquid crystal molecules of different chiral structures at approximately the same depth, for example, the liquid crystal molecules closest to the light incident surface 1004S, have the same rotation angle, as do the continuous rotation angles of the continuous liquid crystal molecules at approximately the same depth, and the net rotation angles of the liquid crystal molecules of each chiral structure.

[0244] CLC1004 comprises a CLC layer 1008 having liquid crystal molecules arranged as multiple chiral structures 1012-1, 1012-2, ..., 1012-i, where each chiral structure comprises multiple liquid crystal molecules, and i is any preferred integer greater than 2. During operation, incident light having a combination of a left-hand circularly polarized beam and a right-hand circularly polarized beam is incident on the surface 1004S of the CLC layer 1008 by Bragg reflection. Light with one of the circular polarization pendulums is reflected by the CLC layer 1004, while light with the opposite circular polarization pendulum is transmitted through the CLC layer 1008 with virtually no interference. As described herein and throughout this disclosure, pendulum is defined as being visible in the direction of propagation. According to the embodiment, incident light is reflected when the polarization direction or polarization palmarity of the light beams 1016-L, 1016-R are matched such that they have the same rotational direction as the liquid crystal molecules of the chiral structures 1012-1, 1012-2, ..., 1012-i. As shown in the illustration, the incident light on the surface 1004S becomes a light beam 1016-L with left-hand circular polarization and a light beam 1016-R with right-hand circular polarization. In the illustrated embodiment, the liquid crystal molecules of the chiral structures 1012-1, 1012-2, ..., 1012-i are continuously rotated clockwise, and in that direction the incident light beams 1016-L, 1016-R travel, i.e., in the positive x-direction, which is the same rotational direction as the light beam 1016-R with right-hand circular polarization. As a result, the light beam 1016-R, which has right-hand circular polarization, is substantially reflected, while the light beam 1016-L, which has left-hand circular polarization, is substantially transmitted through the CLC layer 1004.

[0245] As described above, by matching the polarization palmarity of incident elliptical or circularly polarized light to the rotational direction of the liquid crystal molecules in the chiral structure of the CLC layer, the CLC layer can be configured as a Bragg reflector. Furthermore, one or more CLC layers having different helical pitches can be configured as wavelength-selective Bragg reflectors with high bandwidth. Based on the concepts described herein with respect to various embodiments, the CLC layer can be configured as an off-axis or on-axis mirror configured to selectively reflect a first wavelength range, e.g., infrared wavelengths (e.g., near-infrared), while transmitting another wavelength range, e.g., visible wavelengths.

[0246] Figure 23 illustrates an embodiment of an eye-tracking system 2300 employing a cholesteric liquid crystal reflector (CLCR), such as a wavelength-selective CLCR 1150, configured to image the viewer's eye 302, according to various embodiments. Unlike the CLC layer 1004 described above with respect to Figure 22, the chiral structures within the wavelength-selective CLCR 1150 adjacent to each other in the lateral direction, for example, in the x-direction, have liquid crystal molecules arranged differently. That is, the chiral structures are configured such that liquid crystal molecules with different chiral structures at approximately the same depth, for example, the liquid crystal molecules closest to the light incident surface 1004S, have different rotation angles. As a result, the light incident on the CLCR 1150 is angled (θ) with respect to the layer depth direction, as will be further described below in the context of the eye-tracking system 2300. R It is reflected by ).

[0247] Eye tracking can be a useful feature in interactive vision or control systems, including wearable display systems described herein, particularly for virtual / augmented / mixed reality display applications, among other uses. To achieve effective eye tracking, it may be desirable to acquire images of the eye 302 at a low eye-level angle, and in this regard, it may be desirable to position the eye tracking camera 702b near the center of the viewer's eye. However, such a position of camera 702b may interfere with the user's view. Alternatively, the eye tracking camera 702b may be positioned lower or to the side. However, such a position of the camera may increase the difficulty in obtaining robust and accurate eye tracking because the eye image is captured at a steeper angle. By configuring the CLCR1150 to selectively reflect infrared (IR) light 2308 (for example, having a wavelength of 850 nm) from the eye 302 while transmitting visible light 2304 from the world, the camera 702b can be positioned away from the user's view while capturing eye images at a vertical or low eye-level angle. Such a configuration does not interfere with the user's view because visible light is not reflected. The same CLCR1150 can also be configured as an IR illuminator 2320 by reflecting IR light from an IR source, such as an IR LED, into the eye 302, as shown in the figure. The low eye-level angle of the IR illuminator may result in less occlusion from, for example, eyelashes, which is a configuration that allows for more robust detection of specular reflections and can be a useful feature in modern eye-tracking systems.

[0248] Referring still to Figure 23, according to various embodiments, the CLCR1150 comprises one or more cholesteric liquid crystal (CLC) layers, each comprising a plurality of chiral structures, and each chiral structure comprises a plurality of liquid crystal molecules extending in the layer depth direction (e.g., the z-direction) and continuously rotated in a first rotational direction, as described above. The arrangement of the liquid crystal molecules in the chiral structures varies periodically in a lateral direction perpendicular to the layer depth direction, such that one or more CLC layers are configured to substantially Bragg reflect a first incident light having a first wavelength (λ1) while substantially transmitting a second incident light having a second wavelength (λ2). As described above, each of the one or more CLC layers is configured to substantially Bragg reflect first and second elliptical or circularly polarized incident light having polarization palmarity that matches a first rotational direction when viewed in the layer depth direction, while substantially transmitting first and second elliptical or circularly polarized incident light having polarization palmarity opposite to the first rotational direction when viewed in the layer depth direction. According to the embodiment, the arrangement of liquid crystal molecules, which fluctuates periodically in the lateral direction, is arranged to have a lateral period such that the ratio of the first wavelength to the period is about 0.5 to about 2.0. According to the embodiment, the first wavelength is in the near-infrared range of about 600 nm to about 1.4 μm, for example, about 850 nm, and the second wavelength is in the visible range having one or more colors as described elsewhere herein. According to the various embodiments, the chiral liquid crystal molecules are pre-tilted with respect to a direction perpendicular to the layer depth direction. As configured, one or more CLC layers are configured such that the first incident light reaches approximately 50 degrees in the layer depth direction. o , about 60 o , about 70 o , or about 80 o An angle (θ) greater than the depth direction (z-direction) of the layer R It is configured to be reflected by ).

[0249] Thus, the wavelength-selective CLCR1150 comprises one or more cholesteric liquid crystal (CLC) layers, each containing a plurality of liquid crystal molecules extending in the layer depth direction and continuously rotated in a first rotation direction, wherein the arrangement of the chiral liquid crystal molecules is periodically varied in a lateral direction perpendicular to the layer depth direction, such that one or more CLC layers substantially Bragg reflect a first incident light having a first wavelength, e.g., an IR wavelength, while substantially transmitting a second incident light having a second wavelength, e.g., a visible wavelength.

[0250] Similar liquid crystal layers and structures may be used for the reflector 996 and coating 998 described above in relation to Figure 17-20E. The coating 998 may, for example, comprise a liquid crystal coating and, in some implementations, may be wavelength and / or polarization selective. However, other types of coatings 998 and reflectors 996 may be employed.

[0251] For example, as discussed above in relation to Figure 16, lens 980 may be used to modify (e.g., collimate) the propagation of light directed to the coupled optical element 944. This light may be light reflected from the user's eye, such as the anterior surface of the user's eye (e.g., the corneal surface). The distance from the eye, e.g., the anterior surface (e.g., the corneal surface) to the coupled optical element 944 may be, for example, about 20 mm. A positive lens 980, such as a lens with a focal length of about 20 mm, may be configured to collimate light reflected from the eye 210, such as the anterior portion of the eye (e.g., the cornea). The light reflected from the anterior surface of the eye may be coupled into a waveguide 940 and guided thereto to a camera. By using a focal length set to the distance to the anterior surface of the eye, the camera can image such a surface. Therefore, in various implementations, the positive lens 980 may have a focal length that is equal to or substantially equal to the distance to the portion of the eye 210 that should be imaged from the lens, for example, to the cornea.

[0252] Although refractive optical elements are shown, other types of lenses or optical elements with refractive power, such as positive refractive power, may be used. For example, the lens may include diffractive optical elements such as diffractive lenses or holograms. In some implementations, such lenses may be disposed between the eye and the coupling optical element 944.

[0253] In various implementations, the coupled optical element 944 may include refractive power. The coupled optical element 944 may, for example, comprise a diffractive optical element having refractive power. The diffractive optical element may, for example, comprise a diffraction grating. The diffractive optical element may comprise a holographic optical element or a hologram. The diffractive optical element may have diffraction features, such as surface features, configured to both redirect light into the waveguide and provide refractive power. Other types of diffractive optical elements are also possible. In various implementations, the diffractive optical element may comprise a liquid crystal or a liquid crystal grating. The diffractive optical element may also comprise a polarizing grating. In addition, the diffractive optical element may comprise a liquid crystal polarizing grating. Several non-limiting embodiments of liquid crystal gratings, liquid crystal polarizing gratings, and other liquid crystal optical elements are incorporated herein by reference, respectively, as a whole and for all purposes, by the following published applications: U.S. Patent Publication No. 2018 / 0143438, titled "MULTILAYER LIQUID CRYSTAL DIFFRACTIVE GRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES," filed November 16, 2017; U.S. Patent Publication No. 2018 / 0143485, titled "SPATIALLY VARIABLE LIQUID CRYSTAL DIFFRACTION GRATINGS," filed November 16, 2017; and "WAVEGUIDE LIGHT MULTIPLEXER USING CROSSED This is discussed in U.S. Patent Publication No. 2018 / 0143509, filed on November 16, 2017, titled "GRATINGS," U.S. Patent Publication No. 2018 / 0239177, filed on February 22, 2018, titled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION," and U.S. Patent Publication No. 2018 / 0164627, filed on December 7, 2017, titled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL."

[0254] The diffractive optical element may have refractive power that modifies the propagation of light incident upon it. The diffractive optical element may collimate reflected light as a surface having a distance from the diffractive optical element corresponding to the focal length of the diffractive optical element. Such a distance may be, for example, about 15–20 mm (e.g., 20 mm or an approximation thereof). Such a focal length can provide collimation of light reflected from anterior surfaces of the eye, such as the corneal surface (e.g., the cornea). Other distances are also possible. For example, the distance may be in the range of about 10–40 mm, 10–50 mm, 5–40 mm, or 5–50 mm, or any range between any of the distance values ​​specified herein. Values ​​outside these ranges are also possible.

[0255] Figure 24 shows an embodiment of a coupled optical element (e.g., a coupled grating) 2111 positioned on the waveguide 2102 of the eyepiece 950. The coupled optical element 2111 comprises a diffractive optical element configured to couple light incident on it into the waveguide 2102. The diffractive optical element also includes refractive power. For example, the diffractive optical element includes diffractive features such as surface diffraction features, configured to collimate light incident on it from, for example, the anterior surface of the eye (e.g., the corneal surface), which provide refractive power. A focusing element 2116 and an external coupled optical element 2120 are also shown positioned on the waveguide 2102. The focusing element 2116 is positioned to receive light coupled into the waveguide 2102 by the coupled element 2111. The focusing element 2116 is configured to redirect light incident on it from the coupled optical element 2111 to the external coupled optical element 2120. The focusing element 2116 is configured to reduce the lateral spatial range of light (e.g., a light beam) from the at least one coupling element before it reaches the at least one external coupling optical element 2120. In some configurations, fewer optical elements may be used, for example, to reduce cost and / or optical loss, or for other reasons. For example, the focusing element 2116 may be omitted in some embodiments. In such embodiments, the light may be internally coupled from the input coupling element 2111 (e.g., after being reflected from the eye 210) and directly coupled to the external coupling optical element 2120. The light may propagate through the eyepiece 950 between the input coupling element 2111 and the external coupling optical element 2120. Other configurations are also possible. A camera is positioned relative to the external coupling optical element 2120 and receives light from it. The external coupling optical element 2120 is configured to direct the light received from the focusing element 2116 towards the camera in order to capture an image.

[0256] Figure 24 also shows an internal coupling optical element 2104, a light dispersion element 2108, and an image content external coupling optical element 2110, which are arranged on the waveguide 2102. The internal coupling optical element 2104 may be configured to couple light received from the image projector into the waveguide 2102. The light distribution element 2108 may be configured to redirect the light received from the internal coupling optical element 2104 to the external coupling optical element 2110, and in addition, to increase the spatial range of the light, as discussed above. The external coupling optical element 2110 may be configured to couple light induced in the waveguide 2102 out of the waveguide and direct such light towards the eye in order to view image content from the projector.

[0257] In various implementations, one or more of these optical elements 2111, 2116, 2120, 2110, 2108, and 2104 may be located inside or on the waveguide 2102. Similarly, as discussed above, one or more of these optical elements 2111, 2116, 2120, 2110, 2108, and 2104 may comprise diffractive optical elements.

[0258] In the implementation illustrated in Figure 24, the coupled optical element (e.g., coupled grating) 2111 may be displaced laterally from the external coupled optical element 2110 on the waveguide 2102. In the implementation shown, space separates the coupled optical element 2111 laterally from the external coupled optical element 2110.

[0259] Figure 25 shows another similar implementation in which the coupled optical element (e.g., coupled grating) 2111 is displaced laterally from the external coupled optical element 2110 on the waveguide 2102. However, in the implementation shown in Figure 25, space does not laterally separate the coupled optical element 2111 from the external coupled optical element 2110.

[0260] Displacing the coupled optical element (e.g., coupled grating) 2111 laterally from the external coupled optical element 2110 on the waveguide 2102 allows the coupled optical element to include refractive power such that, for example, the refractive power collimates the light received from and / or through the external coupled optical element 2110 to the eye. The image presented to the eye from the image projector and the view of the environment in front of the user and head-mounted display do not need to be affected by the refractive power of the coupled optical element 2111 (e.g., distorted or blurred).

[0261] Figure 26 illustrates an implementation of the imaging system 900 configured to image multiple parts of the eye. For example, the imaging system 900 shown in Figure 26 may be configured to image both the anterior surface of the eye (e.g., the corneal surface) and the retina. The imaging system 900 includes a pair of internally coupled optical elements, namely a first internally coupled optical element 2111a and a second internally coupled optical element, located on the waveguide 2102 of the eyepiece. The imaging system 900 also includes a pair of focusing elements, namely a first focusing optical element 2116a and a second focusing element 2116b, located on the waveguide 2102. In some configurations, fewer optical elements may be used, for example, to reduce cost and / or optical loss, or for other reasons. For example, the focusing elements 2116a and 2116b may be omitted in some embodiments. In such embodiments, light may be internally coupled from the first and second input coupling elements 2111a, 2111b (after being reflected from, for example, a part of the eye 210 such as the retina and / or cornea) and directly coupled to the corresponding first and second external coupling optical elements 2120a, 2120b. The light may propagate through the eyepiece 950 (for example, via a waveguide such as a waveguide 2102) between the first and second input coupling elements 2111a, 2111b and the corresponding external coupling optical elements 2120a, 2120b. Other configurations are also possible. In addition, the imaging system 900 includes a pair of optical external coupling optical elements, namely a first external coupling optical element 2120a and a second external coupling optical element 2120b, which are located on the waveguide 2102.

[0262] The first coupling optical element 2111a is configured to couple light incident on it into the waveguide 2102. The first focusing element 2116a is positioned to receive the light that is coupled into the waveguide 2102 by the first coupling element 2111a. The first focusing element 2116a is configured to redirect the light incident on it from the first coupling optical element 2111a to the first external coupling optical element 2120a. The first focusing element 2116a is also configured to reduce the lateral spatial range of the light from the first coupling element 2111a (e.g., a light beam) prior to reaching the first external coupling optical element 2120a. A camera is positioned relative to the first external coupling optical element 2120a and receives light from it. The camera is shown in Figure 26, but an area corresponding to the camera's detection area 2130 is shown. The first external coupling optical element 2120a is configured to direct the light received from the first light-gathering element 2116a towards the camera, specifically the detection area 2130 shown in Figure 26, in order to capture an image.

[0263] Similarly, the second coupling optical element 2111b is configured to couple the light incident on it into the waveguide 2102. The second focusing element 2116b is positioned to receive the light that is coupled into the waveguide 2102 by the second coupling element 2111b. The second focusing element 2116b is configured to redirect the light incident on it from the second coupling optical element 2111b to the second external coupling optical element 2120b. The second focusing element 2116b is also configured to reduce the lateral spatial range of the light from the second coupling element 2111a (e.g., a light beam) prior to reaching the second external coupling optical element 2120a. A camera is positioned relative to the second external coupling optical element 2120b and receives light from it. The camera is shown in Figure 26, but an area corresponding to the camera's detection area 2130 is shown. The second external coupling optical element 2120b is configured to direct the light received from the second light-gathering element 2116b towards the camera, specifically the detection area 2130 shown in Figure 26, in order to capture an image.

[0264] Figure 26 also shows an internal coupling optical element 2104, a light dispersion element 2108, and an image content external coupling optical element 2110, which are arranged on the waveguide 2102. The internal coupling optical element 2104 may be configured to couple light received from the image projector into the waveguide 2102. The light distribution element 2108 may be configured to redirect the light received from the internal coupling optical element 2104 to the external coupling optical element 2110, and in addition, to increase the spatial range of the light, as discussed above. The external coupling optical element 2110 may be configured to couple light induced in the waveguide 2102 out of the waveguide and direct such light towards the eye in order to view image content from the projector.

[0265] In the implementation illustrated in Figure 26, the first and second coupled optical elements (e.g., coupled gratings) 2111a and 2111b are displaced laterally from the external coupled optical element 2110 on the waveguide 2102. In the implementation shown, space separates the first and second coupled optical elements 2111a and 2111b laterally from the external coupled optical element 2110. Furthermore, in the embodiment of Figure 26, the first and second coupled optical elements 2111a and 2111b are displaced laterally from each other on the waveguide 2102. In other implementations, it is not necessary for two or more of the first coupled optical element (e.g., coupled grating) 2111a, the second coupled optical element 2111b, and the external coupled optical element 2110 to be displaced laterally from each other on the waveguide 2102.

[0266] In various implementations, one or more of these optical elements 2111a, 2111b, 2116a, 2116b, 2120a, 2120b, 2110, 2108, and 2104 may be located inside or on the waveguide 2102. Similarly, as discussed above, one or more of these optical elements 2111a, 2111b, 2116a, 2116b, 2120a, 2120b, 2110, 2108, and 2104 may comprise diffractive optical elements.

[0267] The imaging system 900 in Figure 26 can be configured to image multiple parts of the eye. For example, the imaging system 900 may be configured to image both the anterior surface of the eye (e.g., the corneal surface) and the retina. The first internal coupling optical element 2111a may have, for example, a lens having or with refractive power, positioned in front of it. The first coupling optical element 2111a may also include a diffractive optical element configured to couple light into the waveguide 2102, for example, but also configured to impart refractive power thereto. In addition, or alternatively, a lens having refractive power may be positioned in front of the first coupling optical element 2111a. The refractive power may be configured to modify the propagation of light received by the first internal coupling optical element 2111a so that a specific part of the eye can be imaged. In some embodiments, the refractive power may be positive. Furthermore, the refractive power may be such that the anterior surface of the eye can be imaged. The refractive power may correspond to a focal length of, for example, about 15–25 mm (e.g., about 20 mm). As a result, light reflected from the anterior surface of the eye, which may be about 15–25 mm (e.g., about 20 mm) from the first coupled optical element 2111a, can be collimated and coupled into the waveguide. Other focal lengths are also possible. For example, the focal length may be in the range of about 10–40 mm, or 10–50 mm, 5–40 mm, or 5–50 mm, or any range between any of the distance values ​​specified herein. Values ​​outside these ranges are also possible.

[0268] In contrast, in various implementations, the second coupled optical element 2111b may have no refractive power and may not include a lens positioned in front of it. The lack of refractive power associated with the second coupled optical element 2111b would result in light from the anterior part of the eye (e.g., the corneal surface) not being collimated and therefore not being imaged by the camera. However, light reflected from the retina may be collimated as it passes through the user's natural lens. This light, collimated by the eye's natural lens, may be coupled into the waveguide 2102 by the second coupled optical element 2111b and imaged by the camera. Thus, the light collected by the first coupled optical element 2111a may image the anterior surface of the eye (e.g., the corneal surface), and the second coupled optical element 2111b may image the retina of the user's eye. In some implementations, such a configuration allows light collected by the first combined optical element 2111a to form an image of a flash on the eye, for example, the anterior surface of the eye (e.g., the corneal surface of the eye). The second combined optical element 2111b may image the retina of the user's eye, as discussed above. In some embodiments, the second combined optical element 2111b and / or the lens positioned in front of it may have a certain amount of net refractive power (e.g., a non-zero amount of net refractive power) that is weaker than that of the first combined optical element 2111a and / or the lens positioned in front of the first combined optical element 2111a. In particular, the second combined optical element 2112b may have refractive power and / or a lens associated therewith, however, the total refractive power of the second combined optical element 2112b and / or any lens associated with the second combined optical element is less than the refractive power of the first combined optical element and / or any lens associated with the first combined optical element.

[0269] In some implementations, the image formed by the light collected by the first coupled optical element 2111a is adjacent to (e.g., not superimposed on) the image formed by the light collected by the second coupled optical element 2111b. For example, an image of the anterior surface of the user's eye (e.g., the cornea) may be formed adjacent to (e.g., not superimposed on) the image of the retina.

[0270] Polarization techniques can be used to attenuate or remove light from the anterior surface (e.g., the corneal surface) so as not to affect the image formed by the light collected by the second coupling optical element 2111b. For example, the eye can be illuminated with polarized light having a first polarization, and the camera can form an image using light from the second external coupling optical element 2120b using light of a second different polarization. For example, the second external coupling optical element 2111b may be a polarization-selective coupling element that selectively externally couples light of a second polarization different from the first polarization. In addition, or alternatively, a polarizer 2140 that filters out the first polarization (for example, selectively transmits the second polarization) may be included between the second external coupling optical element 2120b and the camera 920, as shown in Figure 27, which is a cross-section through the waveguides 950, 2102, focusing optical elements 2116a, 2116b, and external coupling optical elements 2120a, 2120b, as shown in Figure 26.

[0271] Such a configuration may be used to reduce undesirable reflections (e.g., flashes) from the cornea or other sources when imaging the retina. Reflections from the cornea would be specular reflections. Therefore, when light of a first polarization is incident on the cornea, the light reflected from the cornea would retain the first polarization. In contrast, the retina is diffusive. When light of a first polarization is incident on the retina, the light reflected from the retina would not simply retain the first polarization. Diffusive reflection is more likely to result in unpolarized light. Therefore, a second polarization, different from the first polarization, would be present in the light reflected from the retina. As a result, an image of the retina would be acquired by forming an image using light coupled outward from a second external coupling optical element 2120b using light of the second polarization, while images of the cornea or flashes would be suppressed. Similarly, by illuminating with the first polarization and imaging with the second different polarization, the retina can be imaged with reduced glare from the cornea.

[0272] Therefore, in various implementations, polarization-specific optical filters or polarization-selective optical elements (e.g., coupled gratings) may be used to reduce undesirable light reflected from the eye 210 (e.g., the cornea). For example, undesirable light, glare, or flash may be reflected from the cornea so as to saturate the image captured by the camera. As discussed above, light reflected from the cornea is specular and can maintain its polarization. In contrast, light reflected from the retina may be more diffusely reflected and may not be as uniformly polarized. Similarly, a combination of polarizers may be used to remove some or most of the undesirable light reflected from the cornea. Initially, polarization can be used to illuminate the user's eye. In some designs, a polarized illumination source (e.g., a light source) may be used. In addition, or alternatively, a first polarizer (e.g., a polarization-specific optical filter or polarization-selective optical coupled element that couples illumination light into an illumination waveguide) may be positioned at the beginning of the optical path of the illumination source to provide the initial polarization of the light to the eye. A second polarizer (e.g., a polarization-specific optical filter or a polarization-selective coupling element) may be positioned in the optical path before light enters the camera. The second polarizer is positioned 90 degrees from the first polarizer. oThe polarizers may be rotated (for example, the polarizers may be "crossed"). As a result, the eye will be illuminated with first polarization, accompanied by first polarization light reflected from the cornea. This light will not pass through the crossed polarizer located proximal to the camera (which preferentially passes second polarization light). However, the light reflected from the retina will contain second polarization. Similarly, the light diffusely reflected from the retina will pass through the proximal polarizer 2140 of the camera, allowing the camera to capture an image of the retina. Thus, in such a configuration, undesirable light received from the eye (e.g., the cornea) and incident on the camera can be reduced or eliminated from the image captured using light from the second coupled optical element 2111b. Other configurations are also possible. For example, polarization-selective coupled elements 2111b and / or polarization-selective external coupled optical elements 2120b may be used in addition to, or as a substitute for, polarizers such as the proximal polarizer 2140 of the camera. In addition, or alternatively, a polarization source may be used for illumination (e.g., to illuminate the eye). This effect may again be to reduce or remove unwanted light received from the eye (e.g., the cornea) before it enters the imaging device 920.

[0273] As illustrated in Figure 27, in various implementations, such polarizers are not used in the optical path from the first coupled optical element 2111a and the camera, or between the first external coupled optical element 2120a and the camera. As a result, images of the cornea and flash can be obtained from the light coupled into the waveguide by the first coupled optical element 2111a and / or the light coupled out of the waveguide from the first external coupled optical element 2120a. As discussed above, the first coupled optical element 2111a may have refractive power or a lens associated therewith, which is specifically used to image the cornea and / or flash. Similarly, a polarization-selective coupled optical element 2111a or a polarization-selective external coupled optical element 2120a that filters out the first polarization of light would not be used as the first coupled optical element 2111a and the external coupled optical element 2120a, respectively. In addition, polarizers 2140, which filter out the first polarized light, will not be used, either between the coupled optical element 2111a and the camera 920 or between the external coupled optical element 2111a and the camera 920.

[0274] Various modifications are possible. For example, while the first and second external coupling optical elements 2120a and 2120b are described above as coupling light to form an image on a single camera (e.g., a single detection area 2130), in other implementations, the first and second external coupling optical elements 2120a and 2120b may couple light to form an image on separate first and second cameras. Other modifications are also possible.

[0275] In addition, in some implementations, one or more of the eyepieces described above with reference to Figure 24-27 (e.g., eyepiece 950) may be a dedicated imaging eyepiece layer (e.g., omitting the internal coupling optical element 2104, the optical dispersion element 2108, and / or the external coupling element 2110). In such implementations, the imaging eyepiece layer may be included as a layer in the waveguide stack. One or more other layers in the waveguide stack may include the internal coupling optical element 2104, the optical dispersion element 2108, and / or the external coupling element 2110.

[0276] In some embodiments, a dedicated imaging eyepiece layer may be configured to capture an image of the environment. In some such embodiments, the imaging eyepiece layer may be positioned closest to the environment when deployed in a head-mounted display. In some such configurations, the dedicated imaging eyepiece layer may be the outermost layer in a waveguide stack so as to be positioned between other layers (e.g., waveguides) and the environment. In some embodiments, the dedicated imaging eyepiece layer may be configured to capture an image of the user's eye 210. In some such embodiments, the imaging eyepiece layer may be positioned closest to the user when deployed in a head-mounted display. In some implementations, the dedicated imaging eyepiece layer may be the innermost layer in a waveguide stack so as to be positioned between other layers (e.g., waveguides) and the user. Other configurations are also possible.

[0277] In some embodiments, the first and second coupling elements 2111a, 2111b are aligned laterally but may be displaced in depth (for example, along the z-axis, i.e., towards the far side of the page in Figure 26). For example, in such embodiments, the first and second coupling elements 2111a, 2111b may be located on opposite sides of the same waveguide. In some embodiments, the first input coupling element 2111a may be located on or inside the first waveguide, and the second input coupling element 2111b may be located on or inside the second waveguide. In addition, or alternatively, the first and second focusing elements 2116a, 2116b and / or the first and second external coupling optical elements 2120a, 2120b may be located on opposite sides of the same waveguide and / or on or inside corresponding separate waveguides.

[0278] Referring to Figure 24-27, one or more of the coupled optical elements 2111, 2111a, and 2111b described above may be wavelength-selective such that the optical element is configured to interact only with a certain wavelength or band of wavelengths. The wavelength or band of wavelengths may include invisible light (e.g., infrared light or its specific band). In some implementations, the first input coupled element 2111a may be configured to act at a lower wavelength than that for the second input coupled element 2111b (or vice versa). For example, the first coupled optical element 2111a may be configured to interact with light at about 800 nm, and / or the second coupled optical element 2111b may be configured to interact with light at about 950 nm (or vice versa). Optical filters may also be used to provide similar wavelength selectivity. In various implementations, the light coupled into the waveguide by the first coupling optical element, which reaches at least one camera, can therefore have a different wavelength than the light coupled into the waveguide by the second optical element, which also reaches at least one camera. Other configurations are also possible.

[0279] Various modifications are possible in the structure and design of the coupling optical element 2120 and the external coupling optical element 2012. For example, the size and shape of the coupling optical element 2120 for coupling light into the waveguide 2102 and the external coupling optical element 2112 for coupling light out of the waveguide 2102 to the camera, in particular the coupling region of the coupling optical element 2120 and the size and shape of the external coupling optical element 2112 can be varied. As used herein, the coupling region refers to the area of ​​an optical element configured to receive light and couple the light into or out of the waveguide for use by the system, for example, to image (e.g., the environment in front of the eye or the user). In some implementations, the coupling region may correspond to the area of ​​the optical element as a whole or a portion of the area of ​​the optical element configured for use in the system to couple light into or out of the waveguide for use by the system. For example, in some implementations, a diffractive optical element is used to couple light (from the environment in front of the eye or user) into a waveguide. The coupling region of the coupling element is the area of ​​the diffractive optical element used by the system to couple light into the waveguide. The coupling optical element, e.g., a diffraction grating or diffractive optical element, may in some cases have a larger coupling region. For example, opaque objects such as opaque elements or layers having small apertures within them can obstruct the propagation of light to part of the diffraction grating or diffractive optical element, thereby reducing the coupling region. Thus, the size and shape of the coupling region can be controlled by the size and surroundings (e.g., the spatial range of the diffractive optical element or grating) of the optical element and other optical elements or other components or features that can block light from reaching part of the optical element. Other factors may also potentially reduce, alter, or otherwise affect the size and shape of the coupling region.

[0280] Figures 28A and 28B show a coupling optical element 2120 for coupling light into a waveguide 2102, the coupling optical element having a reduced size. In particular, the coupling optical element 2120 has a pinhole-sized coupling region. The size of the coupling region may be, for example, about 1.5 mm × 1.5 mm. As shown in Figure 28A, for example, the coupling region of the coupling optical element 2120 may have a length L and a thickness T. Both the length and thickness may be about 1.5 mm in some implementations. The reduced size of the coupling region eliminates the collection of multiple (e.g., afterimages) images by the coupling optical element 2120. The pinhole-sized coupling region of the coupling optical element 2120 for coupling light from an object such as the user's eye or an object in the environment in front of the user and eyewear has an effect similar to that of a pinhole camera with respect to light collection and the resulting imaging. The reduced coupling region size of the coupling optical element 2120 is similar to the reduced size of the pinhole camera coupling region. As a result, a large depth of field is provided without the need for a lens. The combined optical element 2120 also does not need to have refractive power, and a lens does not need to be provided to the combined optical element 2120. Nevertheless, a wide range of object distances are in focus with such a design.

[0281] The external coupling optical element 2112, shown in Figures 28A and 28B, for coupling light from the waveguide 2102 to the camera, is also reduced in size. In this particular implementation, the size and shape of the coupling optical element 2120 and the external coupling optical element 2112 are similar or identical. The external coupling optical element 2112 may also be 1.5 × 1.5 mm.

[0282] As discussed above, the coupling optical element 2120 and the external coupling optical element 2112 may include diffractive optical elements such as diffraction gratings. The coupling optical element 2120 and the external coupling optical element 2112 may include holographic or holographic optical elements. As discussed herein, the coupling optical element 2120 and the external coupling optical element 2112 may include liquid crystals, liquid crystal gratings, polarizing gratings, and / or liquid crystal polarizing gratings. The coupling optical element 2120 may be configured to receive light (e.g., from the user's eye or the environment in front of the user) and couple at least a portion of that light into the waveguide 2102. The coupling optical element 2120 may also be configured to redirect a portion of the light and direct it towards the external coupling optical element 2112. The external coupling optical element 2112 may be configured to couple the light received from the coupling optical element 2120 and induced in the waveguide 2102 out of the waveguide, for example, to a camera. The shown configuration does not include any focusing optical elements as described above.

[0283] The size and shape of the coupling regions of the coupling optical element 2120 and the external coupling optical element 2112 may vary. For example, although the coupling optical element 2120 and the external coupling optical element 2112 are shown as square, the shape of one or both may differ. In some implementations, the respective sizes and relative placements of the optical elements 2120 and 2112 may be selected, at least in part, based on a desired distance from the user's eye (i.e., focal length), the wavelength of light to be captured, or both. The desired distance between the waveguide and the user's eye may be, for example, about 15 mm to 25 mm. Other configurations are also possible. In some embodiments, the optical elements 2120 and 2112 may be arranged, for example, on or inside the waveguide, 15 mm to 25 mm apart from each other. The shape of the coupling region may be, for example, circular or otherwise rounded. The size of the coupling region may also differ. For example, the dimension (e.g., length L or thickness T) (e.g., average or maximum value) along one direction of the joint region may be 0.1–3 mm, 0.1–0.3 mm, 0.1–0.5 mm, 0.5–3 mm, 0.5–2 mm, 1–3 mm, 1–2 mm, 0.5–2.5 mm, 1.0–2.5 mm, 1.0 mm–1.5 mm, or 1.5 mm–2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. Similarly, the dimensions (e.g., length L or thickness T) (e.g., maximum or average value) along the other direction of the joint region may be 0.1–3 mm, 0.1–0.3 mm, 0.1–0.5 mm, 0.5–3 mm, 0.5–2 mm, 1–3 mm, 1–2 mm, 0.5–2.5 mm, 1.0–2.5 mm, 1.0 mm–1.5 mm, or 1.5 mm–2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. In some implementations, these two directions may be orthogonal. The dimensions along the two directions do not need to be identical. Therefore, the joint region may be symmetrical or asymmetrical.For example, the aspect ratio of the bond region (measured by a length-to-thickness ratio, such as maximum length to maximum thickness or average length to average thickness) may be 1–2, 1–1.75, 1–1.5, 1–1.3, 1–1.2, or 1–1.1, or any range formed by any of these values. Values ​​outside these ranges are also possible.

[0284] As discussed above, other shapes are also possible. For example, Figures 29A and 29B show a coupling optical element 2120 for coupling light into the waveguide 2102 and an external coupling optical element 2112 for coupling light out of the waveguide to the camera, with the coupling optical element 2120 having an arc-shaped slit coupling region. (The external coupling optical element 2112 has a pinhole coupling region similar to that shown in Figures 28A and 28B.)

[0285] A slit can be advantageous in that it is larger than a pinhole and therefore can collect more light. However, the narrowness of the slit provides an effect similar to that of a pinhole. For example, a reduced thickness T of the coupling region can be used to remove multiple images so that they are not collected by the coupling optical element 2120. Afterimages and / or blur can be reduced thereby. As shown in Figure 25A, the coupling region of the coupling optical element 2120 may have a length L and a thickness T. The thickness T may be about 1.0 mm or 1.5 mm in some implementations. The pinhole-sized thickness T of the coupling optical element 2120 for coupling light from an object such as an eye or an object in the environment in front of the user and eyewear has an effect similar to that of a pinhole camera with respect to light collection and the resulting imaging. The reduced size of the coupling region of the coupling optical element 2120 is similar to the reduced size of the coupling region of a pinhole camera. As a result, a large depth of field is provided without the need for a lens. The combined optical element 2120 does not need to have refractive power, nor does it need to be supplied with a lens. Nevertheless, a wide range of object distances can be focused with such a design.

[0286] The external coupling optical element 2112, shown in Figures 29A and 29B, for coupling light from the waveguide to the camera, is also reduced in size. In this particular implementation, the size and shape of the external coupling optical element 2112 are similar to or identical to those described above in relation to Figures 28A and 28B. The external coupling optical element 2112 may also be, for example, 1.5 × 1.5 mm.

[0287] As discussed above, the coupled optical element 2120 and the external coupled optical element 2112 may comprise diffractive optical elements such as diffraction gratings. The coupled optical element 2120 and the external coupled optical element 2112 may comprise holograms or holographic optical elements. As discussed herein, the coupled optical element 2120 and the external coupled optical element 2112 may comprise liquid crystals, liquid crystal gratings, polarizing gratings, liquid crystal polarizing gratings, or any combination thereof. The coupled optical element 2120 may be configured to receive light (e.g., from the user's eyes or the environment in front of the user) and couple at least a portion of that light into the waveguide 2102. The coupled optical element 2120 may also be configured to redirect a portion of the light and direct it towards the external coupled optical element 2112. The external coupled optical element 2112 may be configured to couple the light received from the coupled optical element 2120 and guided to the external coupled optical element 2112 within the waveguide 2102 out of the waveguide, for example, to a camera. The configuration shown does not include the light-gathering optical elements described above.

[0288] The shape of the coupling region of the coupling optical element 2120 and the external coupling optical element 2112 may vary. Theoretically, a curved slit may potentially facilitate the directing of light from the coupling optical element 2120 to the external coupling optical element. Furthermore, in some implementations, the arc-shaped slit may have a curvature described by the radius of curvature and the center of curvature. The curvature of the slit may be such that the external coupling optical element 2112 is at the center of curvature of the arc-shaped slit. However, the curvature may be different, for example, larger or smaller. Further variations of the shape are also possible. For example, the edges of the arc-shaped slit may be rounded.

[0289] The size of the joint region may also vary. For example, the shorter dimension along one direction of the joint region, i.e., the thickness T, may be 0.1–0.3 mm, 0.1–0.5 mm, 0.3–0.5 mm, 0.2–0.5 mm, 0.1–1.0 mm, 0.3–1.0 mm, 0.5–3 mm, 0.5–2 mm, 1–3 mm, 1–2 mm, 0.5–2.5 mm, 1.0–2.5 mm, 1.0 mm–1.5 mm, or 1.5 mm–2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. This dimension, i.e., the thickness T, may be smaller than the other dimension, i.e., the length L. The other dimension, i.e., the length L, may correspond to the length from one end to the other or the resulting displacement. Alternatively, the path length P, in this case the arc length, may be used to provide a measurement of the larger dimension of the joint region. In some implementations, the length L, path length, or arc length P may be 5mm–40mm, 10–40mm, 10–30mm, 15–30mm, 15–25mm, 1–5mm, or 3–5mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. In some implementations, these two directions may be orthogonal, however they do not need to be orthogonal. The slit is asymmetrical. For example, the aspect ratio of the joint region (e.g., measured by the ratio of length L to thickness, where length can be either the resulting displacement from one end to the other or the path length such as arc length P) may be 5–100, 10–100, 15–100, 20–100, 10–40, or 10–50, or any range formed by any of these values. Values ​​outside these ranges are also possible. In some embodiments, the dimension of the coupling region of the coupling optical element 2120 along one direction may be less than or equal to 2.5% of the distance between the center of the coupling optical element 2120 and the center of the external coupling optical element 2112. For example, the center-to-center distance between the optical elements 2120 and 2112 may be about 20 mm, and this dimension (e.g., thickness T) may be less than or equal to about 0.5 mm (e.g., 2.5% of 20 mm). Other configurations or values ​​are also possible.For example, the dimension (e.g., thickness T) may be less than or equal to 2%, 1.5%, and / or 1% of the distance between the center of the combined optical element 2120 and the center of the external combined optical element 2112. Dimensions referred to herein as thickness T, length L, path length P, etc., may be single measurements, average values, maximum values, or minimum values.

[0290] The shape and size of the external coupling optical element 2112 may also vary. For example, although the external coupling optical element 2112 is shown as a square, its shape may differ. The shape of the coupling region may be, for example, circular or otherwise rounded. The size of the coupling region may also differ. For example, the dimension along one direction of the coupling region, e.g., thickness T, may be 0.1-0.3 mm, 0.1-0.5 mm, 0.3-0.5 mm, 0.5-3 mm, 0.5-2 mm, 1-3 mm, 1-2 mm, 0.5-2.5 mm, 1.0-2.5 mm, 1.0 mm-1.5 mm, or 1.5 mm-2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. Similarly, the dimensions along the other direction of the joint region, for example, length L, may be 0.1–0.3 mm, 0.1–0.5 mm, 0.3–0.5 mm, 0.5–3 mm, 0.5–2 mm, 1–3 mm, 1–2 mm, 0.5–2.5 mm, 1.0–2.5 mm, 1.0 mm–1.5 mm, or 1.5 mm–2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. In some implementations, these two directions may be orthogonal. The dimensions along the two directions do not need to be identical. Therefore, the joint region may be symmetrical or asymmetrical. For example, the aspect ratio of the joint region (e.g., measured by the ratio of length L to thickness T, where length may be either the resulting displacement from one end to the other or the path length such as arc length P) may be 1 to 2, 1 to 1.75, 1 to 1.5, 1 to 1.3, 1 to 1.2, or 1 to 1.1, or any range formed by any of these values. Values ​​outside these ranges are also possible. As stated above, dimensions referred to herein as thickness T, length L, path length P, etc., may be single measurements, average values, maximum values, or minimum values.

[0291] As discussed above, different shapes and sizes of coupling regions are also possible. Figures 30A and 30B show a coupling optical element 2120 for coupling light into the waveguide 2102, having a non-arc-shaped slit coupling region. In particular, the coupling optical element 2120 has a linear slit coupling region. The slit is rectangular in shape. In contrast, the external coupling optical element 2112 has a pinhole-sized coupling region, similar to those shown in Figures 28A and 28B and Figures 29A and 29B.

[0292] As discussed above, a slit is advantageous in that it has a larger pinhole and therefore can collect more light. However, the narrowness of the slit provides an effect similar to that of a pinhole. For example, a reduced thickness T of the coupling region can eliminate multiple images from being collected by the coupling optical element 2120. Afterimages and / or blur can be reduced thereby. As illustrated in Figure 30A, the coupling region of the coupling optical element 2120 may have a length L and a thickness T. The thickness T may be about 1.0 mm or 1.5 mm in some implementations. The pinhole size thickness T of the coupling optical element 2120 for coupling light from an object such as an eye or an object in the environment in front of the user and eyewear has an effect similar to that of a pinhole camera with respect to light collection and the resulting imaging. The reduced size of the coupling region of the coupling optical element 2120 is similar to the reduced size of the aperture of a pinhole camera. As a result, a large depth of field is provided without the need for a lens. The combined optical element 2120 does not need to have refractive power, nor does it need to be supplied with a lens. Nevertheless, a wide range of object distances can be focused with such a design.

[0293] The external coupling optical element 2112, shown in Figures 30A and 30B, for coupling light from the waveguide to the camera, is also reduced in size. In this particular implementation, the size and shape of the external coupling optical element 2112 are similar to or identical to those described above in relation to Figures 28A and 28B and 29A and 29B. The external coupling optical element 2112 may also be, for example, 1.5 × 1.5 mm.

[0294] As discussed above, the coupling optical element 2120 and the external coupling optical element 2112 may comprise diffractive optical elements such as diffraction gratings. The coupling optical element 2120 and the external coupling optical element 2112 may comprise holograms or holographic optical elements. As discussed herein, the coupling optical element 2120 and the external coupling optical element 2112 may comprise liquid crystals, liquid crystal gratings, polarizing gratings, liquid crystal polarizing gratings, or any combination thereof. The coupling optical element 2120 may be configured to receive light (e.g., from the user's eyes or the environment in front of the user) and couple at least a portion of that light into the waveguide 2102. The coupling optical element 2120 may also be configured to redirect a portion of the light and direct it towards the external coupling optical element 2112. The external coupling optical element 2112 may be configured to couple the light received from the coupling optical element 2120 and directed to the external coupling optical element 2112 in the waveguide 2102 out of the waveguide, for example, to a camera. The configuration shown does not include the light-gathering optical elements described above.

[0295] As illustrated, the shape of the coupling region of the coupling optical element 2120 and the external coupling optical element 2112 may vary. As discussed above, theoretically, a curved slit can potentially facilitate the directing of light from the coupling optical element 2120 to the external coupling optical element. However, non-arc-shaped slits can also be used. A straight or rectangular slit, as shown in Figures 30A and 30B, can direct enough light to the external coupling optical element 2112 to capture an image using a camera. Other variations in shape are also possible. For example, the edges of the slit may be rounded.

[0296] The size of the joining region may also vary. For example, the shorter dimension along one direction of the joining region, i.e., the thickness T, may be 0.1–0.3 mm, 0.1–0.5 mm, 0.3–0.5 mm, 0.5–3 mm, 0.5–2 mm, 1–3 mm, 1–2 mm, 0.5–2.5 mm, 1.0–2.5 mm, 1.0 mm–1.5 mm, or 1.5 mm–2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. This dimension, i.e., the thickness T, may be smaller than the other dimension, i.e., the length L. The other dimension, i.e., the length L, may correspond to the length from one end to the other or the resulting displacement. As discussed above, alternatively, the path length P may be used to provide a measurement of the larger dimension of the joining region. In this embodiment, if the slit is straight, the length L, as measured by the displacement from one end to the other, is the same as the path length P. In some implementations, the length L or path length or arc length P may be 1–5 mm, 1–3 mm, 5–40 mm, 10–40 mm, 10–30 mm, 15–30 mm, or 15–25 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. As will be discussed, these two directions may be orthogonal with respect to a straight or rectangular slit in some implementations, however the directions do not need to be orthogonal. The slit is asymmetric. For example, the aspect ratio of the joint region (e.g., measured by the ratio of length L to thickness T, where length can be either the resulting displacement from one end to the other or the path length such as arc length P) may be 5–100, 10–100, 15–100, 20–100, 10–40, or 10–50, or any range formed by any of these values. Values ​​outside these ranges are also possible. As described above, dimensions referred to in this specification as thickness T, length L, path length P, etc., may be a single measurement, average value, maximum value, or minimum value.

[0297] The shape and size of the external coupling optical element 2112 may also vary. For example, although the external coupling optical element 2112 is shown as square, the shape of one or both may differ. The shape of the coupling region may be, for example, circular or otherwise rounded. The size of the coupling region may also differ. For example, the dimension along one direction of the coupling region, e.g., thickness T, may be 0.1-0.3 mm, 0.1-0.5 mm, 0.3-0.5 mm, 0.5-3 mm, 0.5-2 mm, 1-3 mm, 1-2 mm, 0.5-2.5 mm, 1.0-2.5 mm, 1.0 mm-1.5 mm, or 1.5 mm-2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. Similarly, the dimensions along the other direction of the joint region, for example, length L, may be 0.1–0.3 mm, 0.1–0.5 mm, 0.3–0.5 mm, 0.5–3 mm, 0.5–2 mm, 1–3 mm, 1–2 mm, 0.5–2.5 mm, 1.0–2.5 mm, 1.0 mm–1.5 mm, or 1.5 mm–2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. In some implementations, these two directions may be orthogonal. The dimensions along the two directions do not need to be identical. Therefore, the joint region may be symmetrical or asymmetrical. For example, the aspect ratio of the joint region (e.g., measured by the ratio of length L to thickness T, where length may be either the resulting displacement from one end to the other or the path length such as arc length P) may be 1 to 2, 1 to 1.75, 1 to 1.5, 1 to 1.3, 1 to 1.2, 1 to 1.1, or any range formed by any of these values. Values ​​outside these ranges are also possible. Dimensions referred to herein as thickness T, length L, path length P, etc. may be single measurements, average values, maximum values, or minimum values ​​(e.g., average thickness, maximum thickness, average length, maximum length).

[0298] Figure 31 shows an eyepiece including a coupling optical element 2112 for coupling light into a waveguide 2102 and an external coupling optical element 2120 for coupling light out of the waveguide 2102 to a camera. The coupling optical element 2112 has a non-arc-shaped (e.g., straight-rectangular) slit-shaped coupling region, which includes an image content internal coupling optical element 2104 for receiving light from an image projector, a light distribution element 2108 for directing light from the image content internal coupling optical element 2104 to the external coupling optical element 2110, and the external coupling optical element 2110 for coupling light induced in the waveguide to the user for viewing the image content. In some implementations, the coupling optical element 2112 has an associated lens that includes or provides refractive power. This refractive power can collimate light from an object at a specific distance, such as from the anterior surface of the eye (e.g., the cornea), and facilitate image capture of that object (e.g., a flash on the cornea). The refractive power of the first combined optical element 2112a and / or the lens associated therewith may correspond to focal lengths in the range of approximately 15–25 mm, 10–40 mm, 10–50 mm, 5–40 mm, or 5–50 mm, or any range between any of the specified distance values. Values ​​outside these ranges are also possible.

[0299] The slit-coupled optical element 2112 is displaced laterally from the external-coupled optical element 2110. In other embodiments, such as those shown in Figures 30A and 30B, the system 900 does not need to be included on the eyepiece with such components for presenting the image to the user.

[0300] Figures 32A and 32B show first and second coupling optical elements 2112a, 2112b for coupling light into the waveguide 2102, and first and second external coupling optical elements 2120a, 2120b for coupling light out of the waveguide to a camera, wherein the coupling optical elements 2112a and 2112b have a non-arc-shaped (e.g., straight-rectangular) slit-shaped coupling region. Such a configuration may be useful for imaging different objects simultaneously. In some implementations, for example, different parts of the eye (or flashes on them), such as the retina and cornea, can be imaged using a pair of coupling optical elements 2112a, 2112b, a pair of external coupling optical elements 2120a, 2120b, and one or more cameras. In some implementations, the first combined optical element 2112a has refractive power or is associated with a lens, while the second combined optical element 2112b does not have similar refractive power or similar lens (e.g., it has neither refractive power nor an associated lens). In some implementations, the second combined optical element 2112b may have refractive power and / or an associated lens, however the total refractive power of the second combined optical element 2112b and / or any lens associated with the second combined optical element is less than the refractive power of the first combined optical element and / or any lens associated with the first combined optical element. As discussed above, such a system 900 may be configured, for example, to use the first combined optical element 2112a to image the anterior portion of the eye and to use the second combined optical element 2112b to image the retina. As discussed above, in some implementations, the eye is illuminated with light having a first polarization, and has a polarizer associated with it, the second external coupling optical element 2120b is polarimetrically selective or configured to filter out the light of the first polarization so as not to degrade the image of the retina, such as flashes of light.

[0301] Figure 33 shows first and second coupling optical elements 2112a, 2112b for coupling light into the waveguide 2102, and first and second external coupling optical elements 2120a, 2120b for coupling light out of the waveguide 2102 to the camera, with coupling optical elements 2112a and 2112b comprising non-arc-shaped (e.g., straight-rectangular) slit-shaped coupling regions integrated on the eyepiece. The eyepiece further includes an image content internal coupling optical element 2104 for receiving light from the image projector, and a light distribution element 2108 for directing light from the internal coupling optical element 2104 to an external coupling optical element 2110 for coupling light induced in the waveguide 2102 to the user for viewing the image content. In other embodiments, such as those shown in Figures 32A and 32B, the system 900 does not need to be mounted on a waveguide with such components 2104, 2108, and 2110 in order to present images to the user.

[0302] Figures 34A and 34B show first and second coupling optical elements 2112a, 2112b for coupling light into the waveguide 2102, and first and second external coupling optical elements 2120a, 2120b for coupling light out of the waveguide to a camera, wherein the coupling optical elements 2112a, 2112b have a non-arc-shaped (e.g., straight-rectangular) slit-shaped coupling region. Such a configuration may be useful for imaging different objects simultaneously. In some implementations, for example, different parts of the eye (or flashes on them), such as the retina and cornea, can be imaged using a pair of coupling optical elements 2112a, 2112b, a pair of external coupling optical elements 2120a, 2120b, and one or more cameras. In some implementations, the first combined optical element 2112a has refractive power or is associated with a lens, while the second combined optical element 2112b does not have similar refractive power or similar lens (e.g., it does not have refractive power and an associated lens). In particular, as discussed above, in some implementations, the second combined optical element 2112b may have refractive power and / or an associated lens, however the total refractive power of the second combined optical element 2112b and / or any lens associated with the second combined optical element is less than the refractive power of the first combined optical element and / or any lens associated with the first combined optical element. The refractive power of the first combined optical element 2112a and / or the lens associated therewith may correspond to focal lengths in the range of approximately 15–25 mm, 10–30 mm, 10–40 mm, 10–50 mm, 5–40 mm, or 5–50 mm, or any range between any of the distance values ​​specified herein. Values ​​outside these ranges are also possible.

[0303] As discussed above, such a system 900 may be configured, for example, to image the anterior portion of the eye using a first coupled optical element 2112a and to image the retina using a second coupled optical element 2112b. As discussed above, in some implementations, the eye is illuminated with light having a first polarization and has a polarizer associated with it, the second external coupled optical element 2120b is polarization-selective, i.e., configured to filter out the light of the first polarization so as not to degrade the image of the retina, such as flashes from the cornea. The arrangement shown in Figures 34A and 34B differs from that shown in Figures 32A and 32B. The first and second coupled optical elements 2112a, 2112b and the first and second external coupled optical elements 2120a, 2120b are aligned on the same axis.

[0304] Figure 35 shows first and second coupling optical elements 2112a, 2112b for coupling light into the waveguide 2102, similar to those shown in Figures 33A and 33B, and first and second external coupling optical elements 2120a, 2120b for coupling light out of the waveguide 2102 to the camera. The coupling optical elements 2112a and 2112b have a non-arc-shaped (e.g., straight-rectangular) slit-shaped coupling region that is aligned along a common axis and integrated onto the eyepiece. The eyepiece further includes an image content internal coupling optical element 2104 for receiving light from the image projector, and a light distribution element 2108 for directing light from the image content internal coupling optical element 2104 to an external coupling optical element 2110 for coupling light induced in the waveguide 2102 to the user for viewing the image content. In other implementations, such as those shown in Figures 34A and 34B, the system 900 does not need to be mounted on a waveguide with such components 2104, 2108, and 2110 in order to present images to the user.

[0305] As discussed above, polarization techniques can be used to attenuate or remove light from the anterior surface (e.g., the corneal surface) so as not to affect the image formed by the light collected by the second coupling optical element 2112b. For example, the eye may be illuminated with polarization having a first polarization, and the camera may form an image using light from the second external coupling optical element 2120b using light of a second different polarization. For example, the second external coupling optical element 2112b may be a polarization-selective coupling element that selectively externally couples light of a second polarization different from the first polarization. In addition, or alternatively, a polarizer 2140 that filters out the first polarization (e.g., selectively transmits the second polarization) may be included between the second external coupling optical element 2120b and the camera 920, as shown in Figure 27.

[0306] Such a configuration may be used to reduce undesirable reflections (e.g., flashes) from the cornea or other sources when imaging the retina. Reflections from the cornea would be specular reflections. Therefore, when light of a first polarization is incident on the cornea, the light reflected from the cornea would retain the first polarization. In contrast, the retina is diffusive. When light of a first polarization is incident on the retina, the light reflected from the retina would not simply retain the first polarization. Diffusive reflection is more likely to result in unpolarized light. Therefore, a second polarization, different from the first polarization, would be present in the light reflected from the retina. As a result, an image of the retina would be acquired by forming an image using light coupled outward from a second external coupling optical element 2120b using light of the second polarization, while images of the cornea or flashes would be suppressed. Similarly, by illuminating with the first polarization and imaging with the second different polarization, the retina can be imaged with reduced glare from the cornea.

[0307] Therefore, in various implementations, polarization-specific optical filters or polarization-selective optical elements (e.g., coupled gratings) may be used to reduce undesirable light reflected from the eye 210 (e.g., the cornea). For example, undesirable light, glare, or flash may be reflected from the cornea so as to saturate the image captured by the camera. As discussed above, light reflected from the cornea is specular and can maintain its polarization. In contrast, light reflected from the retina may be more diffusely reflected and may not be as uniformly polarized. Similarly, a combination of polarizers may be used to remove some or most of the undesirable light reflected from the cornea. Initially, polarization can be used to illuminate the user's eye. In some designs, a polarized illumination source (e.g., a light source) may be used. In addition, or alternatively, a first polarizer (e.g., a polarization-specific optical filter or polarization-selective optical coupled element that couples illumination light into an illumination waveguide) may be positioned at the beginning of the optical path of the illumination source to provide the initial polarization of the light to the eye. A second polarizer (e.g., a polarization-specific optical filter or a polarization-selective coupling element) may be positioned in the optical path before the light enters the camera. The second polarizer may be rotated 90° from the first polarizer (e.g., the polarizers may be "crossed"). As a result, the eye will be illuminated with first-polarized light, accompanied by first-polarized light reflected from the cornea. This light will not pass through a cross-polarizer located proximal to the camera (which preferentially passes second-polarized light). However, the light reflected from the retina will contain second polarization. Similarly, diffusely reflected light from the retina will pass through the proximal polarizer 2140 of the camera, allowing the camera to capture an image of the retina. Thus, in such a configuration, undesirable light received from the eye (e.g., the cornea) and incident on the camera can be reduced or eliminated from the image captured using light from the second coupling optical element 2112b. Other configurations are also possible. For example, the polarization-selective coupling element 2112b and / or the polarization-selective external coupling optical element 2120b may be used in addition to, or as a substitute for, a polarizer such as the camera's proximal polarizer 2140.In addition, or alternatively, a polarization source may be used for illumination (e.g., to illuminate the eye). This effect may again be to reduce or remove unwanted light received from the eye (e.g., the cornea) before it enters the imaging device 920.

[0308] As illustrated in Figure 27, in various implementations, such polarizers are not used in the optical path from the first coupled optical element 2112a and the camera or between the first external coupled optical element 2120a and the camera. As a result, images of the cornea and flash can be obtained from the light coupled into the waveguide by the first coupled optical element 2112a and / or the light coupled out of the waveguide from the first external coupled optical element 2120a. As discussed above, the first coupled optical element 2112a may have refractive power or a lens associated therewith, which is specifically used to image the cornea and / or flash. Similarly, a polarization-selective coupled optical element 2112a or a polarization-selective external coupled optical element 2120a that filters out the first polarization of light would not be used as the first coupled optical element 2112a and the external coupled optical element 2120a, respectively. In addition, polarizers 2140, which filter out the first polarized light, will not be used, either between the coupled optical element 2112a and the camera 920 or between the external coupled optical element 2112a and the camera 920.

[0309] In the aforementioned specification, the present invention has been described with reference to its specific embodiments. However, it will become apparent that various modifications and changes can be made therein without departing from the broader spirit and scope of the invention. The specification and drawings should therefore be considered illustrative, not restrictive.

[0310] In fact, each of the systems and methods described herein has several innovative aspects, and it should be understood that none of them alone are involved in or required for the desirable attributes disclosed herein. The various features and processes described above may be used independently of each other or in various combinations. All possible combinations and secondary combinations are intended to fall within the scope of this disclosure.

[0311] Some features described herein in the context of a separate embodiment may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any preferred secondary combination. Furthermore, features described above as acting in a combination, and further exemplified first as such, may in some cases be removed from the exemplified combination, and the exemplified combination may be subject to secondary combinations or variations of secondary combinations. No single feature or group of features is required or essential in any embodiment.

[0312] In particular, conditional statements used herein, such as “can,” “could,” “might,” “may,” “eg,” and equivalents, should be understood to generally convey that one embodiment includes certain features, elements, and / or ste...

Claims

[Claim 1] The invention described herein.

Images