System for providing illumination of eye

Abstract

To provide a system for providing illumination of the eye.SOLUTION: Various implementations explained herein include a display system configured to provide illumination and / or image projection to the eye. Additionally or alternatively, the display system can form an image on the eye and / or environment. A thin transparent layer can be integrated in a head-mounted display device and disposed in front of the eye of a wearer. The thin transparent layer may be configured to output light such that light is directed onto the eye to create reflection therefrom which can be used, for example, for glint-based tracking. The thin transparent layer can be configured to reduce obstructions in the field of the view of the user.SELECTED DRAWING: Figure 38A

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 821,121, filed March 20, 2019, and entitled "SYSTEM FOR PROVIDING ILLUMINATION OF THE EYE," the disclosure of which is incorporated herein by reference in its entirety.

[0002] The present disclosure relates to optical devices, including augmented reality imaging and visualization systems. [Background technology]

[0003] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that appears or may be perceived as real. Virtual reality, or "VR", scenarios typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual inputs, and augmented reality, or "AR", scenarios typically involve the presentation of digital or virtual image information as an extension to the visualization of the real world around the user. Mixed reality, or "MR", scenarios are a type of AR scenario that typically involve virtual objects that are integrated into and responsive to the natural world. For example, MR scenarios may include AR image content that appears blocked by or is perceived to otherwise interact with objects in the real world.

[0004] Referring to FIG. 1, an augmented reality scene 10 is depicted. A user of the AR technology sees a real-world park-like setting 20 featuring people, trees, buildings in the background, and a concrete platform 30. The user also "sees" and perceives "virtual content," such as a robotic figure 40 standing on the real-world platform 30, and a flying cartoon-like avatar character 50 that appears to be an anthropomorphic bumblebee. These elements 50, 40 are "virtual" in that they do not exist in the real world. The human visual perception system is complex, making it difficult to produce AR technology that facilitates a comfortable, natural-feeling, and rich presentation of virtual image elements among other virtual or real-world image elements.

[0005] The systems and methods disclosed herein address various challenges associated with AR or VR technology.

[0006] Polarizing beam splitters may be used in display systems to direct polarized light to a light modulator, which then directs this light to a viewer. Generally, there is a continuing demand to reduce the size of display systems, and as a result, there is also a demand to reduce the size of display system components, including components that utilize polarizing beam splitters. Summary of the Invention [Means for solving the problem]

[0007] Various implementations described herein include a display system configured to provide illumination and / or image projection to the eye. Additionally or alternatively, the display system can image the eye and / or the environment.

[0008] In some embodiments, the head mounted display system is configured to project light into the user's eye and display augmented reality image content within the user's field of view. The head mounted display system can include a frame configured to be supported on the user's head. The display system can also include an image projector configured to project an image into the user's eye and display image content within the user's field of view. The display system can include a camera, at least one waveguide, at least one coupling optical element configured for light to be coupled into and guided within the waveguide, and at least one external coupling element. The at least one external coupling element can be configured to couple light guided within the waveguide out of the waveguide and direct the light to the camera. The camera can be disposed in an optical path relative to the at least one external coupling optical element to receive at least a portion of the light coupled into and guided within the waveguide via the coupling element and coupled out of the waveguide by the external coupling element such that an image can be captured by the camera. The present invention provides, for example, the following: (Item 1) 1. A head mounted display system configured to project light to an eye of a user and display augmented reality image content within a field of view of the user, the head mounted display system comprising: a frame configured to be supported on a head of the user; and an image projector configured to project an image into the user's eye and display image content within the user's field of view; a transparent layer supported on the frame, the transparent layer positioned at a location in front of the user's eye such that when the user wears the head mounted display, the transparent layer transmits light from an environment in front of the user to the user's eye to provide a view of the environment in front of the user; at least one elongated light guide having a first end and a second end, the first end being disposed within the transparent layer, the first end of the elongated light guide having an output for emitting light such that the light is directed to the eye; at least one camera positioned to receive at least a portion of the light reflected from the eye and to capture an image using the light reflected from the user's eye; A head mounted display system comprising: (Item 2) Item 1 . The system of item 1 , wherein the first end of the elongated light guide includes an angled reflector configured to couple light through the output of the elongated light guide. (Item 3) Item 3. The system of item 2, wherein the angled reflector comprises an oblique surface at the first end of the elongated light guide. (Item 4) 4. The system of any of items 2 or 3, wherein the angled reflector comprises a cleaved surface at the first end of the elongated light guide. (Item 5) 5. The system of any of claims 2-4, wherein the angled reflector comprises metallization. (Item 6) 6. The system of any of claims 2-5, wherein the angled reflector comprises an IR reflective coating. (Item 7) 7. The system of any of items 2-6, wherein the elongated light guide extends along a length within the transparent layer and the angular reflector is oriented at an angle of 35° to 55° relative to the length of the elongated light guide. (Item 8) 8. The system of any of items 2-7, wherein the transparent layer has oppositely facing front and back major surfaces and the angular reflector is oriented at an angle of 35° to 55° relative to the front and back major surfaces. (Item 9) 2. The system of claim 1, wherein the transparent layer comprises an index matching material disposed between front and back glass covers. (Item 10) 2. The system of claim 1, wherein the transparent layer comprises a channel configured to receive the elongated light guide. (Item 11) 2. The system of claim 1, wherein the elongated light guide is disposed within a channel in the transparent layer. (Item 12) The system of any of the preceding items, wherein the second end of the elongated light guide has an input for receiving light, and the elongated light guide is configured such that light received by the input of the elongated light guide at the second end is guided within the elongated light guide to the output at the first end of the elongated light guide. (Item 13) 2. The system of claim 1, wherein the at least one elongated light guide comprises an optical fiber having a core and a cladding, the core being between 8 μm and 110 μm in diameter. (Item 14) 2. The system of claim 1, wherein the at least one elongate waveguide comprises an optical fiber having a core and a cladding, the cladding being between 100 μm and 150 μm in diameter. (Item 15) Item 15. The system of item 14, wherein the cladding has a 125 μm diameter. (Item 16) 2. The system of claim 1, wherein the second end of the elongated light guide extends outside the transparent layer such that the input for receiving light is outside the transparent layer. (Item 17) 2. The system of claim 1, wherein the at least one elongated light guide comprises a plurality of elongated light guides. (Item 18) The system of any of the preceding items, wherein the transparent layer has a first and second side, and the at least one elongated light guide comprises a first elongated light guide integrated into the first side of the transparent layer and a second elongated light guide integrated into the second side of the transparent layer. (Item 19) The system of any of the preceding items, wherein the transparent layer has a first and second side, and the at least one elongated light guide comprises a first pair of elongated light guides integrated into the first side of the transparent layer and a second pair of elongated light guides integrated into the second side of the transparent layer. (Item 20) 20. The system of claim 18 or 19, wherein the first and second sides are nasal and temple sides of the transparent layer. (Item 21) 21. The system of any of items 18-20, wherein the first and second sides are upper and lower sides of the transparent layer. (Item 22) 2. The system of claim 1, wherein the output of the elongated light guide creates a point source that illuminates the user's eye. (Item 23) The system of any one of items 17-22, wherein each of the plurality of elongated light guides has an output that creates a point source that illuminates the user's eye. (Item 24) The system of any one of items 17-23, wherein the first and second pairs of elongated light guides each have an output that creates a point source that illuminates the user's eye. (Item 25) The system of any of the preceding claims, further comprising at least one light source optically coupled to the elongated light guide and injecting light therein such that light is output from the first end of the elongated light guide. (Item 26) 26. The system of claim 25, wherein the at least one light source comprises one or more infrared light sources. (Item 27) 27. The system of any of items 25 or 26, wherein the at least one light source comprises one or more light emitting diodes (LEDs) or lasers. (Item 28) The system of any of the preceding items, further comprising an eyepiece disposed on the frame, the eyepiece configured to direct light into the user's eye and display augmented reality image content in the user's field of view, at least a portion of the eyepiece being transparent and positioned in a location in front of the user's eye when the user wears the head-mounted display so as to transmit light from an environment in front of the user to the user's eye and provide a view of the environment in front of the user. (Item 29) 30. The system of claim 28, wherein the eyepiece is configured to receive light from the image projector, direct the light into the user's eye, and display augmented reality image content in the user's field of view. (Item 30) 30. The system of claim 28 or 29, wherein the eyepiece lens comprises at least one waveguide. (Item 31) Item 31. The system of item 30, wherein the image projector is configured to direct light into the at least one waveguide. (Item 32) Item 32. The system of item 30 or 31, wherein the at least one waveguide comprises a stack of the waveguides. (Item 33) Item 33. The system of item 32, wherein different waveguides of the stack of waveguides are configured to output light with different wavefront divergences as if projected from different distances from the user's eye. (Item 34) Item 34. The system of item 32 or 33, wherein different waveguides of the stack of waveguides are configured to output light with different colors. (Item 35) 35. A system according to any of claims 28-34, wherein the transparent layer comprises a layer within the eyepiece. (Item 36) 35. A system according to any of claims 28-34, wherein the transparent layer comprises a protective layer for the eyepiece. (Item 37) at least one waveguide; at least one coupling optical element, the at least one coupling optical element configured such that light from the output of the elongated light guide that is reflected from the eye is coupled into and guided within the waveguide; at least one external coupling element configured to couple light guided within the waveguide out of the waveguide and direct the light to the at least one camera to capture an image of the eye; 2. The system of claim 1, further comprising: (Item 38) 2. The system of claim 1, wherein the system is configured to perform eye tracking based on images of the eye captured by the camera. (Item 39) 2. The system of claim 1, further comprising a diffuser at the output of the elongated light guide at the first end of the elongated light guide. (Item 40) 13. The system of any preceding claim, wherein the angled reflector is polished. (Item 41) 2. The system of claim 1, wherein the at least one light source comprises a UV or visible light source. (Item 42) 2. The system of claim 1, wherein the at least one elongated light guide has a length-to-thickness ratio of at least 10. (Item 43) 2. The system of claim 1, wherein the at least one elongated light guide has a length-to-thickness ratio of at least 20. (Item 44) 2. The system of claim 1, wherein the at least one elongated light guide comprises an optical fiber. (Item 45) 2. The system of claim 1, wherein the at least one elongated light guide comprises an optical rod comprising an optically transmissive material. (Item 46) 2. The system of claim 1, wherein the at least one elongated light guide comprises a plurality of optical fibers. (Item 47) 2. The system of claim 1, wherein the at least one elongated light guide comprises a plurality of optical rods. (Item 48) 1. A head mounted display system configured to project light to an eye of a user and display augmented reality image content within a field of view of the user, the head mounted display system comprising: a frame configured to be supported on a head of the user; and an image projector configured to project an image into the user's eye and display image content within the user's field of view; a transparent layer supported on the frame, the transparent layer positioned at a location in front of the user's eye such that when the user wears the head mounted display, the transparent layer transmits light from an environment in front of the user to the user's eye to provide a view of the environment in front of the user, the transparent layer having front and back major surfaces surrounded by a plurality of edges, the transparent layer further comprising at least one sloped surface, the sloped surface being angled such that light guided within the transparent layer is reflected off the sloped surface and directed out of the transparent layer through the back major surface and towards the eye; at least one camera positioned to receive at least a portion of light reflected from the user's eye and capture an image using the light reflected from the user's eye; A head mounted display system comprising: (Item 49) 13. The system of claim 1, wherein the sloped surface comprises metallization. (Item 50) 2. The system of claim 1, wherein the inclined surface comprises an IR reflective coating. (Item 51) 2. The system of claim 1, wherein the inclined surface is angled at between 35° and 55° relative to the front and / or back major surfaces of the transparent layer. (Item 52) The system of any of the preceding items, further comprising at least one light source positioned relative to the at least one transparent layer such that light from the at least one light source is injected into the at least one transparent layer, incident on the inclined surface, and reflected therefrom toward the user's eye. (Item 53) The system of any of the preceding items, further comprising at least one light source positioned relative to at least one edge of the transparent layer such that light from the at least one light source is injected into the at least one transparent layer, impinges on the inclined surface, and is reflected therefrom toward the user's eye. (Item 54) 54. The system of claim 52 or 53, wherein the at least one light source comprises a plurality of spaced apart light sources. (Item 55) 55. The system of any of claims 52-54, wherein the light source comprises one or more infrared light sources. (Item 56) 56. The system of claim 54 or 55, wherein the plurality of light sources comprises three light sources. (Item 57) 57. The system of any of items 54-56, wherein the plurality of light sources comprises four light sources. (Item 58) 58. A system according to any of items 52-57, wherein the light source creates a point source that illuminates the user's eye. (Item 59) 59. The system of any of items 54-58, wherein the multiple light sources create multiple point sources that illuminate the user's eyes. (Item 60) 2. The system of claim 1, wherein the four light sources create four point sources that illuminate the user's eyes. (Item 61) 61. The system of any of items 52-60, wherein the light source comprises one or more light emitting diodes (LEDs) or lasers. (Item 62) 62. The system of any of items 52-61, wherein the light from the light source is guided within the transparent layer by total internal reflection from the front and back surfaces of the transparent layer. (Item 63) 2. The system of claim 1, wherein the transparent layer comprises plastic. (Item 64) 13. The system of claim 1, wherein the inclined surface is a plane. (Item 65) The system of any of the preceding items, further comprising an eyepiece disposed on the frame, the eyepiece configured to direct light into the user's eye and display augmented reality image content in the user's field of view, at least a portion of the eyepiece being transparent and positioned in a location in front of the user's eye when the user wears the head-mounted display so as to transmit light from an environment in front of the user to the user's eye and provide a view of the environment in front of the user. (Item 66) Item 66. The system of item 65, wherein the eyepiece is configured to receive light from the image projector, direct the light into the user's eye, and display augmented reality image content in the user's field of view. (Item 67) 67. The system of claim 65 or 66, wherein the eyepiece lens comprises at least one waveguide. (Item 68) Item 68. The system of item 67, wherein the image projector is configured to direct light into the at least one waveguide. (Item 69) Item 69. The system of item 67 or 68, wherein the at least one waveguide comprises a stack of the waveguides. (Item 70) 70. The system of claim 69, wherein different waveguides of the stack of waveguides are configured to output light with different wavefront divergences as if projected from different distances from the user's eye. (Item 71) 71. The system of claim 69 or 70, wherein different waveguides of the stack of waveguides are configured to output light with different colors. (Item 72) 72. A system according to any of claims 65-71, wherein the transparent layer comprises a layer within the eyepiece. (Item 73) 72. A system according to any of claims 65-71, wherein the transparent layer comprises a protective layer for the eyepiece. (Item 74) at least one waveguide; at least one coupling optical element, the at least one coupling optical element configured such that light output from the transparent layer that is reflected from the eye is coupled into and guided within the waveguide; at least one external coupling element configured to couple light guided within the waveguide out of the waveguide and direct the light to the camera to capture an image of the eye; 2. The system of claim 1, further comprising: (Item 75) 2. The system of claim 1, wherein the system is configured to perform eye tracking based on images of the eye captured by the camera. (Item 76) 2. The system of claim 1, wherein the at least one light source comprises a UV or visible light source. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates a user's view of an augmented reality (AR) device.

[0010] [Diagram 2] FIG. 2 illustrates an example of a wearable display system.

[0011] [Diagram 3] FIG. 3 illustrates a conventional display system for simulating a three-dimensional image for a user.

[0012] [Figure 4] FIG. 4 illustrates aspects of an approach for simulating a three-dimensional image using multiple depth planes.

[0013] [Diagram 5] 5A-5C illustrate the relationship between the radius of curvature and the radius of focus.

[0014] [Figure 6] FIG. 6 illustrates an embodiment of a waveguide stack for outputting image information to a user.

[0015] [Figure 7] FIG. 7 illustrates an example of an output beam output by a waveguide.

[0016] [Figure 8] FIG. 8 illustrates an example of a stacked waveguide assembly where each depth plane contains an image formed using multiple different primary colors.

[0017] [Figure 9A]9A illustrates a cross-sectional side view of an example of a set of stacked waveguides, each including an internal coupling optical element. As discussed herein, the stack of waveguides may include an eyepiece.

[0018] [Figure 9B] FIG. 9B illustrates a perspective view of the multiple stacked waveguide embodiment of FIG. 9A.

[0019] [Figure 9C] FIG. 9C illustrates a top-down plan view of the multiple stacked waveguide embodiment of FIGS. 9A and 9B.

[0020] [Figure 10] FIG. 10 diagrammatically illustrates a cross-sectional side view of an exemplary imaging system comprising an eyepiece, an image projector, a light source for illuminating the eye, and a camera for capturing an image of the eye.

[0021] [Figure 11A] FIG. 11A diagrammatically illustrates how both a light source for illuminating the eye and an image projector for projecting an image into the eye emit light toward an internal coupling optic on the waveguide of the eyepiece.

[0022] [Figure 11B] FIG. 11B illustrates diagrammatically how projected light from a light source and an image projector is coupled into a waveguide.

[0023] [Figure 11C] FIG. 11C illustrates diagrammatically how the incoupled light can propagate through the waveguide by total internal reflection.

[0024] [Figure 11D] FIG. 11D illustrates diagrammatically how light from the light source and image projector is coupled out through the eyepiece.

[0025] [Figure 11E] 11E diagrammatically illustrates a waveguide and a coupling optical element configured to propagate the intercoupled light at least along all dimensions of the coupling optical element (e.g., along the x-direction). Light entering the eye is shown from an extended source (e.g., imaging light that would capture a region of the retina).

[0026] [Figure 12A] FIG. 12A is a cross-sectional view showing diagrammatically light reflected from the retina exiting the eye and impinging on the eyepiece.

[0027] [Figure 12B] FIG. 12B diagrammatically illustrates an example light being coupled into a waveguide of the eyepiece.

[0028] [Figure 12C] FIG. 12C illustrates diagrammatically how collimated, incoupled light from the eye propagates through a waveguide towards an imaging device.

[0029] [Figure 12D] FIG. 12D illustrates diagrammatically how incoupled light from the eye propagates to one or more outcoupling optical elements.

[0030] [Figure 12E] FIG. 12E diagrammatically illustrates how light from the eye is coupled out of the waveguide by an outcoupling optical element and directed to a camera so that an image of the eye (e.g., the retina) can be captured by the camera.

[0031] [Figure 13A] FIG. 13A diagrammatically illustrates how an imaging system can image various portions of the eye, e.g., the retina, which can enable eye orientation to be determined and eye position to be tracked.

[0032] [Figure 13B]13B illustrates a pattern of sequentially displayed fixation targets used to orient the eye in different directions while the retina is imaged. The resulting images correspond to different portions of the retina. For example, as the eye is oriented in different directions and views differently located fixation targets on the display, the images captured by the camera include different portions of the retina. These images can be assembled to form a larger map or composite image of the retina.

[0033] [Figure 14A] 14A diagrammatically illustrates a cross-sectional view of an imaging system comprising an eyepiece and a camera for collecting light from an environment in front of the eyepiece. The light from the environment is shown as reflected or emitted from one or more physical objects in the environment. The collection of light from objects in the environment in front of the eyepiece can allow an image of the environment to be captured.

[0034] [Figure 14B] FIG. 14B illustrates diagrammatically how light from the environment is coupled into the waveguide of the eyepiece by a coupling optical element.

[0035] [Figure 14C] FIG. 14C diagrammatically illustrates an imaging system for collecting light from the environment using a refractive optical element such as a refractive optical element (e.g., a lens such as a wide-angle lens) in front of the eyepiece.

[0036] [Figure 15A] 15A diagrammatically illustrates an exemplary imaging system comprising a polarization-selective in-coupling optical element for receiving light from an illumination source and coupling the light into a waveguide in an eyepiece. The eyepiece further includes a polarization-selective light-coupling element for coupling the light out of the waveguide. A polarizer may be used to polarize the light from the illumination source, and a half-wave retarder may be used to rotate the orientation of the linearly polarized light as it is redirected into the waveguide by the polarization-selective in-coupling optical element.

[0037] [Figure 15B] FIG. 15B diagrammatically illustrates how light from the eye (e.g., from the retina illuminated with infrared light from an illumination source) is coupled back into the waveguide and directed to a camera for image capture.

[0038] [Figure 16] FIG. 16 diagrammatically illustrates an imaging system configured for imaging an anterior portion of the eye (e.g., the cornea). The imaging system comprises an eyepiece as described above. The imaging system further includes a positive lens for collimating light collected from the anterior portion of the eye for coupling into a waveguide via an optical coupling element and propagating to a camera for image capture. The system further comprises a negative lens for offsetting the positive refractive power introduced by the positive lens and preventing inversion of the image of the environment in front of the eyepiece that would otherwise be caused by the positive lens.

[0039] [Figure 17] 17 diagrammatically illustrates another exemplary imaging system configured for imaging an anterior portion of the eye (e.g., the cornea). The imaging system comprises a curved wavelength-selective reflector that collimates light from the anterior portion of the eye for coupling into a waveguide via an optical coupling element and propagating to a camera for image capture. The wavelength-selective reflector may operate reflectively for infrared light reflected from the eye and transmissively for visible light from the environment in front of the user.

[0040] [Figure 18] FIG. 18 also diagrammatically illustrates an exemplary imaging system including a curved wavelength-selective reflector that collimates light from the front portion of the eye for coupling into a waveguide via an optical coupling element and propagation to a camera for image capture. Polarization selectivity may be employed to help control the path of light reflected from the eye. Eye illumination is provided via the waveguide instead of multiple light sources between the waveguide and the eye as shown in FIG.

[0041] [Figure 19] FIG. 19 illustrates diagrammatically an imaging system including a shutter to assist in the procedure for removing noise.

[0042] [Figure 20A] 20A-20E diagrammatically illustrate an alternative procedure for removing noise using wavelength modulation in conjunction with a curved wavelength-selective reflector. [Figure 20B] 20A-20E diagrammatically illustrate an alternative procedure for removing noise using wavelength modulation in conjunction with a curved wavelength-selective reflector. [Figure 20C] 20A-20E diagrammatically illustrate an alternative procedure for removing noise using wavelength modulation in conjunction with a curved wavelength-selective reflector. [Figure 20D] 20A-20E diagrammatically illustrate an alternative procedure for removing noise using wavelength modulation in conjunction with a curved wavelength-selective reflector. [Figure 20E] 20A-20E diagrammatically illustrate an alternative procedure for removing noise using wavelength modulation in conjunction with a curved wavelength-selective reflector.

[0043] [Figure 21] FIG. 21 shows an exemplary eyepiece that can be used to project light into a user's eye and provide image content thereto while simultaneously receiving image data of the user's eye or the environment in front of the user.

[0044] [Figure 22] FIG. 22 illustrates a cross-sectional side view of an embodiment of a cholesteric liquid crystal grating (CLCG) having a plurality of uniform chiral structures.

[0045] [Diagram 23]FIG. 23 illustrates an example of an imaging system comprising a forward-facing camera configured to image a wearer's eye using a cholesteric liquid crystal (CLC) off-axis mirror.

[0046] [Figure 24] 24 illustrates another exemplary eyepiece that may be used to receive image data of a user's eye or the environment in front of the user while simultaneously projecting light into the user's eye and providing image content thereto. In this example, the coupling optic configured to receive light from the user or the environment in front of the user is laterally displaced from the image content out-coupling optic (e.g., an exit pupil expander).

[0047] [Diagram 25] Fig. 25 shows another example eyepiece, similar to that shown in Fig. 24, in which a coupling optical element configured to receive light from a user or an environment in front of the user is spaced laterally from an image content out-coupling optical element (e.g., an exit pupil expander). However, in the implementation shown in Fig. 25, no space laterally separates the coupling optical element 2111 from the out-coupling optical element 2110.

[0048] [Figure 26] FIG. 26 shows an exemplary eyepiece including first and second coupling optical elements configured to couple light received from a user's eye or the environment into a waveguide to be guided therein, and first and second out-coupling optical elements configured to couple light guided within the waveguide out of the waveguide to one or more cameras.

[0049] [Figure 27] Figure 27 is a cross section through the waveguide, collection optics, and outcoupling optics shown in Figure 26, showing a polarizer in the optical path between one of the outcoupling optics and the camera. The polarizer may be used to remove unwanted glint reflections from the cornea when acquiring an image of the retina.

[0050] [Figure 28] 28A and 28B are front and perspective views of a coupling optical element for coupling light into the waveguide and an out-coupling optical element for coupling light out of the waveguide to a camera, the coupling optical element having a pinhole coupling area. The out-coupling optical element is similarly sized and shaped in this implementation.

[0051] [Figure 29] 29A and 29B are front and perspective views of a coupling optical element for coupling light into a waveguide and an out-coupling optical element for coupling light out of the waveguide to a camera, the coupling optical element having an arc-slit shaped coupling area, and the out-coupling optical element having a pinhole size coupling area.

[0052] [Diagram 30] 30A and 30B are front and perspective views of a coupling optical element for coupling light into a waveguide and an out-coupling optical element for coupling light out of the waveguide to a camera, the coupling optical element having a non-arcuate (e.g., rectilinear rectangular) slit-shaped coupling area, the out-coupling optical element having a pinhole size coupling area.

[0053] [Diagram 31] 31 is a front view of an eyepiece including a coupling optical element for coupling light into a waveguide and an out-coupling optical element for coupling light out of the waveguide to a camera, the coupling optical element having a non-arcuate (e.g., rectilinear rectangular) slit-shaped coupling area. The eyepiece further includes an image content in-coupling optical element for receiving light from an image projector and a light distribution element for directing light from the in-coupling optical element to an out-coupling optical element for coupling light guided in the waveguide to a user for viewing the image content.

[0054] [Diagram 32]32A and 32B are front and perspective views of a pair of coupling optical elements for coupling light into a waveguide and a pair of out-coupling optical elements for coupling light out of the waveguide to a camera, the coupling optical elements having a non-arcuate (e.g., rectilinear rectangular) slit-shaped coupling area. Such a configuration may be useful for imaging different parts of the eye, such as the retina and cornea (or phosphenes thereon).

[0055] [Diagram 33] 33 is a front view of an eyepiece including a pair of coupling optical elements for coupling light into a waveguide and a pair of out-coupling optical elements for coupling light out of the waveguide to a camera, the coupling optical elements having a non-arcuate (e.g., rectilinear rectangular) slit-shaped coupling area. The eyepiece further includes an image content in-coupling optical element for receiving light from an image projector, and a light distribution element for directing light from the in-coupling optical element to the out-coupling optical element for coupling light guided in the waveguide to a user for viewing the image content.

[0056] [Diagram 34] 34A and 34B are front and perspective views of a pair of coupling optical elements for coupling light into a waveguide and a pair of out-coupling optical elements for coupling light out of the waveguide to a camera, where the coupling optical elements have a non-arcuate (e.g., rectilinear rectangular) slit-shaped coupling area. The arrangement is different from that shown in FIGS. 32A and 32B.

[0057] [Diagram 35] Figure 35 is a front view of an eyepiece similar to that shown in Figures 30A and 30B including a pair of coupling optical elements for coupling light into a waveguide and a pair of out-coupling optical elements for coupling light out of the waveguide to a camera. The eyepiece further includes an image content in-coupling optical element for receiving light from an image projector, and a light distribution element for directing light from the in-coupling optical element to the out-coupling optical element for coupling light guided in the waveguide to a user for viewing the image content.

[0058] [Figure 36A] 36A and 36B are side views of a transparent layer that can be integrated into an eyepiece to direct light to the eye and used, for example, for phosphene-based eye tracking. [Figure 36B] 36A and 36B are side views of a transparent layer that can be integrated into an eyepiece to direct light to the eye and used, for example, for phosphene-based eye tracking.

[0059] [Figure 37A] 37A and 37B are side views showing example transparent layers positioned at different locations within the eyepiece for directing light to a user's eye. [Figure 37B] 37A and 37B are side views showing example transparent layers positioned at different locations within the eyepiece for directing light to a user's eye.

[0060] [Figure 38A] 38A and 38B are front and side views of an exemplary transparent layer including multiple optical fibers therein that can be configured to direct light toward the eye. [Figure 38B] 38A and 38B are front and side views of an exemplary transparent layer including multiple optical fibers therein that can be configured to direct light toward the eye.

[0061] [Figure 39] FIG. 39 is a perspective view of an exemplary transparent layer including a plurality of optical rods therein that can be configured to direct light toward the eye.

[0062] [Figure 40A] 40A-40C are side, perspective, and top views, respectively, of an exemplary transparent layer that includes a pair of angled surfaces that reflect light guided through the transparent layer toward the eye. [Figure 40B]40A-40C are side, perspective, and top views, respectively, of an exemplary transparent layer that includes a pair of angled surfaces that reflect light guided through the transparent layer toward the eye. [Figure 40C] 40A-40C are side, perspective, and top views, respectively, of an exemplary transparent layer that includes a pair of angled surfaces that reflect light guided through the transparent layer toward the eye.

[0063] [Figure 40D] 40D-40F are top views of additional example designs of a transparent layer including a plurality of angled surfaces configured to reflect light guided through the transparent layer toward the eye. [Figure 40E] 40D-40F are top views of additional example designs of a transparent layer including a plurality of angled surfaces configured to reflect light guided through the transparent layer toward the eye. [Diagram 40F] 40D-40F are top views of additional example designs of a transparent layer including a plurality of angled surfaces configured to reflect light guided through the transparent layer toward the eye.

[0064] The drawings are provided to illustrate example embodiments and are not intended to limit the scope of the present disclosure.Like reference numbers refer to like parts throughout. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0065] Reference is now made to the drawings, in which like reference numbers refer to like parts throughout.

[0066] FIG. 2 illustrates an example of a wearable display system 60. The display system 60 includes a display 70 and various mechanical and electronic modules and systems to support the functionality of the display 70. The display 70 may be coupled to a frame 80 that is wearable by a display system user or viewer 90 and configured to position the display 70 in front of the eye of the user 90. The display 70 may be considered eyewear in some embodiments. In some embodiments, a speaker 100 is coupled to the frame 80 and configured to be positioned adjacent to the ear canal of the user 90 (in some embodiments, another speaker, not shown, may also be optionally positioned adjacent the user's other ear canal to provide stereo / shapeable sound control). The display system may also include one or more microphones 110 or other devices to detect sound. In some embodiments, the microphones may be configured to allow the user to provide input or commands (e.g., voice menu command selections, natural language questions, etc.) to the system 60 and / or enable audio communication with other persons (e.g., other users of similar display systems). The microphone may further be configured as an ambient sensor to collect audio data (e.g., sounds from the user and / or the environment). In some embodiments, the display system may also include an ambient sensor 120a that is separate from the frame 80 and may be mounted on the body of the user 90 (e.g., on the head, torso, limbs, etc. of the user 90). The ambient sensor 120a may, in some embodiments, be configured to obtain data that characterizes a physiological state of the user 90. For example, the sensor 120a may be an electrode.

[0067] 2, the display 70 is operably coupled by a communication link 130, such as by wired leads or wireless connectivity, to a local data processing module 140, which may be mounted in a variety of configurations, such as fixedly attached to the frame 80, fixedly attached to a helmet or hat worn by the user, built into headphones, or otherwise removably attached to the user 90 (e.g., in a backpack configuration, in a belt-connected configuration). Similarly, the sensor 120a may be operably coupled to the local data processing module 140 by a communication link 120b, such as by wired leads or wireless connectivity. The local processing and data module 140 may comprise a hardware processor and digital memory, such as non-volatile memory (e.g., flash memory or hard disk drive), both of which may be utilized to aid in processing, caching, and storing data. The data includes a) data captured from sensors (e.g., which may be operatively coupled to the frame 80 or otherwise attached to the user 90), such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, gyroscopes, and / or other sensors disclosed herein, and / or b) data obtained and / or processed using the remote processing module 150 and / or remote data repository 160 (including data related to virtual content), possibly for passing to the display 70 after processing or retrieval. The local processing and data module 140 may be operatively coupled to the remote processing module 150 and the remote data repository 160 by communication links 170, 180, such as via wired or wireless communication links, such that these remote modules 150, 160 are operatively coupled to each other and available as resources to the local processing and data module 140.In some embodiments, local processing and data module 140 may include one or more of an image capture device, a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a wireless device, and / or a gyroscope. In some other embodiments, one or more of these sensors may be mounted to frame 80 or may be a freestanding structure that communicates with local processing and data module 140 by a wired or wireless communication path.

[0068] 2, in some embodiments, remote processing module 150 may comprise one or more processors configured to analyze and process data and / or image information. In some embodiments, remote data repository 160 may comprise a digital data storage facility, which may be available through the Internet or other networking configurations in a "cloud" resource configuration. In some embodiments, remote data repository 160 may include one or more remote servers that provide information to local processing and data module 140 and / or remote processing module 150, e.g., information for generating augmented reality content. In some embodiments, all data is stored and all calculations are performed in the local processing and data module, allowing for fully autonomous use from the remote modules.

[0069] Referring now to FIG. 3, the perception of an image as "three-dimensional" or "3-D" may be achieved by providing a slightly different presentation of the image to each eye of a viewer. FIG. 3 illustrates a conventional display system for simulating a three-dimensional image for a user. Two distinct images 190, 200 are output to the user, one for each eye 210, 220. The images 190, 200 are spaced apart from the eyes 210, 220 by a distance 230 along an optical or z-axis parallel to the viewer's line of sight. The images 190, 200 are flat, and the eyes 210, 220 may focus on the images by assuming a single accommodated state. Such a 3-D display system relies on the human visual system to combine the images 190, 200 and provide a perception of depth and / or scale of the combined image.

[0070] However, it should be appreciated that the human visual system is more complex and more difficult to provide a realistic perception of depth. For example, many viewers of conventional "3-D" display systems may find such systems uncomfortable or may not perceive any sensation of depth at all. Without being limited by theory, it is believed that a viewer of an object may perceive the object as "three-dimensional" due to a combination of vergence-divergence and accommodation. The vergence-divergence of the two eyes relative to one another (i.e., the rotational movement of the pupils toward or away from one another to converge the gaze of the eyes and fixate on an object) is closely linked to the focusing (or "accommodation") of the eye's lens and pupil. Under normal conditions, changing the focus of the eye's lens or accommodating the eye to change focus from one object to another at a different distance will automatically produce a matching change in vergence-divergence to the same distance, a relationship known as the "accommodation-vergence-divergence reflex" and pupil dilation or constriction. Similarly, changes in vergence-divergence will induce corresponding changes in accommodation of the lens shape and pupil size under normal conditions. As described herein, many stereoscopic or "3-D" display systems display a scene using a slightly different presentation (and therefore a slightly different image) to each eye so that a three-dimensional perspective is perceived by the human visual system. However, such systems are uncomfortable for many viewers because, among other things, they simply provide a different presentation of the scene, but work against the "accommodation-vergence-divergence reflex" when the eyes view the entire image information in a single accommodated state. Display systems that provide a better match between accommodation and vergence-divergence may produce a more realistic and comfortable simulation of a three-dimensional image.

[0071] FIG. 4 illustrates aspects of an approach for simulating a three-dimensional image using multiple depth planes. With reference to FIG. 4, objects at various distances from the eyes 210, 220 on the z-axis are accommodated by the eyes 210, 220 such that the objects are in focus. The eyes 210, 220 assume a particular accommodated state and focus objects at different distances along the z-axis. As a result, a particular accommodated state may be said to be associated with a particular one of the depth planes 240 having an associated focal length such that an object or part of an object at a particular depth plane is in focus when the eye is in an accommodated state relative to that depth plane. In some embodiments, a three-dimensional image may be simulated by providing a different presentation of an image for each eye 210, 220 and by providing a different presentation of an image corresponding to each of the depth planes. Although shown as separate for clarity of illustration, it should be understood that the fields of view of the eyes 210, 220 may overlap, for example, as the distance along the z-axis increases. Additionally, while shown as flat for ease of illustration, it should be understood that the contour of the depth plane may be curved in physical space such that all features within the depth plane are in focus with the eye in a particular accommodated state.

[0072] The distance between an object and the eye 210 or 220 may also change the amount of divergence of light from that object as viewed by that eye. Figures 5A-5C illustrate the relationship between distance and divergence of light rays. The distance between the object and the eye 210 is represented in the order of decreasing distances R1, R2, and R3. As shown in Figures 5A-5C, the light rays become more divergent as the distance to the object decreases. As the distance increases, the light rays become more collimated. In other words, the light field generated by a point (an object or part of an object) may be said to have a spherical wavefront curvature that is a function of the distance the point is away from the user's eye. The curvature increases with decreasing distance between the object and the eye 210. As a result, at different depth planes, the divergence of the light rays is also different, and the divergence increases with decreasing distance between the depth plane and the viewer's eye 210. Although only a monocular 210 is illustrated in FIGS. 5A-5C and other figures herein for clarity of illustration, it should be understood that the discussion regarding eye 210 may apply to both eyes 210 and 220 of a viewer.

[0073] Without being limited by theory, it is believed that the human eye is typically capable of interpreting a finite number of depth planes to provide depth perception. As a result, a highly realistic simulation of perceived depth may be achieved by providing the eye with different presentations of images corresponding to each of these limited number of depth planes. The different presentations may be focused separately by the viewer's eye, thereby serving to provide depth cues to the user based on the ocular accommodation required to focus on different image features for a scene located on different depth planes and / or based on observing different image features on different depth planes that are out of focus.

[0074] FIG. 6 illustrates an example of a waveguide stack for outputting image information to a user. The display system 250 includes a stack of waveguides or a stacked waveguide assembly 260 that can be utilized to provide a three-dimensional perception to the eye / brain using multiple waveguides 270, 280, 290, 300, 310. In some embodiments, the display system 250 is the system 60 of FIG. 2, and FIG. 6 illustrates several portions of the system 60 in greater detail. For example, the waveguide assembly 260 may be part of the display 70 of FIG. 2. It should be understood that the display system 250 may be considered a light field display in some embodiments. In addition, the waveguide assembly 260 may also be referred to as an eyepiece.

[0075] Continuing with reference to FIG. 6, the waveguide assembly 260 may also include a number of features 320, 330, 340, 350 between the waveguides. In some embodiments, the features 320, 330, 340, 350 may be one or more lenses. The waveguides 270, 280, 290, 300, 310 and / or the number of lenses 320, 330, 340, 350 may be configured to transmit image information to the eye with various levels of wavefront curvature or light beam divergence. Each waveguide level may be associated with a particular depth plane and may be configured to output image information corresponding to that depth plane. The image input devices 360, 370, 380, 390, 400 may act as light sources for the waveguides and may be utilized to input image information into the waveguides 270, 280, 290, 300, 310, each of which may be configured to distribute incident light across each individual waveguide for output towards the eye 210, as described herein. Light exits output surfaces 410, 420, 430, 440, 450 of the image input devices 360, 370, 380, 390, 400 and is input into corresponding input surfaces 460, 470, 480, 490, 500 of the waveguides 270, 280, 290, 300, 310. In some embodiments, each of the input surfaces 460, 470, 480, 490, 500 may be an edge of the corresponding waveguide or may be a portion of a major surface of the corresponding waveguide (i.e., one of the waveguide surfaces that directly faces the world 510 or the viewer's eye 210). In some embodiments, a single beam of light (e.g., a collimated beam) may be launched into each waveguide, outputting a total field of cloned collimated beams that are directed toward the eye 210 at a particular angle (and divergence) corresponding to the depth plane associated with the particular waveguide. In some embodiments, a single one of the image launch devices 360, 370, 380, 390, 400 may be associated with and launch light into multiple (e.g., three) waveguides 270, 280, 290, 300, 310.

[0076] In some embodiments, each of the image input devices 360, 370, 380, 390, 400 is a discrete display that generates image information for input into a corresponding waveguide 270, 280, 290, 300, 310, respectively. In some other embodiments, the image input devices 360, 370, 380, 390, 400 are the output of a single multiplexed display that may send image information to each of the image input devices 360, 370, 380, 390, 400, for example, via one or more optical conduits (such as fiber optic cables). It should be understood that the image information provided by the image input devices 360, 370, 380, 390, 400 may include light of different wavelengths or colors (e.g., different primary colors as discussed herein).

[0077] In some embodiments, the light injected into the waveguides 270, 280, 290, 300, 310 is provided by a light projector system 520 comprising a light module 540, which may include light emitters such as light emitting diodes (LEDs). Light from the light module 540 may be directed through a beam splitter 550 to and modified by a light modulator 530, e.g., a spatial light modulator. The light modulator 530 may be configured to change the perceived intensity of the light injected into the waveguides 270, 280, 290, 300, 310. Examples of spatial light modulators include liquid crystal displays (LCDs), including liquid crystal on silicon (LCOS) displays. It should be understood that image input devices 360, 370, 380, 390, 400 are illustrated diagrammatically, and in some embodiments these image input devices may represent different light paths and locations within a common projection system that are configured to output light into associated ones of waveguides 270, 280, 290, 300, 310.

[0078] In some embodiments, the display system 250 may be a scanning fiber display comprising one or more scanning fibers configured to project light in various patterns (e.g., raster scan, spiral scan, Lissajous pattern, etc.) into one or more waveguides 270, 280, 290, 300, 310 and ultimately to the viewer's eye 210. In some embodiments, the illustrated image injection devices 360, 370, 380, 390, 400 may diagrammatically represent a single scanning fiber or a bundle of scanning fibers configured to inject light into one or more waveguides 270, 280, 290, 300, 310. In some other embodiments, the illustrated image injection devices 360, 370, 380, 390, 400 may diagrammatically represent multiple scanning fibers or multiple bundles of scanning fibers, each configured to inject light into an associated one of the waveguides 270, 280, 290, 300, 310. It is understood that one or more optical fibers may be configured to transmit light from the optical module 540 to one or more of the waveguides 270, 280, 290, 300, 310. It is understood that one or more intervening optical structures may be provided between the scanning fiber or fibers and one or more of the waveguides 270, 280, 290, 300, 310, for example, to redirect light exiting the scanning fiber into one or more of the waveguides 270, 280, 290, 300, 310.

[0079] The controller 560 controls the operation of one or more of the stacked waveguide assemblies 260, including the operation of the image input devices 360, 370, 380, 390, 400, the light source 540, and the light modulator 530. In some embodiments, the controller 560 is part of the local data processing module 140. The controller 560 includes programming (e.g., instructions in a non-transient medium) that coordinates the timing and provision of image information to the waveguides 270, 280, 290, 300, 310, for example, according to any of the various schemes disclosed herein. In some embodiments, the controller may be a single integrated device or a distributed system connected by wired or wireless communication channels. The controller 560 may be part of the processing module 140 or 150 (FIG. 2) in some embodiments.

[0080] Continuing with reference to FIG. 6, the waveguides 270, 280, 290, 300, 310 may be configured to propagate light within each individual waveguide by total internal reflection (TIR). Each of the waveguides 270, 280, 290, 300, 310 may be planar or have another shape (e.g., curved) with major top and bottom surfaces and edges extending between the major top and bottom surfaces. In the illustrated configuration, each of the waveguides 270, 280, 290, 300, 310 may include an out-coupling optical element 570, 580, 590, 600, 610 configured to extract light from the waveguide by redirecting light propagating within each individual waveguide out of the waveguide and outputting image information to the eye 210. The extracted light may also be referred to as out-coupled light, and the out-coupling optical element may also be referred to as a light extraction optical element. The extracted beam of light may be output by the waveguide at a location where the light propagating within the waveguide strikes the light extraction optical element. The outcoupling optical element 570, 580, 590, 600, 610 may be a grating, for example, including diffractive optical features as further discussed herein. Although shown disposed on the bottom major surface of the waveguide 270, 280, 290, 300, 310 for ease of explanation and clarity of drawing, in some embodiments the outcoupling optical element 570, 580, 590, 600, 610 may be disposed on the top and / or bottom major surfaces and / or directly within the volume of the waveguide 270, 280, 290, 300, 310, as further discussed herein. In some embodiments, the outcoupling optical elements 570, 580, 590, 600, 610 may be formed within a layer of material that is attached to a transparent substrate and forms the waveguide 270, 280, 290, 300, 310. In some other embodiments, the waveguide 270, 280, 290, 300, 310 may be a monolithic piece of material and the outcoupling optical elements 570, 580, 590, 600, 610 may be formed on and / or within that piece of material.

[0081] With continued reference to FIG. 6, as discussed herein, each waveguide 270, 280, 290, 300, 310 is configured to output light and form an image corresponding to a particular depth plane. For example, the waveguide 270 closest to the eye may be configured to deliver collimated light (injected into such waveguide 270) to the eye 210. The collimated light may represent an optical infinity focal plane. The next upper waveguide 280 may be configured to send collimated light that passes through a first lens 350 (e.g., a negative lens) before it can reach the eye 210. Such a first lens 350 may be configured to generate a slight convex wavefront curvature such that the eye / brain interprets the light originating from the next upper waveguide 280 as originating from a first focal plane closer inward from optical infinity toward the eye 210. Similarly, the third upper waveguide 290 passes its output light through both the first 350 and second lenses 340 before reaching the eye 210. The combined refractive powers of the first 350 and second lenses 340 may be configured to produce another incremental amount of wavefront curvature such that the eye / brain interprets the light originating from the third waveguide 290 as originating from a second focal plane that is closer inward from optical infinity towards the person than was the light from the next upper waveguide 280.

[0082] The other waveguide layers 300, 310 and lenses 330, 320 are similarly configured, with the highest waveguide 310 in the stack sending its output through all of the lenses between it and the eye for an aggregate focal force that represents the focal plane closest to the person. To compensate the stack of lenses 320, 330, 340, 350 when viewing / interpreting light originating from the world 510 on the other side of the stacked waveguide assembly 260, a compensating lens layer 620 may be placed on top of the stack to compensate for the aggregate force of the lens stacks 320, 330, 340, 350 below. Such a configuration provides as many perceived focal planes as there are waveguide / lens pairs available. Both the outcoupling optical elements of the waveguides and the focusing sides of the lenses may be static (i.e., not dynamic or electroactive). In some alternative embodiments, one or both may be dynamic using electroactive features.

[0083] In some embodiments, two or more of the waveguides 270, 280, 290, 300, 310 may have the same associated depth plane. For example, multiple waveguides 270, 280, 290, 300, 310 may be configured to output images set at the same depth plane, or multiple subsets of the waveguides 270, 280, 290, 300, 310 may be configured to output images set at the same depth planes, with one set per depth plane. This may provide an advantage for forming tiled images to provide an extended field of view at those depth planes.

[0084] Continuing with reference to FIG. 6, the outcoupling optical elements 570, 580, 590, 600, 610 may be configured to redirect light out of their respective waveguides and output the light with an appropriate amount of divergence or collimation for the particular depth plane associated with the waveguide. As a result, waveguides with different associated depth planes may have different configurations of outcoupling optical elements 570, 580, 590, 600, 610 that output light with different amounts of divergence depending on the associated depth plane. In some embodiments, the light extraction optical elements 570, 580, 590, 600, 610 may be solid or surface features that may be configured to output light at a specific angle. For example, the light extraction optical elements 570, 580, 590, 600, 610 may be solid holograms, surface holograms, and / or diffraction gratings. In some embodiments, the features 320, 330, 340, 350 may not be lenses. Rather, they may simply be spacers (eg, cladding layers and / or structures to form an air gap).

[0085] In some embodiments, the outcoupling optical elements 570, 580, 590, 600, 610 are diffractive features or "diffractive optical elements" (also referred to herein as "DOEs") that form a diffraction pattern. Preferably, the DOEs have a sufficiently low diffraction efficiency such that only a portion of the light in the beam is deflected toward the eye 210 at each intersection of the DOE, while the remainder continues traveling through the waveguide via TIR. The light carrying the image information is thus split into several related exit beams that exit the waveguide at multiple locations, resulting in a very uniform pattern of exit emission toward the eye 210 for this particular collimated beam bouncing within the waveguide.

[0086] In some embodiments, one or more DOEs may be switchable between an "on" state in which they actively diffract and an "off" state in which they do not significantly diffract. For example, a switchable DOE may comprise a layer of polymer dispersed liquid crystal in which the microdroplets comprise a diffractive pattern within a host medium, and the refractive index of the microdroplets may be switched to substantially match the refractive index of the host material (in which case the pattern does not significantly diffract incident light), or the microdroplets may be switched to a refractive index that does not match that of the host medium (in which case the pattern actively diffracts incident light).

[0087] In some embodiments, a camera assembly 630 (e.g., a digital camera, including visible and infrared light cameras) may be provided to capture images of the eye 210 and / or tissue surrounding the eye 210, for example, to detect user input and / or monitor a physiological condition of the user. As used herein, a camera may be any image capture device. In some embodiments, the camera assembly 630 may include an image capture device and a light source that projects light (e.g., infrared light) onto the eye, which may then be reflected by the eye and detected by the image capture device. In some embodiments, the camera assembly 630 may be mounted to the frame 80 (FIG. 2) and may be in electrical communication with the processing modules 140 and / or 150, which may process image information from the camera assembly 630. In some embodiments, one camera assembly 630 may be utilized per eye, monitoring each eye separately.

[0088] 7, an example of an output beam output by a waveguide is shown. Although one waveguide is shown, it should be understood that other waveguides in the waveguide assembly 260 (FIG. 6) may function similarly and that the waveguide assembly 260 includes multiple waveguides. Light 640 is launched into the waveguide 270 at the input surface 460 of the waveguide 270 and propagates within the waveguide 270 by TIR. At the point where the light 640 impinges on the DOE 570, a portion of the light exits the waveguide as an output beam 650. The output beam 650 is shown as approximately parallel, but may be redirected to propagate to the eye 210 at an angle (e.g., forming a diverging output beam), as discussed herein and depending on the depth plane associated with the waveguide 270. It should be understood that a nearly collimated exit beam may refer to a waveguide with outcoupling optics that outcouples light to form an image that appears to be set at a depth plane at a large distance (e.g., optical infinity) from the eye 210. Other waveguides or other sets of outcoupling optics may output a more divergent exit beam pattern, which would require the eye 210 to accommodate to a closer distance and focus on the retina, and would be interpreted by the brain as light from a distance closer to the eye 210 than optical infinity.

[0089] In some embodiments, a full color image may be formed at each depth plane by overlaying an image in each of the primary colors, for example, three or more primary colors. FIG. 8 illustrates an example of a stacked waveguide assembly in which each depth plane includes an image formed using multiple different primary colors. The illustrated embodiment shows depth planes 240a-240f, but more or less depths are also contemplated. Each depth plane may have three or more primary color images associated with it, including a first image of a first color G, a second image of a second color R, and a third image of a third color B. The different depth planes are indicated in the diagram by different numbers for diopters (dpt) following the letters G, R, and B. Merely by way of example, the numbers following each of these letters indicate the diopters (1 / m), i.e., the inverse distance of the depth plane from the viewer, and each box in the diagram represents an individual primary color image. In some embodiments, the exact locations of the depth planes for the different primary colors may be varied to account for differences in the eye's focusing of light of different wavelengths. For example, different primary color images for a given depth plane may be placed on depth planes that correspond to different distances from the user. Such an arrangement may increase visual acuity and user comfort, and / or reduce chromatic aberration.

[0090] In some embodiments, the light of each primary color may be output by a single dedicated waveguide, such that each depth plane may have multiple waveguides associated with it. In such embodiments, each box in the figure containing the letter G, R, or B may be understood to represent an individual waveguide, and three waveguides may be provided per depth plane, with three primary color images provided per depth plane. The waveguides associated with each depth plane are shown adjacent to each other in this drawing for ease of illustration, but it should be understood that in a physical device, the waveguides may all be arranged in a stack with one waveguide per level. In some other embodiments, multiple primary colors may be output by the same waveguide, such that, for example, only a single waveguide may be provided per depth plane.

[0091] 8, in some embodiments, G is green, R is red, and B is blue. In some other embodiments, other colors associated with other wavelengths of light, including magenta and cyan, may be used in addition to or in place of one or more of red, green, or blue.

[0092] It should be appreciated that references to a given color of light throughout this disclosure will be understood to encompass one or more wavelengths of light within the range of wavelengths of light that are perceived by a viewer as being of that given color. For example, red light may include one or more wavelengths of light within a range of about 620-780 nm, green light may include one or more wavelengths of light within a range of about 492-577 nm, and blue light may include one or more wavelengths of light within a range of about 435-493 nm.

[0093] In some embodiments, the light source 540 (FIG. 6) may be configured to emit light at one or more wavelengths outside the range of visual perception of a viewer, e.g., infrared and / or ultraviolet wavelengths. Additionally, the waveguide in-coupling, out-coupling, and other light redirecting structures of the display 250 may be configured to direct and emit this light from the display toward the user's eye 210, e.g., for imaging and / or user stimulation applications.

[0094] 9A, in some embodiments, light impinging on a waveguide may need to be redirected to in-couple the light into the waveguide. An in-coupling optical element may be used to redirect and in-couple the light into its corresponding waveguide. FIG. 9A illustrates a cross-sectional side view of an example of a plurality or set 660 of stacked waveguides, each including an in-coupling optical element. Each of the waveguides may be configured to output light of one or more different wavelengths or one or more different wavelength ranges. The stack 660 may correspond to the stack 260 (FIG. 6), and the illustrated waveguides of the stack 660 may correspond to a portion of the plurality of waveguides 270, 280, 290, 300, 310, although it should be understood that light from one or more of the image input devices 360, 370, 380, 390, 400 is input into the waveguide from a position that requires the light to be redirected for in-coupling.

[0095] The illustrated set 660 of stacked waveguides includes waveguides 670, 680, and 690. Each waveguide includes an associated incoupling optical element (which may also be referred to as the light input area on the waveguide), for example, incoupling optical element 700 is disposed on a major surface (e.g., the top major surface) of waveguide 670, incoupling optical element 710 is disposed on a major surface (e.g., the top major surface) of waveguide 680, and incoupling optical element 720 is disposed on a major surface (e.g., the top major surface) of waveguide 690. In some embodiments, one or more of the incoupling optical elements 700, 710, 720 may be disposed on the bottom major surface of an individual waveguide 670, 680, 690 (particularly, one or more of the incoupling optical elements are reflective polarizing optical elements). As shown, the incoupling optical elements 700, 710, 720 may be disposed on the upper major surface of the respective waveguides 670, 680, 690 (or on top of the next lower waveguide), and in particular, the incoupling optical elements are transmissive deflecting optical elements. In some embodiments, the incoupling optical elements 700, 710, 720 may be disposed within the body of the respective waveguides 670, 680, 690. In some embodiments, as discussed herein, the incoupling optical elements 700, 710, 720 are wavelength selective such that they selectively redirect one or more wavelengths of light while transmitting other wavelengths of light. Although illustrated on one side or corner of the respective waveguides 670, 680, 690, it should be understood that the incoupling optical elements 700, 710, 720 may be disposed within other areas of the respective waveguides 670, 680, 690 in some embodiments.

[0096] As shown, the in-coupling optical elements 700, 710, 720 may be laterally offset from one another. In some embodiments, each in-coupling optical element may be offset to receive light without that light passing through another in-coupling optical element. For example, each in-coupling optical element 700, 710, 720 may be configured to receive light from a different image input device 360, 370, 380, 390, and 400, as shown in FIG. 6, and may be separated (e.g., laterally spaced) from the other in-coupling optical elements 700, 710, 720 such that it does not receive substantially any light from others of the in-coupling optical elements 700, 710, 720.

[0097] Each waveguide also includes an associated optically dispersive element, for example, optically dispersive element 730 is disposed on a major surface (e.g., a top major surface) of waveguide 670, optically dispersive element 740 is disposed on a major surface (e.g., a top major surface) of waveguide 680, and optically dispersive element 750 is disposed on a major surface (e.g., a top major surface) of waveguide 690. In some other embodiments, optically dispersive elements 730, 740, 750 may be disposed on the bottom major surfaces of associated waveguides 670, 680, 690, respectively. In some other embodiments, optically dispersive elements 730, 740, 750 may be disposed on both the top and bottom major surfaces of associated waveguides 670, 680, 690, respectively, or optically dispersive elements 730, 740, 750 may be disposed on different ones of the top and bottom major surfaces in different associated waveguides 670, 680, 690, respectively.

[0098] The waveguides 670, 680, 690 may be spaced apart and separated, for example, by gas, liquid, and / or solid layers of material. For example, as shown, layer 760a may separate the waveguides 670 and 680, and layer 760b may separate the waveguides 680 and 690. In some embodiments, layers 760a and 760b are formed from a low index material (i.e., a material having a lower index of refraction than the material forming the immediately adjacent ones of the waveguides 670, 680, 690). Preferably, the index of refraction of the material forming layers 760a, 760b is 0.05 or more, or 0.10 or less, relative to the index of refraction of the material forming the waveguides 670, 680, 690. Advantageously, the lower refractive index layers 760a, 760b may act as cladding layers to promote total internal reflection (TIR) ​​of light through the waveguides 670, 680, 690 (e.g., TIR between the top and bottom major surfaces of each waveguide). In some embodiments, the layers 760a, 760b are formed from air. Although not shown, it should be understood that the top and bottom of the illustrated set of waveguides 660 may include immediate cladding layers.

[0099] Preferably, for ease of manufacturing and other considerations, the materials forming the waveguides 670, 680, 690 are similar or the same, and the materials forming the layers 760a, 760b are similar or the same. In some embodiments, the materials forming the waveguides 670, 680, 690 may vary between one or more of the waveguides, and / or the materials forming the layers 760a, 760b may differ while still maintaining the various refractive index relationships described above.

[0100] 9A, light rays 770, 780, 790 are incident on the set of waveguides 660. It should be understood that light rays 770, 780, 790 may be injected into the waveguides 670, 680, 690 by one or more image injection devices 360, 370, 380, 390, 400 (FIG. 6).

[0101] In some embodiments, the light beams 770, 780, 790 have different properties, e.g., different wavelengths or different wavelength ranges, that may correspond to different colors. Each of the incoupling optical elements 700, 710, 720 deflects the incident light such that the light propagates through a respective one of the waveguides 670, 680, 690 by TIR. In some embodiments, each of the incoupling optical elements 700, 710, 720 selectively deflects one or more particular wavelengths of light while transmitting other wavelengths to the underlying waveguide and associated incoupling optical element.

[0102] For example, incoupling optical element 700 may be configured to deflect light beam 770 having a first wavelength or wavelength range while transmitting light beams 780 and 790 having different second and third wavelengths or wavelength ranges, respectively. The transmitted light beam 780 impinges on and is deflected by incoupling optical element 710, which is configured to deflect light of the second wavelength or wavelength range. Light beam 790 is deflected by incoupling optical element 720, which is configured to selectively deflect light of the third wavelength or wavelength range.

[0103] 9A, the deflected light rays 770, 780, 790 are deflected to propagate through the corresponding waveguides 670, 680, 690. That is, the incoupling optical element 700, 710, 720 of each waveguide deflects the light into its corresponding waveguide 670, 680, 690 and incouplings the light into its corresponding waveguide. The light rays 770, 780, 790 are deflected at an angle that causes the light to propagate through the respective waveguides 670, 680, 690 by TIR. The light rays 770, 780, 790 propagate through the respective waveguides 670, 680, 690 by TIR until they strike the corresponding optical dispersive element 730, 740, 750 of the waveguide.

[0104]

[0046] Referring now to Figure 9B, a perspective view of the multiple stacked waveguide embodiment of Figure 9A is illustrated. As described above, the in-coupled light rays 770, 780, 790 are deflected by the in-coupling optical elements 700, 710, 720, respectively, and then propagate by TIR within the waveguides 670, 680, 690, respectively. The light rays 770, 780, 790 then impinge on the optically dispersive elements 730, 740, 750, respectively. The optically dispersive elements 730, 740, 750 deflect the light rays 770, 780, 790 to propagate towards the out-coupling optical elements 800, 810, 820, respectively.

[0105] In some embodiments, the optically dispersive elements 730, 740, 750 are orthogonal pupil expanders (OPEs). In some embodiments, the OPEs deflect or disperse the light to the out-coupling optical elements 800, 810, 820, and in some embodiments may also increase the beam or spot size of this light as it propagates to the out-coupling optical elements. In some embodiments, the optically dispersive elements 730, 740, 750 may be omitted and the in-coupling optical elements 700, 710, 720 may be configured to deflect the light directly to the out-coupling optical elements 800, 810, 820. For example, referring to FIG. 9A, the optically dispersive elements 730, 740, 750 may be replaced with the out-coupling optical elements 800, 810, 820, respectively. In some embodiments, the outcoupling optical elements 800, 810, 820 are exit pupils (EPs) or exit pupil expanders (EPEs) that direct light to the viewer's eye 210 (FIG. 7). It should be understood that an OPE may be configured to increase the size of the eyebox in at least one axis, and an EPE may increase the eyebox in an axis that intersects, e.g., is orthogonal to, the axis of the OPE. For example, each OPE may be configured to redirect a portion of the light striking the OPE to an EPE of the same waveguide, while allowing the remaining portion of the light to continue propagating down the waveguide. In response to striking the OPE again, another portion of the remaining light is redirected to the EPE, which continues to propagate further down the waveguide, etc. Similarly, in response to striking the EPE, a portion of the impinging light is directed out of the waveguide towards the user, and the remaining portion of the light continues to propagate through the waveguide until it strikes the EP again, at which point another portion of the impinging light is directed out of the waveguide, etc. As a result, a single beam of internally coupled light may be "replicated" each time a portion of that light is redirected by an OPE or EPE, thereby forming a cloned beam field of light, as shown in Figure 6. In some embodiments, the OPE and / or EPE may be configured to modify the size of the beam of light.

[0106] Thus, referring to Figures 9A and 9B, in some embodiments, a set of waveguides 660 includes, for each primary color, a waveguide 670, 680, 690, an in-coupling optical element 700, 710, 720, an optical dispersion element (e.g., OPE) 730, 740, 750, and an out-coupling optical element (e.g., EP) 800, 810, 820. The waveguides 670, 680, 690 may be stacked with an air gap / cladding layer between each one. The in-coupling optical element 700, 710, 720 redirects or deflects the incoming light into its waveguide (with different in-coupling optical elements receiving different wavelengths of light). The light then propagates at an angle that will result in TIR within the individual waveguides 670, 680, 690. In the illustrated embodiment, light ray 770 (e.g., blue light) is deflected by the first in-coupling optical element 700 in the manner described above, then continues to bounce back down the waveguide, interacting with the optically dispersive element (e.g., OPE) 730 and then the out-coupling optical element (e.g., EP) 800. Light rays 780 and 790 (e.g., green and red light, respectively) pass through the waveguide 670, where light ray 780 impinges on and is deflected by the in-coupling optical element 710. Light ray 780 will then proceed via TIR back down the waveguide 680, to its optically dispersive element (e.g., OPE) 740 and then the out-coupling optical element (e.g., EP) 810. Finally, light ray 790 (e.g., red light) passes through the waveguide 690 and impinges on the optically in-coupling optical element 720 of the waveguide 690. The light in-coupling optical element 720 deflects the light beam 790 such that it propagates by TIR to a light dispersive element (e.g., OPE) 750 and then by TIR to an out-coupling optical element (e.g., EP) 820. The out-coupling optical element 820 then finally out-couples the light beam 790 to a viewer, who also receives the out-coupled light from the other waveguides 670, 680.

[0107] FIG. 9C illustrates a top-down plan view of an example of the multiple stacked waveguides of FIGS. 9A and 9B. As shown, the waveguides 670, 680, 690 may be vertically aligned with each waveguide's associated light dispersive element 730, 740, 750 and associated out-coupling optical elements 800, 810, 820. However, as discussed herein, the in-coupling optical elements 700, 710, 720 are not vertically aligned. Rather, the in-coupling optical elements are preferably non-overlapping (e.g., laterally spaced apart, as seen in the top-down view). As discussed further herein, this non-overlapping spatial arrangement facilitates the injection of light from different sources into different waveguides on a one-to-one basis, thereby allowing a specific light source to be uniquely coupled to a specific waveguide. In some embodiments, an arrangement including non-overlapping spatially separated in-coupling optical elements may be referred to as a shifted pupil system, and the in-coupling optical elements in these arrangements may correspond to sub-pupils.

[0108] Eye and Environmental Imaging As discussed above, head mounted displays can be used to provide a user with image content integrated with, in conjunction with, and / or superimposed over a view of the world in front of the wearer. Such head mounted display systems can be configured to project light into the user's eyes to form augmented reality image content and transmit light from the environment in front of the user to the user. The head mounted display system may include one or more cameras to image the environment and / or the user's eyes. An outward facing camera may be used to image the environment directly, for example, to determine where the augmented reality image content should be placed relative to objects in the environment. For example, imaging the environment may provide the location of a table so that the head mounted display may render an image of a person standing next to the table instead of on or in the table. An inward facing camera may be used to image the eye directly, for eye tracking, etc. Disclosed herein are examples of head mounted display systems and / or imaging systems that may be configured to image the eye and / or the environment as well. In some designs, the system does not require inwardly and / or outwardly facing cameras to directly image the eye and / or the environment, respectively. Such a system may employ one or more cameras configured to receive light from the eye / environment via an eyepiece, such as one or more waveguides in the eyepiece in optical communication with the one or more cameras. Once the light is collected by the waveguides, the one or more cameras can generate an image of the eye and / or the environment in front of the user. Using a waveguide to collect light for imaging the eye and / or the environment can potentially reduce the form factor of the head mounted display, making the head mounted display potentially more compact and / or aesthetically pleasing.

[0109] FIG. 10 illustrates an exemplary imaging system 900 configured to image an eye integrated with an eyepiece 950 that may be used on a head mounted display. The eyepiece 950, which may be placed in front of a user's eye 210, can be used to both inject image content into the eye and image the eye. FIG. 10 shows one eyepiece 950 in front of one eye 210. Various head mounted display systems such as that shown in FIG. 2 may include a pair of eyepieces 950 and associated components placed in front of respective left and right eyes 210. Although a single waveguide 940 is shown in FIG. 10, the waveguide 940 may include one, two, three, four, six, seven, eight, or more waveguides (e.g., a stack of one or more waveguides).

[0110] The imaging system 900 may include a light source or illumination source 960 to illuminate the eye and facilitate image capture, an eyepiece 950 with a waveguide 940 configured to propagate light therein, and / or an imaging device 920 such as a camera for image capture. Also shown is an image projector 930 for producing an image that may be injected into the eye via the eyepiece 950. The eyepiece 950 may include one or more waveguides 940 configured to carry light from the illumination source 960 and / or image projector 930 to the eye and carry light from the eye to the camera 920. The eyepiece 950 may further include one or more coupling optical elements 944 to illuminate the eye and to couple light from the waveguide 940 out to the eye for image injection and / or from the eye into the waveguide for image capture. The eyepiece 950 may additionally include one or more internal coupling optical elements 942 for coupling light from the illumination source 960 and / or image projector 930 into the waveguide 940, and one or more external coupling optical elements 952 for coupling light from the waveguide to the camera 920.

[0111] The eyepiece 950 may be disposed on a frame that is worn on the head. The eyepiece 950 may be disposed in front of the eye 210. The eyepiece 950 may have an inner or nasal side that is closer to the wearer's nose and an opposing outer or temple side that is closer to and farther from the wearer's temple. In FIG. 10, the coupling optical element 944 is inner or nasal (outer or temple side relative to the coupling optical element 944) relative to the inner coupling 942 and outer coupling 952 optical elements. The illumination source 960 is also inner or nasal (or the image projector is outer or temple side relative to the illumination source) relative to the image projector 930. However, the relative positions may be different. For example, the illumination source 960 may be outer or temple side relative to the image projector 930 in some designs.

[0112] The waveguide 940 may comprise a sheet or layer having two major surfaces (front and rear surfaces) with the largest surface area arranged opposite each other. The front surface may be further from the user's eye 210 when the user wears the head mounted display (closer to the wearer's front environment), and the rear surface is closer to the user's eye (and further from the wearer's front environment). The waveguide 940 may include a transparent material (e.g., glass, plastic) with a refractive index greater than 1.0 so that light can be guided therein between the major surfaces by total internal reflection. Elements with the same number may have the same functionality for one or more of the embodiments described herein.

[0113] A coupling optical element 944 may be disposed on or within the waveguide 940 for coupling light from the eye 210 to the waveguide 940 and / or from the waveguide to the eye. As shown in FIG. 10 , the coupling optical element 944 may be disposed in an optical path between the user's eye 210 and the waveguide 940 such that light coupled from the waveguide 940 via the coupling optical element 944 may be incident on the user's eye 210 (e.g., for illuminating the eye and / or for imaging). The coupling optical element 944 may comprise a number of redirecting features configured to redirect light guided in the waveguide out of the waveguide or redirect light incident on the coupling optical element 944 at an angle into the waveguide to be guided therein by total internal reflection. The coupling optical element 944 and the redirecting features may be physically engaged with the waveguide 940. For example, the coupling optical element 944 may comprise a holographic or diffractive optical element (e.g., a surface relief grating) patterned (e.g., etched) in or on the waveguide 940. The coupling optical element 944 may comprise a layer disposed on the waveguide 940 or may be formed within the waveguide 940. For example, a volume holographic or other diffractive optical element may be formed by varying the refractive index of a material making up the waveguide or a layer disposed thereon. Thus, the coupling optical element 944 may be disposed as a layer disposed within the volume of the waveguide 940 or on it.

[0114] Depending on the design, the combining optical element 944 may be transmissive or reflective and may operate transmissively or reflectively. For example, the combining optical element 944 may include a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates transmissively or reflectively, respectively, e.g., redirecting light transmitted through or reflected therefrom. The combining optical element 944 may include a polarizing optical element, such as a polarization-selective redirecting element (e.g., a polarizer). The polarization-selective redirecting element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may comprise a liquid crystal structure, such as a liquid crystal polarization grating. The combining optical element 944 may be configured to direct light from the image projector 930 and / or the light source 960, which is guided in the waveguide 940 by total internal reflection (TIR), at an angle less than the critical angle (e.g., an angle that is more normal) to be ejected out of the waveguide to the eye 210 of the user. Additionally or alternatively, the coupling optical element 944 may be configured to couple light from the eye 210 into the waveguide 940 at an angle greater than the critical angle (e.g., an angle that is less than normal) so as to be directed therein to the camera 920 by total internal reflection.

[0115] As shown in FIG. 10 , an internal coupling optical element 942 for coupling light from the illumination source 960 and / or the image projector 930 into the waveguide 940 may be disposed on or within the waveguide 940. The internal coupling optical element 942 may be disposed in an optical path between the light source 960 and the waveguide 940 such that light coupled from the light source 960 via the internal coupling optical element 942 is guided within the waveguide 940. The internal coupling optical element 942 may comprise a number of redirecting features configured to redirect light incident thereon at an angle into the waveguide, for example to be guided therein by total internal reflection. The internal coupling optical element 942 may comprise a liquid crystal structure, such as a liquid crystal polarization grating. Additionally or alternatively, the internal coupling optical element 942 may include a blazed grating. The internal coupling optical element 942 may comprise a layer disposed on the waveguide 940, or may be formed (e.g., patterned) on or within the waveguide 940, or may be otherwise fabricated therein. For example, a surface holographic or diffractive optical element (e.g., a surface relief grating) may be fabricated by patterning (e.g., etching) the surface of the waveguide or a layer thereon. A volume holographic or diffractive optical element may also be formed by changing the refractive index of a material that constitutes the waveguide or a layer thereon. Thus, the internal coupling optical element 942 may be disposed within the volume of the waveguide 940 or a layer thereon. Depending on the design, the internal coupling optical element 942 may be transmissive or reflective and may operate transmissively or reflectively. For example, the internal coupling optical element 942 may include a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates transmissively or reflectively, e.g., redirecting light transmitted through or reflected therefrom, respectively.

[0116] The internal coupling optical element 942 may comprise a reflective optical element (e.g., a mirror). For example, the internal coupling optical element 942 may comprise an off-axis reflector. Additionally or alternatively, the internal coupling optical element 942 and / or the coupling optical element 944 may comprise a polarizing optical element, such as a polarization selective redirecting element (e.g., a polarizer). The polarization selective redirecting element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may comprise a liquid crystal structure, such as a liquid crystal polarization grating. For example, one or both of the internal coupling optical element 942 and / or the coupling optical element 944 may include a liquid crystal polarization grating (LCPG). LCPGs can potentially provide high efficiency diffraction at broad wavelengths. Thus, LCPGs may be useful for the internal coupling optical element 942 and / or the coupling optical element 944. LCPGs may be polarization dependent. The LCPG or other type of liquid crystal grating, diffractive optical element, or optical element may include a pattern or arrangement of molecules of liquid crystals configured to provide one or more functions, such as redirecting light into or out of the waveguide. Thus, the internal coupling optical element 942 and / or the coupling optical element 944 may comprise a polarization grating. Additionally or alternatively, the internal coupling optical element 942 and / or the coupling optical element 944 may comprise liquid crystals, and thus in some implementations, one or both may be liquid crystal gratings or liquid crystal diffractive optical elements. Additionally or alternatively, one or both of the internal coupling optical element 942 and / or the coupling optical element 944 may include a blazed grating. In some designs, the internal coupling optical element 942 comprises a liquid crystal reflector, such as a cholesteric liquid crystal reflective lens (e.g., a reflective liquid crystal diffractive lens, a Bragg reflective structure, a reflective liquid crystal diffraction grating, etc.). Some non-limiting examples of liquid crystal gratings, liquid crystal polarizer gratings, and other liquid crystal optical elements are described in the following published applications, each of which is incorporated by reference herein in its entirety and for all purposes: "MULTILAYER LIQUID CRYSTAL DIFFRACTIVE GRATINGS FOR REDIRECTING LIGHT" and "MULTILAYER LIQUID CRYSTAL DIFFRACTIVE GRATINGS FOR REDIRECTING LIGHT" (Patent Publication No. 2005-2009022). No. 2018 / 0143438, entitled "SPATIALLY VARIABLE LIQUID CRYSTAL DIFFRACTION GRATINGS", filed November 16, 2017; U.S. Patent Publication No. 2018 / 0143485, entitled "WAVEGUIDE LIGHT MULTIPLEXER USING CROSSED GRATINGS", filed November 16, 2017; U.S. Patent Publication No. 2018 / 0143509, entitled "WAVEGUIDE LIGHT MULTIPLEXER USING CROSSED GRATINGS", filed November 16, 2017; and U.S. Patent Publication No. 2018 / 0239147, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR", filed February 22, 2018, entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES No. 2018 / 0239177, filed Feb. 22, 2018, entitled "DIFFRACTIVE DEVICES BASED ON POLARIZATION CONVERSION," and U.S. Patent Publication No. 2018 / 0164627, filed Dec. 7, 2017, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL." However, the design of the internal coupling optical element 942 and / or the coupling optical element 944 is not limited thereto and may include other types of optical elements, diffractive optical elements, liquid crystal optical elements, liquid crystal gratings, and liquid crystal polarization gratings. Further information regarding examples of cholesteric liquid crystal structures such as reflectors may also be found in the section below entitled "Cholesteric Liquid Crystal Mirrors." As discussed above, other liquid crystal optical elements and other non-liquid crystal optical elements may be used. Thus, many types of coupling optical elements (e.g., internal coupling optical element 942 and / or coupling optical element 944), diffractive optical elements, gratings, polarization gratings, etc. may be used, both those described herein and other types of gratings, diffractive optical elements, liquid crystal elements, and optical elements in general.In various implementations, the internal coupling optical element 942 may be configured to couple light from the image projector 930 and / or the light source 960 into the waveguide at an angle greater than the critical angle so that it is guided within the waveguide 940 to the user's eye 210 by total internal reflection to the eye.

[0117] The waveguide 940 may comprise one or more waveguides. In some implementations, the one or more waveguides 940 comprise a stack of waveguides. In some designs, for example, different waveguides of a stack of waveguides are configured to output light with different wavefront divergences as if projected from different distances from the user's eye. For example, a first waveguide or group of waveguides may be configured to output light that is collimated or has a first divergence as if projected from a first depth, and a second waveguide or group of waveguides may be configured to output light that is diverging (not collimated) or at a second divergence (greater than the first divergence) as if projected from a second depth that is closer than the first depth. In some designs, the different waveguides may be configured to output light that has different associated colors. For example, a first waveguide may be configured to output red light, a second waveguide may be configured to output green light, a third waveguide may be configured to output blue light, and a fourth waveguide may be configured to output and / or input infrared light.

[0118] An outcoupling optical element 952 for coupling light from the waveguide 940 to the camera 920 as shown in FIG. 10 may comprise a number of redirecting features configured to redirect light at an angle, for example, such that light incident thereon is not guided within the waveguide but is redirected out of the waveguide to the camera. The outcoupling optical element 952 may be disposed within the interior of the waveguide 940 or may be patterned (e.g., etched) in or on a surface (e.g., a major surface) of the waveguide 940. For example, a surface holographic or diffractive optical element (e.g., a surface relief grating) may be fabricated by patterning (e.g., etching) the surface of the waveguide or a layer thereon. A volume holographic or diffractive optical element may also be formed by changing the refractive index of a material that constitutes the waveguide or a layer thereon. Depending on the design, the outcoupling optical element 952 may be transmissive or reflective and may operate transmissively or reflectively. For example, the outcoupling optical element 952 may include a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates transmissively or reflectively, respectively, e.g., redirecting light transmitted through or reflected therefrom.

[0119] The out-coupling optical element 942 may comprise a reflective optical element (e.g., a mirror). For example, the out-coupling optical element 952 may comprise an off-axis reflector. In some designs, the out-coupling optical element 952 may comprise a polarizing optical element, such as a polarization selective redirecting element (e.g., a polarizer). Thus, the polarization selective redirecting element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may comprise a liquid crystal structure, such as a liquid crystal polarization grating. In some implementations, for example, the out-coupling optical element 952 may comprise a liquid crystal polarization grating (LCPG). LCPGs can potentially provide high efficiency diffraction at broad wavelengths. Similarly, LCPGs may be useful for the out-coupling optical element 952. LCPGs may be polarization dependent. LCPGs or other types of liquid crystal gratings may include a pattern or arrangement of molecules of liquid crystals configured to provide one or more functions, such as redirecting light into or out of a waveguide. Thus, the out-coupling optical element 952 may comprise a polarization grating. Additionally or alternatively, the outcoupling optical element 952 can comprise liquid crystals and thus in some implementations may be other liquid crystal optical elements such as liquid crystal gratings or liquid crystal diffractive optical elements. Additionally or alternatively, the outcoupling optical element 952 can include a blazed grating. In some designs, the outcoupling optical element 952 comprises a liquid crystal reflector such as a cholesteric liquid crystal reflective lens (e.g., a reflective liquid crystal diffractive lens, a Bragg reflective structure, a reflective liquid crystal diffraction grating, etc.).Some non-limiting examples of liquid crystal gratings, liquid crystal polarizer gratings, and other liquid crystal optical elements are described in the following published applications, each of which is incorporated by reference herein in its entirety and for all purposes: U.S. Patent Publication No. 2018 / 0143438, entitled "MULTILAYER LIQUID CRYSTAL DIFFRACTIVE GRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES," filed on November 16, 2017; U.S. Patent Publication No. 2018 / 0143485, entitled "SPATIALLY VARIABLE LIQUID CRYSTAL DIFFRACTION GRATINGS," filed on November 16, 2017; and U.S. Patent Publication No. 2018 / 0143485, entitled "WAVEGUIDE LIGHT MULTIPLEXER USING CROSSED ANGLE RANGES," filed on November 16, 2017. No. 2018 / 0143509, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR," filed November 16, 2017; U.S. Patent Publication No. 2018 / 0239147, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR," filed February 22, 2018; U.S. Patent Publication No. 2018 / 0239177, entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION," filed February 22, 2018; and U.S. Patent Publication No. 2018 / 0164627, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL," filed December 7, 2017. However, the design of the outcoupling optical element 952 is not limited thereto and may include other types of optical elements, diffractive optical elements, liquid crystal optical elements, liquid crystal gratings, and liquid crystal polarization gratings. Further information regarding examples of cholesteric liquid crystal structures such as reflectors may also be found in the section entitled "Cholesteric Liquid Crystal Mirrors" below. As discussed above, other liquid crystal optical elements and other non-liquid crystal optical elements may be used.Thus, many types of coupling optical elements (e.g., outcoupling optical element 952), both those described herein and other types of gratings, diffractive optical elements, liquid crystal elements, or optical elements in general, may be used, such as diffractive optical elements, gratings, polarization gratings, etc. As referenced above, outcoupling optical element 952 may be configured to redirect light guided in waveguide 940 at an angle less than the critical angle so that it is not guided in the waveguide by total internal reflection, but is instead injected into camera 920.

[0120] In various designs, the coupling optical element 944 may be transparent in the visible spectrum so that the user can see the environment in front of the user through the coupling optical element 944 and the eyepiece 950. The internal coupling optical element 942 may also redirect light in the visible spectrum, for example, when the internal coupling optical element is used to receive light from the image projector 930 and / or when the illumination source 960 is configured to output visible light and illuminate the eye 210 with visible light. In some embodiments, the internal coupling optical element 942 is configured to redirect infrared light, for example, when the illumination source 960 is configured to output infrared light and illuminate the eye 210 with infrared light. In some designs, such as shown in FIG. 10, the internal coupling optical element 942 may be medial or nasal to the external coupling optical element 952. However, in other designs, the internal coupling optical element 942 may be lateral or temporal to the external coupling optical element 952. In one implementation, as shown in FIG. 10, the out-coupling optical element 952 may be adjacent to the in-coupling optical element 942, although non-adjacent positioning is also possible.

[0121] The illumination source 960 may be disposed on the same side (e.g., posterior or proximal) of the eyepiece 950 as the eye 210 (proximal may refer to the side closest to the eye 210), as shown in FIG. 10 . Alternatively, the illumination source 960 may be disposed on the opposite side (e.g., anterior or distal) from the eye 210. The illumination source 960 may be configured to direct light into at least one of the major surfaces of the waveguide 940 via the internal coupling optical element 942. The light source 960 may be configured to emit invisible light (e.g., infrared light). The light source 960 may include one or more LEDs. The LEDs may comprise infrared LEDs. The light source 960 may be configured to emit coherent light. In some designs, the light source 960 comprises a laser (e.g., an infrared laser). In some designs, the light source 960 emits pulsed light. For example, the camera 920 can be configured to periodically capture images. Thus, the illumination source 960 can be pulsed to coincide with the period during which the camera is acquiring images. The intensity output from the illumination source 960 can be reduced when the camera is not acquiring images. By concentrating the total energy of the illumination for a short period of time, an increased signal-to-noise ratio can be obtained without exposing the eye 210 to unsafe intensity levels. In some cases, for example, the camera 920 captures one image every 30 milliseconds, and the camera exposure time is a few milliseconds. The illumination source 960 can be configured to output pulses with a similar period and duration to match that of the camera 920.

[0122] In some implementations, different light sources having different wavelengths are alternatively pulsed to provide different wavelength illumination at different times, as discussed below.

[0123] The in-coupling optical element 942 may be in direct optical communication with the illumination source 960 and / or the image projector 930, for example, to direct light therein from the image projector 930 and / or the light source 960. For example, light emitted by the light source 960 may be incident on the in-coupling optical element 942 before optically interacting with either the coupling optical element 944 and / or the out-coupling optical element 952.

[0124] As shown in FIGS. 11A-11E, light 902 projected from an image projector 930 may form an image on the retina. The image projector 930 may include a light source, a modulator, and / or projection optics. The light source for the image projector 930 may include one or more LEDs, lasers, or other light sources, and may include one or more visible light sources. The modulator may include a spatial light modulator, such as a liquid crystal spatial light modulator. Such a spatial light modulator may be configured, for example, to modulate the intensity of light at different spatial locations. The projection optics may include one or more lenses. Other types of image projectors 930 capable of projecting and / or forming an image may also be employed. For example, the image projector 930 may include a scanning optical fiber.

[0125] The image projector 930 and the in-coupling optical element 942 may be in direct optical communication with one another. The image projector 930 may be matched with the in-coupling optical element 942, for example, into which light from the image projector 930 is directed. In some cases, the image projector 930 is disposed adjacent to the corresponding in-coupling optical element 942 and / or the waveguide 940. The image projector 930 may also be disposed in an optical path that includes the in-coupling optical element 942, the coupling optical element 944, and the eye 210.

[0126] The image projector 930 may be a separate element from the illumination source 960, as shown in Figures 10 and 11A-11E. However, in some cases, the image projector 930 may be used as an illumination source. For example, in addition to projecting an image into the eye 210, the image projector 930 may be used to direct visible and / or infrared light into and illuminate the eye for image capture. However, alternatively, one or more separate light sources 960 may be used to illuminate the eye 210 for image capture.

[0127] The light emitted by the illumination source 960 may comprise light in a particular wavelength range, such as, for example, invisible light. The illumination source 960 may be configured to project invisible (e.g., infrared) light onto / into the eye 210 to image one or more portions of the eye 210 (e.g., cornea, retina). In one example implementation, the light source 960 may be configured to emit light in a range of about 850 nm to 940 nm. The light source 960 may be configured to emit light extending over a wavelength range of at least about 20 nm. Other ranges are also possible. The emitted wavelength range may be 5 nm, 10 nm, 15 nm, 50 nm, 75 nm, 100 nm, 150 nm, 200 nm, or any range between any of these values. The light source 960 may be configured to emit light across a broad band of wavelengths, such as any range within the infrared spectrum.

[0128] The imaging device 920, which may comprise a camera, may comprise a detector array and possibly imaging optics. The detector array may comprise, for example, a CCD or CMOS detector array, and the imaging optics may comprise one or more lenses. The one or more lenses may have positive refractive power and associated focal lengths. In one design, the camera 920 is focused at infinity. For example, the optics may have a focal length f, and the detector array may be positioned at a distance away from the optics corresponding to the focal length such that objects at large distances are imaged onto the detector array. Similarly, light from the eye or objects in the environment that is collimated will also be focused onto the detector array, forming an image of the eye or object thereon.

[0129] The imaging device 920 may be positioned on an opposite side of the waveguide 940 from the illumination source 960 and / or eye 210. In some designs, the imaging device 920 may be positioned on the same side of the waveguide 940 as the illumination source 960 and / or eye 210. As shown in FIG. 10, the imaging device 920 may be positioned near the outer or temple edge of the eyepiece 950, although other locations are possible.

[0130] 11A-11E illustrate the operation of the example imaging system 900 of FIG. 10. FIG. 11A shows an illumination source 960 emitting light 902 toward an incoupling optic 942 on a waveguide 940. As shown, the light 902 can be directed to be incident on the eyepiece 950 at approximately normal, although other angles are possible. In some designs, the light source 960 is configured to emit collimated light into the eyepiece 950. As shown in FIG. 11B, the illumination light 902 can be coupled into the waveguide 940 via an incoupling optic 942. In some designs where the incoupling optic 942 comprises a diffractive optical element (e.g., a grating, a holographic element), the light incident thereon is diffracted at an angle above the critical angle of the waveguide, causing the incoupling light 904 to be directed within the eyepiece 950 by total internal reflection (TIR). In some designs, the in-coupling optical element 942 may be configured to direct light towards the coupling optical element 944. The in-coupling optical element 942 may be polarization selective. For example, the in-coupling optical element 942 may include a polarization selective redirecting element such as a polarization grating, such as a liquid crystal polarization grating. FIG 11C shows how the in-coupled light 904 propagates through the waveguide 940 by TIR.

[0131] FIG. 11D illustrates an example imaging system 900 that couples light out of the eyepiece 950. As the internally coupled light 904 propagates through the waveguide 940, a portion of the light may be incident on the coupling optical element 944. The coupling optical element 944 can be configured to couple the internally coupled light 904 out of the eyepiece 950 toward the user's eye 210. The coupling optical element 944 may be configured to couple the light as collimated light toward the eye 210. The coupling optical element 944 may be tuned to a particular wavelength range of light. For example, the coupling optical element 944 may be configured to couple infrared light (e.g., about 700 nm to 15,000 nm) out of the waveguide 940. In some designs, the coupling optical element 944 can be configured to couple multiple wavelengths of light out of the eyepiece 950. For example, the coupling optical element 944 may be tuned for both infrared and visible light. The coupling optical element 944 can also be configured to couple light into the waveguide 940, as described more fully below.

[0132] The coupling optical element 944 can be configured to increase one or more eyebox dimensions for the user. For example, the one or more dimensions may be measured along a first axis (e.g., the x-axis). The eyepiece 950 may further include an orthogonal pupil expander (OPE). The OPE may have at least one light redirecting element disposed on or within the waveguide (e.g., on one of the major surfaces), or the OPE may be disposed within the waveguide 940. The OPE may include features similar or the same as those described above with respect to the light dispersive elements 730, 740, 750 above. In some implementations, the light redirecting element may comprise a diffractive optical element. The OPE may be configured to increase the eyebox dimension along a second axis (e.g., the y-axis) that is orthogonal to the first axis.

[0133] 11D shows a portion of the light exiting the eyepiece 950 toward the user's eye 210. In some designs, the combining optical element 944 is configured such that the internally coupled light 904 incident on the combining optical element 944 at various portions of the combining optical element 944 along a first axis (e.g., parallel to the x-axis) exits the eyepiece 950 at each portion of the combining optical element 944 along the first axis. This may provide the user with light to project an image or illuminate the eye for different eye positions or locations.

[0134] As shown in FIGS. 11D-11E, the coupling optical element 944 may be configured to couple the in-coupled light 904 out of the eyepiece 950 as collimated light. This light may also generally be directed approximately normal to the major surfaces of the eyepiece 950 and / or the waveguide 940. The collimated light may be directed into the eye and focused onto the retina by the eye (e.g., the cornea and natural lens of the eye). This light 908 incident on the retina may provide illumination to image the retina and / or provide image content to the eye. A portion of this light 908 may, for example, be reflected or scattered from the retina and exit the eye to provide an image of the retina to be captured. The light source 960 may be an extended light source such that the light will illuminate an area of ​​the retina.

[0135] 12A-12E illustrate how the imaging system 900 of FIGS. 11A-11E may additionally or alternatively be used for image collection of the eye 210. FIG. 12A shows how light 910 reflected from the retina exits the eye 210. As shown, the light 910 scattered or reflected from the retina passing through the eye's natural lens, the pupil within the eye, and the cornea may be collimated. This light may also be incident on the eyepiece 950 at normal incidence (e.g., perpendicular to the major surfaces of the waveguide 940 and / or the coupling optical element 944). The coupling optical element 944 may be configured to couple the light 910 reflected from the retina into the waveguide 940.

[0136] FIG. 12B illustrates the example imaging system 900 as it couples light into the eyepiece 950. The coupling optical element 944 may include redirecting features, such as a diffractive optical element or other structure, that redirect the light at an angle greater than the critical angle to be guided within the waveguide 940. The coupling optical element 944 may be configured to direct the incoupled light 914 generally toward the light source 960 and / or the imaging device 920. The coupling optical element 944 can be configured to couple less than a percentage of this light propagating toward the camera 920 back out of the waveguide 940. For example, a partially reflective element (e.g., a semi-transparent mirror) may be disposed on or within the waveguide 940 to reduce leakage of the incoupled light 914 out of the waveguide 940 along the portion of the waveguide 940 where the coupling optical element 944 is disposed, while allowing a portion of the incoupled light 914 to continue propagating within the waveguide 940 by total internal reflection. The fraction of light that does not leak out may be any percentage between 0 and 1. For example, the fraction may be 0.90, such that 90% of the light rays propagating through the waveguide 940 along the coupling optical element 944 are maintained within the waveguide 940 at each reflection of the light rays. Other fractions are also possible (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or any range between any of these values). Such partially reflective elements may be used in the implementations described below as well.

[0137] As shown in FIG. 12C, the collimated in-coupled light 914 may continue to propagate through the waveguide 940 toward the imaging device 920. FIG. 12D illustrates how a portion of the in-coupled light 914 may continue to propagate until it is incident on one or more out-coupling optical elements 952. To reduce the amount of leakage of the in-coupled light 914 out of the in-coupling optical element 942, the in-coupling optical element 942 can be configured to couple very little of this light propagating toward the camera 920 back out of the waveguide. For example, a partially reflective element (e.g., a semi-transparent mirror) may be disposed on or within the waveguide 940 such that a portion of the in-coupled light 914 continues to propagate within the waveguide 940 by total internal reflection while reducing leakage of the in-coupled light 914 out of the waveguide 940 along the portion of the waveguide 940 where the in-coupling optical element 942 is disposed. The portion of light that does not leak out may be any percentage between 0 and 1. For example, the fraction may be 0.90, such that 90% of the light rays propagating through the waveguide 940 and along the coupling optical element 944 are maintained within the waveguide 940 at each reflection of the light rays. Other fractions are also possible (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or any range between any of these values). Such partially reflective elements can be used in the implementations described below as well.

[0138] 12E, the out-coupling optical element 952 can be configured to couple light guided in the waveguide 940 out of the waveguide 940 to the imaging device 920. As a result, light propagating in the waveguide 940 that is incident on the out-coupling element 952 can be redirected to be ejected out of the waveguide 940, e.g., out of a major surface of the waveguide 940 (e.g., at the front or rear side of the waveguide 940) and directed onto the imaging device 920. The out-coupling optical element 952 may be configured to direct the light 926 to exit the waveguide 940 perpendicularly (e.g., normal) to a major surface of the waveguide 940. In some designs, the out-coupling optical element 952 is configured to direct the collimated light 924 onto the imaging device 920 at normal incidence to a light-sensitive portion of the imaging device 920. As discussed above, the camera 920 may be focused at infinity, and, for example, the imaging optics may be configured to focus collimated light onto the detector array.

[0139] Thus, the waveguide 940 may be configured to guide light coupled from the user's eye 210 into the waveguide 940 to be received by the imaging device 920 (e.g., a camera) to capture an image of at least a portion of the user's eye 210. The same waveguide 940 may be configured to guide light coupled from the image projector 930 such that the light from the image projector 930 may be directed to the user's eye 210 such that an image from the image projector 930 is within the user's field of view. In some implementations, the same waveguide is configured to guide light coupled from the illumination source 960 such that the light from the illumination source may be directed to the user's eye 210 and illuminate the eye such that an image of the eye may be captured by the camera 920.

[0140] In some implementations, the same combining optical element 944 can be configured to (i) couple light from the user's eye 210 into the waveguide 940 for receipt by the imaging device 920, and (ii) couple light from the image projector 930 out of the waveguide 940 to the user's eye 210 to project image content into the user's field of view. In some implementations, the same combining optical element 944 can be configured to couple light from the illumination source 960 out of the waveguide to the user's eye 210 such that light from the illumination source may illuminate the eye.

[0141] In other designs, different waveguides may be used and / or different coupling optical elements 944 may be used. In some designs, for example, a first waveguide 940 may be configured to guide light coupled from the user's eye 210 to be received by the camera 920 to capture an image of at least a portion of the user's eye 210, and a second waveguide may be configured to guide light coupled from the image projector 930 to be directed to the user's eye 210. The first and second waveguides may be stacked on top of each other. Another waveguide may additionally or alternatively be configured to guide light coupled from the illumination source 960 such that light from the illumination source may be directed to the user's eye 210 to illuminate the eye.

[0142] Also, in some implementations, the first coupling optical element 944 can be configured to (i) couple light from the user's eye 210 into the waveguide 940 for receipt by the imaging device 920, and (ii) couple light from the image projector 930 out of the waveguide 940 to the user's eye 210 to project the image content into the user's field of view. Another coupling optical element may additionally or alternatively be configured to couple light from the illumination source 960 out of the waveguide to the user's eye 210 such that light from the illumination source may illuminate the eye.

[0143] In some designs, the coupling optical element 944 can include multiple diffractive optical elements (DOEs). For example, a first DOE can be configured to couple light from the user's eye 210 into the waveguide 940 to be received by the imaging device 920. A second DOE can be configured to couple light from the image projector 930 out of the waveguide 940 to the user's eye 210 to project image content into the user's field of view. Optionally, a third DOE can be configured to couple light from the light source 960 out of the waveguide 940 to the user's eye 210 to illuminate the eye. The first and second (and possibly third) DOEs can be stacked, for example, such that in some implementations, light from an environment in front of the user passes through the first DOE, then impinges on the second DOE, then impinges on the third DOE, and then impinges on the user's eye. However, the order can be different.

[0144] In some designs, the first and second DOEs are integrated within a single element or volume of the waveguide 940. In some implementations, for example, the first and second DOEs are both superimposed on top of one another (e.g., occupying the same or nearly the same volume) within the waveguide 2102. For example, the first and second DOEs may be recorded in the same medium.

[0145] As described above, image capture of the eye, e.g., the retina, can facilitate eye tracking. Figure 13A illustrates an imaging system 900 configured to image, e.g., various portions of the eye 210 (e.g., the retina), e.g., at different times when the eye is in different positions. Phases A and B may refer to images of the eye 210 during different eye orientations. Figure 13A shows images of the eye 210 during both phase A and phase B imaging and the results.

[0146] In some implementations, light emission 928 (e.g., from an illumination source 960 as described above, or from one or more illumination sources configured and / or positioned differently) can be used to acquire one or more images of the retina 962, as shown by FIG. 13A. The image of the retina 962 may comprise one or more regions 964, 966 imaged during different orientations of the eye 210. FIG. 13A shows two regions 964, 966 of the image of the retina 962. For example, the region 964 of the retina imaged in stage A may be imaged while the eye 210 is oriented at a normal angle with respect to the waveguide 940. Image data for the region 966 imaged in stage B may be acquired while the eye 210 is oriented at an acute angle with the waveguide 940. A composite image or map of the retina 962 may be acquired using one or more orientations of the eye 210 during one or more stages of imaging. Processing electronics or a processor, such as data module 140 (see FIG. 2), may be used to find the overlap image data between the two nearby regions. Using the image data of the overlap region, a composite image of the retina 962 can be determined. A larger size (e.g., full size) composite image or map of the user's retina can be stored.

[0147] As described herein, a head mounted display can be used to map the retina of a user's eye based on the direction in which the user's eye is pointed. To use eye gaze to provide a realistic and intuitive interaction with objects in the user's environment and / or to identify the wearer of the head mounted display device, the head mounted display system can use retinal mapping to incorporate the uniqueness of the user's eye features and other conditions that may have some effect on the eye measurements. For example, an image may be identified based on the location of blood vessels in the corresponding retinal image.

[0148] Retinal mapping may involve a process for enabling a computing device to learn how to associate a user's eye gaze (e.g., as identified in a retinal image) with a gaze point in 2D or 3D space. An eye gaze may be associated with a single point in 2D or 3D space. An eye gaze can also be associated with multiple points in space, which can account for the movement of a virtual object (e.g., a series of points, a moving image location).

[0149] The head mounted display system can determine the user's eye line of sight based on the retinal image. The head mounted display system can acquire the retinal image using a sensor (e.g., an eye camera, such as imaging device 920). The head mounted display system can image one or both of the user's eyes while the user changes their eye line of sight (e.g., as the user looks around to follow a moving or shifting calibration target or a fixed target). To map the user's retina, the head mounted display system can present a virtual target, e.g., a fixed target, for the user to view. The virtual target may be associated with one or more known points of line of sight in 2D or 3D space. While the user is looking at the target, the head mounted display system can obtain a retinal image and associate the image with a line of sight point. The head mounted display system can calculate and / or generate a mapping matrix based on the association of the individual retinal images with points of line of sight associated with the target.

[0150] The retinal mapping results can reflect the uniqueness within each person's eye. For example, the head mounted display system can generate a mapping matrix that is customized to one or both eyes of a specific individual. For example, a user may have a different amount of eye movement or eye line of sight in response to a specific target. Additionally or alternatively, a user may have a different location, size, shape, and / or orientation of blood vessels within the retina. As a result, by generating calibration results specific to an individual user, the head mounted display system may enable more accurate user interaction with eye line of sight and / or enable identification of a particular user.

[0151] Thus, when a user puts on a head mounted display device, the system can detect whether the user is a previous user or a new user. A confusion matrix can be calculated and scores for a particular eye gaze image stored in the system memory are compared to the corresponding image of the current user. The confusion matrix can include comparison scores for multiple eye gazes and associated retinal images. Based on the comparison scores, the system can make a decision regarding the identity of the user (e.g., whether the user is the same individual with whom the stored retinal image or composite map is associated) and / or a confidence level regarding the decision. The confidence level can include, for example, an identification factor. The stored image, e.g., composite image or map, may be compared to a later acquired image, referred to as an instantaneous or real-time image acquired for the current user. The system may provide an alert or take other action if the system detects that the user is a new user.

[0152] The system may apply filtering, such as digital filtering or image processing, to the retinal images captured by the camera. Such filtering or image processing may enhance features that may be used, for example, for identification, stitching, assembling composite images, eye tracking, etc. Such filtering or image processing may include edge enhancement. Such filters may include, for example, Frangi filters, although other types of filters may be used. Such filters or processing (e.g., edge enhancement or Frangi filters) can be used to enhance and / or detect image features such as blood vessels or tubular structures or fibers in the retinal images.

[0153] FIG. 13B illustrates a pattern of sequentially displayed fixation targets that may be used in the retinal mapping process. These virtual targets, onto which the user's eye would direct its gaze, may redirect the eye's gaze in a variety of different directions while the retina is imaged. The resulting images associated with the different gaze directions correspond to different portions of the retina. As discussed above, when the eye gazes in different directions and views differently located fixation targets on the display, the images captured by the camera include different portions of the retina. These images may be assembled to form a larger map or composite image of the retina.

[0154] FIG. 13B shows a virtual target at 16 different locations within a user's field of view (FOV) 1200. In various implementations, the virtual target will be presented at a given location at a given time. One or more retinal images will be acquired during the time that the virtual target is presented to the user at that particular location. This image or images may be associated with the target location and / or the corresponding gaze direction. More or fewer target locations may be used. In the example shown in FIG. 13B, 16 target locations 1202a-1202p are shown. More or fewer target locations may be used. The target locations may also be different. The order in which the targets are presented at the different locations may vary. For example, the target may move in a raster pattern from left to right of the user's field of view, then reverse from right to left, then left to right again, lowering the target's position in the field of view, with each lateral pass traversing the field of view. However, other patterns and approaches are also possible. Similarly, targets can be rendered at different locations, either the same or differently. For example, the rendered targets can vary in size, shape, color, etc. Targets can be rendered to the user sequentially during the eye tracking calibration process. For example, as discussed above, the head mounted display system may render targets in a serpentine pattern. For example, target 1202a may be followed by 1202b, then 1202c, then 1202d, then 1202h, then 1202g, etc. Other patterns are possible. For example, targets may be displayed in a more random or non-sequential pattern. In some embodiments, a single target is displayed to the user, and the target moves around the user's field of view (e.g., passing through or stopping briefly at positions 1202a-1202p during the target's movement). The head mounted display system can obtain images of the user's retina while the user is looking at these targets.For example, the head mounted display system can acquire a first image when the user is looking at a target at a first location 1202a, acquire a second image when the user is looking at a target at a second location 1202b, acquire a third image when the user is looking at a target at a third location 1202c, and so on. The wearable system can associate the first image with the first location 1202a, associate the second image with the second location 1202b, associate the third image with the third location 1202c, and so on. The adjacent images may be stitched together in a database to create a complete or partial retinal map. For example, two images can be stitched together in proper alignment using a feature or part of a feature (e.g., a blood vessel or part thereof) common to the images. In various implementations, adjacent target locations will produce overlapping images that can be aligned and stitched together. For example, target location 1202a and target location 1202b and target location 1202c may produce overlapping and adjacent retinal images that may be stitched together. Thus, several different retinal images may be acquired using different eye lines of sight to assemble a larger image (e.g., a composite image or map) of the retina.

[0155] As discussed above, eye tracking can be performed using the synthetic retinal image or map. For example, after the target is no longer displayed, the user may move his / her eye gaze as the user looks at a different real object in front of the user and the head mounted display or augmented reality (virtual) image content displayed by the head mounted display. One or more retinal images may be acquired at these times. The term "instantaneous" or "real-time" images may be used herein to describe these images acquired following calibration that may be used for eye tracking (or other purposes such as acquiring biometric data). These "instantaneous" or "real-time" images likely correspond to a portion of the synthetic retinal image or map. The system may be configured to sufficiently match this "instantaneous" or "real-time" retinal image with the synthetic retinal image or portion of the retinal map. Such a match may be based on a feature or portion of a feature (a blood vessel or portion thereof) that is common to both the "instantaneous" or "real-time" retinal image and the synthetic retinal image or portion of the map. Based on the location of the portion of the synthetic retinal image or map to which this "instantaneous" or "real-time" retinal image matches, the gaze direction may be inferred. Different gaze directions will result in retinal images that correspond to different portions of the retinal map. Thus, identifying the location of the "instantaneous" or "real-time" retinal image on the synthetic retinal image or map will provide information regarding the direction of the user's gaze. Eye tracking, e.g., tracking eye movements and changes in eye gaze, may be performed using such or similar methods. As discussed above, edge enhancement, edge detection, or other digital filtering and / or processing may be used to enhance and / or correlate different image features with the synthetic retinal image or retinal map.

[0156] In various implementations, after completion of an initial calibration process in which virtual or fixation targets are displayed (e.g., in multiple locations) to assemble a synthetic retinal image or map, the synthetic retinal image or map may still be refined. For example, as additional retinal images are acquired, the synthetic retinal image or map may be further refined or improved using the additional images. Thus, as additional "instantaneous" or "real-time" retinal images are acquired, for example, for purposes of providing eye tracking, the synthetic retinal image or map may be further refined or improved using "instantaneous" or "real-time". As the user continues to look at various locations in the display (with or without the aid of a calibration target), the retinal synthetic image or map may be further refined using additional images obtained following the initial calibration in which the virtual or fixation targets were displayed. The quality of the synthetic retinal image or map may thus be increased.

[0157] Additional non-limiting examples of how eye tracking may be performed and / or synthetic retinal images or maps may be produced and how retinal images are used are described in U.S. Patent Publication No. 2017 / 0205875, entitled "EYE IMAGE COLLECTION," filed January 17, 2017, the disclosure of which is incorporated herein by reference in its entirety.

[0158] Thus, as discussed above, larger portions of the retina may be recorded and mapped by acquiring retinal images and / or other images of the eye using an imaging system as described herein, and such images may facilitate eye tracking. For example, the image of eye 210 shown in FIG. 13A may be captured when the eye is in an arbitrary position. Processing electronics or processors (such as the same or different as described above as forming a composite image) may then compare the real-time captured image of the user's retina with a stored composite or larger size (e.g., full size) image of the user's retina and track eye movement. A given image of the user's retina captured in real time may show a specific portion of the user's retina. As described above, by comparing such captured image with a stored image of the user's mapping of a larger portion of the user's retina, the system can determine the portion of the user's retina shown in the captured image, and thereby the eye position / orientation that would produce such an image. See, for example, FIG. 13A, which shows two different images of a portion of the retina produced when the eye is in two different positions and / or orientations. Thus, the position and / or orientation of the eye may be determined by capturing different images of the retina and determining the portions of the retina that are visible. Such determination may be performed even if a composite image is not formed; rather, multiple images of the retina for different eye positions / orientations are recorded and stored in a database. As future images of the retina are acquired, they may be compared to images in the database of stored images to determine images in the database that are similar to the recently acquired eye image. Matching the recent images to one or more of the images in the database that have an associated position and / or orientation associated with them may enable the determination of the orientation and / or position of the more recent images. Other approaches to eye tracking may also be used based on images captured using the designs described herein.

[0159] As described herein, retinal images may be employed for other purposes as well. For example, retinal images may be used to verify that a user is the same user for whom a synthetic retinal image or map was acquired. Retinal images acquired while a user is wearing the head mounted display system (e.g., during a calibration process and / or during subsequent use) may be compared to a stored, previously acquired (e.g., created the previous day or when the head mounted display was previously booted up) synthetic retinal image or map. If a recently acquired retinal image does not sufficiently match a portion of the synthetic retinal image or map, it may be concluded that the current user is different from the previous user (e.g., for whom the synthetic virtual image or map was created). Such methods may be used for security purposes, for example, to verify that a current user of a head mounted display device is the owner or typical user of the device. Thus, biometric data acquired via retinal imaging may be used for security purposes.

[0160] Retinal imaging may also be used to collect biometric data to monitor the health of a user. Medically relevant data may be obtained from the retinal images. Such medical data may be useful for monitoring the health of a user.

[0161] Various applications of eye imaging, such as eye tracking, health monitoring, and collecting biometric data for security, are discussed herein in the context of retinal imaging, although imaging of other parts of the user, such as the user's eyes, may also be employed for these and other purposes.

[0162] Although the eyepiece 950 is described above as being capable of being utilized to facilitate ocular imaging, the eyepiece can also be used to image the world in front of the user. FIGS. 14A-14B illustrate an example imaging system 900 that may be used, for example, to image a portion of the environment in front of the user and / or objects within the portion of the environment. The imaging system 900 used may be a similar system to that described with respect to FIGS. 11A-11E and / or 12A-12E, except that light is collected by the eyepiece 950 from the environment in front of the eyepiece and the user. FIG. 14A illustrates light 970 from the environment, for example, reflected from and / or emitted by one or more physical objects 972 in the environment in front of the user and eyepiece 950. As shown, light 970 from the environment may be nearly collimated (e.g., at infinity), for example, because physical objects 972 in the environment may be located a long enough distance from imaging system 900 for the light rays reaching imaging system 900 to be collimated or nearly collimated. In some implementations, imaging system 900 may be configured to image the environment and / or objects in the environment without the use of any optical elements (e.g., lenses, mirrors) having refractive power within imaging system 900.

[0163] The imaging system 900 shown in Figures 14A and 14B is similar to the imaging system described above. The imaging system includes an eyepiece 950 with one or more waveguides 940, including a coupling optical element 944, configured to direct light from an image projector 930 (not shown) into the eye 210 and form an image therein. The one or more waveguides may include a plurality of waveguides (e.g., a stack of waveguides) configured to in-couple / out-couple a plurality of corresponding colors / wavelengths. Each waveguide in the stack of waveguides may be configured to direct light of a particular color (e.g., red, green, blue). For example, the most distal waveguide (e.g., a stack of waveguides) may be configured for visible light (e.g., red, blue, green) such that the waveguides are configured to in-couple and out-couple the same wavelengths of visible light. Additionally or alternatively, a waveguide configured to in-couple and out-couple non-visible (e.g., infrared) light may be located proximal to the eye 210. Such multiple waveguides corresponding to the waveguide 940 may be used in any other implementations described herein. The imaging system 900 may also include an imaging device (e.g., a camera) 920 and an out-coupling optical element 952 configured to redirect light reflected from the eye 210 to be propagated in the waveguide 940 to the camera. In Figures 14A and 14B, the illumination source 960 is omitted because an illumination source may not be required to image the environment in front of the user. However, an illumination source (e.g., the light source 960 described above) may be used in some designs.

[0164] The eyepiece 950, the waveguide 940, the coupling optical element 944, the out-coupling optical element 952, and the camera 920 may be the same or similar as described above. For example, the coupling optical element 944 may be physically engaged with the waveguide 940. For example, the coupling optical element 944 and / or the out-coupling optical element 952 may be disposed in an optical path between the environment in front of the eyepiece 950 and the camera 920 such that light from the environment is coupled into the waveguide 940 via the coupling optical element 944 and coupled out of the waveguide via the out-coupling optical element to be incident on the camera 210 (e.g., to form an image of at least a portion of the environment). The coupling optical element 944 may comprise a number of redirecting features configured to redirect light guided in the waveguide out of the waveguide or redirect light incident on the coupling optical element 944 at an angle into the waveguide to be guided therein by total internal reflection. The out-coupling optical element 952 may comprise a number of redirecting features configured to redirect light (from the environment) that is guided at an angle within the waveguide such that the light is not guided within the waveguide by total internal reflection, but is directed outwardly towards the camera. The coupling optical element 944, the out-coupling optical element 952, and the redirecting features associated with each may physically engage with the waveguide 940. For example, the coupling optical element 944 and / or the out-coupling optical element 952 may comprise one or more holographic or diffractive optical elements (e.g., surface relief gratings) patterned (e.g., etched) within or on the waveguide 940. The coupling optical element 944 and / or the out-coupling optical element 952 may comprise a layer disposed on the waveguide 940 or may be formed within the waveguide 940. For example, a volume holographic or diffractive optical element may be formed by varying the refractive index of a material that constitutes the waveguide or a layer disposed thereon. Thus, the coupling optical element 944 and / or the outcoupling optical element 952 may be disposed within the volume of the waveguide 940 or a layer disposed thereon. Depending on the design, the coupling optical element 944 and / or the outcoupling optical element 952 may be transmissive or reflective and may operate transmissively or reflectively.For example, the coupling optical element 944 and / or the out-coupling optical element 952 may each include a transmissive or reflective diffractive optical element (e.g., a grating) or a holographic optical element that operates transmissively or reflectively, e.g., redirecting light transmitted therethrough or reflected therefrom. The coupling optical element 944 and / or the out-coupling optical element 952 may include a polarizing optical element, such as a polarization-selective redirecting element (e.g., a polarizer). The polarization-selective redirecting element may include one or more polarization gratings, diffractive optical elements, and / or holographic optical elements, and may comprise a liquid crystal structure, such as a liquid crystal polarization grating. In some implementations, the reflective optical element may include a reflector (e.g., a mirror). Other elements, such as the waveguide 940, may be similar to those described above.

[0165] FIG. 14B illustrates the operation of the imaging system 900 shown in FIG. 14A. Light 970 from the environment is coupled into the waveguide 940 by the coupling optical element 944. The coupling optical element 944 may be configured to redirect this collimated light at an angle above the critical angle of the waveguide 940 such that at least a portion of the collimated light is guided towards the camera 920 by total internal reflection within the waveguide. The out-coupling optical element 952 may be configured to receive at least a portion of the light from the environment in front of the user that is coupled into and guided within the waveguide 940 via the coupling optical element 944. The out-coupling optical element 952 may be configured to couple the in-coupled light out of the waveguide 940 to the camera 920 such that an image of the environment may be captured by the camera 920. The image of the environment may be passed to processing electronics (e.g., one or more processors), such as the data module 140 (see FIG. 2). The data module 140 may be configured to recreate a modified image of the environment in an augmented reality context. The processing electronics may communicate with the camera 920 via wired or wireless electronic signals. Additionally or alternatively, the processing electronics may communicate with the camera 920 using one or more remote receivers. The processing electronics may reside remotely (e.g., a cloud computing device, a remote server, etc.).

[0166] The present imaging system 900 may therefore be used to image an environment directly, which may be useful for a variety of reasons. For example, imaging the environment may be used to determine where augmented reality image content should be placed relative to objects in the environment. For example, imaging the environment may provide the location of a table so that a head mounted display may render an image of a person standing next to a table instead of on or in the table. The imaging system 900 described with respect to imaging an environment may also be used to image an eye 210 as described with respect to FIGS. 10, 11A-11E, and / or 12A-12E.

[0167] It may be desirable to image a wide field of an environment using the imaging system 900. FIG. 14C diagrammatically illustrates an imaging system 900 for collecting light from an environment using a refractive optical element or lens, such as a refractive optical element 980 (e.g., a wide-angle lens) in front of an eyepiece. The refractive optical element 980 may have a positive refractive power. The refractive optical element 980 (e.g., a positive lens) focuses the collimated light 970 from the environment toward the waveguide 940. Lenses of other types than the lens shown in FIG. 14C may be employed. The transmitted light (not shown) may pass through a refractive optical element or lens, such as a refractive optical element 990 (e.g., a negative lens), configured for an equal but opposite negative refractive power of the refractive optical element 980. The negative lens 990 may have a similar or identical refractive power as the positive lens 980, offsetting or canceling the refractive power of the positive lens or a portion thereof. In this way, light from the environment (e.g., distal to the waveguide 940) may pass through the negative lens 990, the eyepiece 950, and the positive lens 980 with substantially no net change in the refractive power introduced to the eye by these two lenses. The negative lens 990 may be configured to offset or counteract the refractive power of the positive lens 980 such that the user would not be subject to the refractive power of the positive lens when viewing the environment in front of the eyepiece 950. The negative lens 990 would also counteract the effect of the positive lens 980, inverting the image of objects in the environment in front of the wearer. Some light 970 from the environment may be incoupling into the waveguide 940 by the coupling optical element 944, despite some of the light rays converging. The incoupling light incident on the outcoupling optical element 952 may be ejected out of the waveguide 940.

[0168] Implementations (e.g., those described by FIGS. 14A-14C) may be used outside of an augmented reality context. For example, an imaging system 900 configured to image an environment is intended to be implemented in a wearable device, such as, for example, glasses (including non-refractive glasses) or bifocal glasses. Such an imaging system 900 may not require an image projector 930 and / or a light source 960. Additionally or alternatively, such an imaging system 900 may not require an internal coupling optical element configured for a corresponding image projector 930 and / or light source 960.

[0169] It may be advantageous to implement such an imaging system 900 to image an environment onto a viewing screen (e.g., television screen, computer screen) of a handheld device (e.g., cell phone, tablet) or the like. The imaging system 900 may improve video chat capabilities. For example, a viewer who sees a chat partner may look at the screen and appear to be looking directly at the viewer. This may be possible because the light rays captured by the imaging system 900 will be captured within the same area the user is looking at (as opposed to viewing a screen, which has a light ray captured by a separate outward-facing camera positioned in a different location, for example).

[0170] In implementations in which the imaging system 900 of FIG. 14C is also used to image the eye 210, the light source 960 and / or image projector 930 may be configured to inject light into the waveguide 940. A positive power type refractive optical element may be disposed between the light source 960 and / or image projector 930 and the waveguide 940, since the light reflected from the eye that is incoupled into the waveguide will pass through a refractive optical element 990 (e.g., a negative lens). The positive lens may be configured to offset or cancel any refractive power provided by the refractive optical element 990 before the incoupled light from the light source and / or light projector is incident on the eye 210. Lenses of types other than those shown in FIG. 14C may also be used as the optical element 990. Alternatively, or in addition, processing electronics in communication with the light source and / or image projector can be configured to modify the image sufficiently to present an undistorted image to a user after the light passes through the refractive optical element 990. The corresponding in-coupling optical element, out-coupling optical element, and / or combining optical element may be configured to act on uncollimated light (e.g., diverging, converging light) in some designs.

[0171] In various implementations, the same waveguide 940 may be used to (i) deliver light from the eyepiece 950 and the environment in front of the user to the camera 940, and (ii) deliver light from the image projector 930 to the eye 210 and form image content therein. Using the same waveguide 940 may simplify the system and / or eyepiece, making it more compact and potentially providing a reduced form factor. Reducing the thickness of the eyepiece 950 by reducing the number of waveguides 940 may be advantageous for other reasons as well. Lower cost and a more simplified manufacturing process may be some such advantages.

[0172] Also, in various designs, the same or different imaging systems may be used within the same head mounted display to image the eye, for example, by transmitting light from the eye through a waveguide in the eyepiece 950 to the camera 940, as described above. Such systems may also use the eyepiece to transfer light from an illumination source to the eye 210 to illuminate the eye. In some designs, the eyepiece may additionally be used to transmit light from the image projector 930 to the eye 210 and form image content therein. Using the eyepiece to image the environment and assist in imaging the eye (and potentially illuminating the eye) may simplify the system and / or make the system more compact, potentially providing a reduced form factor.

[0173] Further, in some implementations, the same waveguide 940 may be used to (i) transmit light from the environment in front of the eyepiece 950 to the camera 940 and (ii) transmit light from the eye 210 to the camera to capture an image of the eye. The same waveguide may be used to transmit light from the image projector 930 to the eye 210 to form image content therein and / or transmit light from the illumination source 960 to the eye 210 to illuminate the eye for image capture. Using the same waveguide 940 may simplify the system and / or eyepiece, making the system and / or eyepiece more compact and potentially providing a reduced form factor. Reducing the thickness of the eyepiece 950 by reducing the number of waveguides 940 may be advantageous for other reasons as well. Lower cost and more simplified manufacturing processes may be some such advantages.

[0174] Similarly, in addition to coupling light from the environment into the waveguide 940, the same coupling optical element 944 may be configured to direct light from the image projector 930 to the eye 210 to form image content therein and / or to direct light from the eye into the waveguide 940 to be directed therein to the camera 920. Additionally or alternatively, the same coupling optical element 944 may be configured to couple light from the illumination source 960 that is guided within the waveguide 940 out of the waveguide to the user's eye 210.

[0175] As discussed above, one or more of the coupling optical element 944, the in-coupling optical element 942, or the out-coupling optical element 952 may comprise a polarization-selective coupling element. Thus, in various designs, light input into the eyepiece 950 or the waveguide 940 is polarized so as to be appropriately acted upon by the polarization-selective redirecting element.

[0176] Thus, in some embodiments, illumination source 960 comprises a polarized source of a suitable polarization to be appropriately acted upon by the polarization selective combining / redirecting element.

[0177] One or more polarization-specific optical filters and polarization-modifying elements may be included in various imaging systems 900, such as those in which the image projector 930 and / or the light source 960 are positioned directly opposite one another through the waveguide 940. The polarizing light-sensitive elements may be useful in reducing directional light emission into the imaging device 920 and / or reducing saturation of the imaging device 920, for example, in configurations in which these elements are aligned on opposite sides of the waveguide 940 at the same lateral position. FIGS. 15A-15B illustrate such configurations. The light source 960 as shown in FIG. 15A can be configured to direct light through a polarization-specific optical filter 982, such as a polarizer (e.g., a linear polarizer), and / or through a polarization-modifying element 986 configured to alter the polarization state of the incident light, such as a polarization rotator. A retarder, such as a half-wave retarder, may, for example, rotate linear polarization. Thus, a suitably oriented half-wave retarder or half-wave plate can rotate s-polarized light to p-polarized light or vice versa. Thus, in various implementations, a polarization-specific optical filter 982 and / or a polarization modifying element 986 is positioned in the optical path between the light source 960 and the in-coupling optical element 942 to provide the in-coupling optical element with a suitably oriented polarization. In some implementations, the imaging system 900 does not include a polarization modifying element, but does include a suitably oriented polarizing optical filter, such as a polarizer.

[0178] Light emitted by light source 960 may pass through an array of optical elements in a particular order. For example, as shown in FIG. 15A, light may first pass from light source 960 through a polarization-specific optical filter 982 (e.g., a polarizer) and then through a polarization-modifying element 986 (e.g., a rotator). After the light passes through the polarization-modifying element 986, it may be incident on an internal coupling optical element 942, which may direct the light into the waveguide 940 to be guided therein.

[0179] For example, the light source 960 may be configured to emit light of mixed polarization (e.g., s-polarized and p-polarized). The polarization specific optical filter 982 may be configured to transmit only light of a first polarization state (e.g., p-polarized). As the light continues, the polarization modifying element 986 may be configured to change the polarization state of the light (e.g., from p-polarized to s-polarized). The internal coupling optical element may be configured to redirect the s-polarized light to an angle above the critical angle of the waveguide such that the s-polarized light is guided within the waveguide. The internally coupled light 904 may be substantially polarized in a second polarization (s-polarized) as it propagates through the waveguide 940. The coupling optical element 944 may be configured to redirect only light of the second polarization state (s-polarized). The coupling optical element 944 may be configured to couple the internally coupled light 904 out of the waveguide 940 and into the eye 210 to provide illumination for image capture.

[0180] To prevent direct illumination (e.g., saturation) of the imaging device 920, a polarization modifying element 958 and / or a polarization specific optical filter 984 may be disposed in or on the waveguide 940 such that only light of a certain polarization state (e.g., p-polarized) may pass through the polarization specific optical filter 984 to the imaging device 920. The polarization modifying element 958 (e.g., a half-wave plate) may be configured to change the state of polarization (e.g., from s-polarized to p-polarized). The polarization specific optical filter 984 may be configured to transmit only light of a certain polarization (e.g., p-polarized) therethrough. In this way, light passing through the polarization specific optical filter 982 will not be configured to transmit directly through the polarization specific optical filter 984. In any of the above implementations (e.g., image projector 930 and / or light source 960 are on the same optical axis, as shown in FIG. 15A ), as in FIGS. 10 , 11A-11E, and 12A-12E, the configuration of polarization-specific optical filter 982, polarization-modifying element 986, incoupling optical element 942, polarization-modifying element 958, and / or polarization-specific optical filter 984 may be implemented according to the design of FIG. 15A . Polarization-specific optical filter 984 may be a transmissive-reflective polarizer (e.g., a polarizer beamsplitter) configured to transmit light of a first polarization and redirect or reflect light of a second polarization different from the first.

[0181] A partially reflective element (e.g., a semi-transparent mirror) may be included to redirect the incoupled light 904 to the imaging device 920. The partially reflective element may be disposed between the incoupling optical element 942 and the polarization modifying element 986 such that a portion of the incoupling light 914 is reflected toward the imaging device 920 while reducing leakage of the incoupling light 914 out of the waveguide 940. The fraction of light that does not leak out may be any percentage between 0 and 1. For example, the fraction may be 0.90, such that 90% of the light propagating through the waveguide 940 along the coupling optical element 944 is maintained within the waveguide 940 at each reflection of the light. Other fractions are also possible (e.g., 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or any value in the range between these values).

[0182] 15B illustrates the propagation of light reflected or scattered from the retina. A portion of the light 910 reflected from the retina, having a second polarization (s-polarization), incident on the coupling optical element 944, may be redirected by the coupling optical element 944 at an angle above the critical angle of the waveguide 940 and thus guided therein. A portion of the light may not be coupled into the waveguide 940 and will transmit therethrough as non-incoupled light 912. The incoupled light 904 may propagate through the waveguide 940 towards the camera.

[0183] Other implementations may also benefit from the use of polarization selective elements proximate the light source and the camera. For example, various systems can be configured to provide illumination with a first polarization and capture images with a camera using light with a different polarization. For example, such configurations may be used to reduce undesired reflections from the cornea, etc., when imaging the retina. Reflections from the cornea will be specular. Thus, if light of a first polarization is incident on the cornea, the light reflected from the cornea will retain its first polarization. In contrast, the retina is diffuse. If light of a first polarization is incident on the retina, the light reflected from the retina will not retain only the first polarization. Diffuse reflections are more likely to result in unpolarized light. Thus, a second polarization different from the first polarization will be present in the reflected light. Similarly, by illuminating with a first polarization and imaging with a second, different polarization, the retina can be imaged with reduced glare from the cornea.

[0184] Thus, in various implementations, the polarization-specific optical filters 982, 984 may both be used to reduce unwanted light reflected from the eye 210 (e.g., from the cornea). For example, unwanted light, glare, or glints may be reflected from the cornea that may saturate the image captured by the imaging device 920. Light reflected from the cornea may be specular and maintain its polarization. In contrast, light reflected from the retina may be more diffusely reflected and less homogeneously polarized. Similarly, a combination of polarizers may be used to remove some or most of the unwanted reflected light. Initially, polarized light may be used to illuminate the eye 210. In some designs, a polarized illumination source (e.g., light source 960) may be used. Additionally or alternatively, a first polarizer (e.g., polarization-specific optical filter 982) may be positioned at the beginning of the optical path of the illumination source to provide an initial polarization of the light. A second polarizer (e.g., a polarization-specific optical filter 984) may be positioned in the optical path before the light enters the imaging device 920. The second polarizer may be 90 degrees from the first polarizer. oThe polarizers 982, 984 may be rotated (e.g., the polarizers 982, 984 may be "crossed"). As a result, the eye is illuminated with a first polarization, and some light of the first polarization will be reflected from the cornea. This light will not pass through the camera-proximal polarizer 984. However, the light reflected from the retina will include the second polarization. Similarly, light diffusely reflected from the retina will pass through the camera-proximal polarizer 984, allowing an image of the retina to be captured by the camera. Thus, in such a configuration, undesired light received from the eye (e.g., from the cornea) that may enter the imaging device 920 may be reduced or eliminated. Other configurations are also possible. For example, the polarization-selective in-coupling optical element 942 for coupling light from the light source 960 into the waveguide 940 and the polarization-selective out-coupling optical element for coupling light out of the waveguide to the camera 920 may be employed to have different polarization selectivity properties. For example, a polarization-selective in-coupling optical element may selectively redirect light from an illumination source having a first polarization into a waveguide, while an out-coupling optical element may selectively redirect light of a second, different polarization out of the waveguide to a camera. The effect may again be to reduce or eliminate undesired light received from the eye (e.g., from the cornea) before entering the imaging device 920.

[0185] Various imaging systems 900 are discussed herein that can collect light and image the retina using an eyepiece 950. However, the imaging system 900 can also be configured to image other portions of the eye, such as the anterior portion of the eye. FIG. 16 illustrates how the imaging system 900 can be used to image the anterior portion of the eye 210 (e.g., the cornea). The imaging system 900 may include one or more elements of the exemplary imaging system 900 described above. In addition, the exemplary imaging system 900 may include one or more refractive power type optical elements or lenses, such as refractive power type refractive optical elements 980, 990, having refractive power. For example, a positive refractive power lens or positive lens 980 may be positioned on the proximal side of the eyepiece 950 (e.g., closer to the eye 210) between the eye 210 and the eyepiece. A negative refractive power lens or negative lens 990 may be disposed on the distal side of the eyepiece 950 between the eyepiece and the environment in front of the user. One or both of the lenses 980, 990 may be a variable focus element (e.g., a variable focus lens) and / or may include a liquid crystal element. In some designs, one or both of the lenses 980, 990 include a Fresnel lens. The lenses 980, 990 may incorporate liquid crystals to produce Fresnel lens functionality. Such functionality may enable variable focus of one or both of the lenses 980, 990. In some designs, one or more of the lenses 980, 990 may be integrated with and / or manufactured on or in (e.g., formed in) the eyepiece 950.

[0186] In various embodiments, the coupling optical element 944 is configured to redirect collimated light reflected from the eye 210 into the light guide to be guided therein. The positive lens 980 may thus be configured to collimate light reflected from the eye 210, such as the anterior portion of the eye (e.g., the cornea). The positive lens 980 may thus have a focal length equal or substantially equal to the distance of the lens to the portion of the eye 210 to be imaged, e.g., the cornea.

[0187] The negative lens 990 may have a similar or identical refractive power as the positive lens 980, offsetting or canceling the refractive power of the positive lens. In this way, light from the environment (e.g., distal to the waveguide 940) may pass through the negative lens 990, the eyepiece 950, and the positive lens 980, with substantially no net change in the refractive power introduced by these two lenses. Thus, the negative lens 990 may be configured to offset or cancel the refractive power of the positive lens 980, such that the user would not be subject to the refractive power of the positive lens when viewing the environment in front of the eyepiece 950. The negative lens 990 would also counteract the effect of the positive lens 980, inverting the image of objects in the environment in front of the wearer.

[0188] FIG. 16 illustrates light 928 incident on and scattered from the cornea. The imaging system 900 may be configured to capture this light 988 reflected from the cornea. For example, a positive lens 980 may collect a portion of the light 988 scattered from the cornea and collimate the light 988. The light 988 collimated by the positive lens 980 is incident on a coupling optical element 944, which is configured to redirect the collimated light into a waveguide 940 at an angle greater than the critical angle of the waveguide, such that the light is guided therein by TIR. The coupling optical element 944, the out-coupling optical element 952, and / or the waveguide 940 may be as described above. The resulting out-coupled light 906 may be directed by the out-coupling optical element 952 out of the waveguide 940 to a camera (not shown).

[0189] FIG. 16 illustrates light 928, such as collimated light, that may result from an eyepiece 950 as described above. An illumination source 960 may couple light into a waveguide 940, and a coupling element 944 may couple this light from the illumination source 960 out of the waveguide. The coupling element 944 may be configured to couple the light out of the waveguide 940 as collimated light. This light illuminates and scatters from an anterior portion of the eye (e.g., the cornea). As discussed above, this scattered light 988 may be collected by a positive lens 980 and imaging system 900 to form an image of an anterior portion of the eye 210. Also, as discussed above, this illumination 928 directed onto the eye 210 may be non-visible (e.g., infrared) light.

[0190] FIG. 16 also illustrates alternative arrangements for illuminating the eye 210. In some designs, one or more light sources 934, such as LEDs or emitters, may be positioned relative to the eye 210 and guided by TIR through a waveguide 940 to direct light thereon, rather than directed onto the eye 210. In some implementations, the eyepiece 950 or the waveguide 940 are not in the optical path between the one or more light sources 934 and the eye 210. In some designs, multiple such light sources 934 may be arranged in a pattern (e.g., a circular or ring-like pattern) near and / or around the eye. In some designs, the pattern of light sources 934 may define an illumination axis parallel (e.g., coaxial) with the optical axis of one or more lenses 980, 990. The one or more light sources 934 may be similar to the one or more light sources 960 described above, and may be, for example, pulsed. Similarly, the one or more light sources 934 may comprise an infrared light source, such as an infrared LED, or another type of invisible light. Alternatively, the one or more light sources may comprise a visible light source that emits visible light, or the one or more light sources may emit both visible and invisible (e.g., infrared) light.

[0191] FIG. 17 illustrates another exemplary imaging system 900 configured to image a portion of the eye 210, such as the anterior portion of the eye (e.g., the cornea). The imaging system 900 shown in FIG. 17 employs a reflective optical element 996 configured to collimate light from the eye, in contrast to the transmissive optical element (lens) 980 shown in FIG. 16. A reflective optical element will have less aberrations than a transmissive optical element, since chromatic aberrations are generally not applicable to reflective optical elements, such as the reflector 996 shown in FIG. 17. Thus, by using a reflective surface in collecting light from the eye 210, less (e.g., chromatic) aberrations are introduced into the captured image of the eye.

[0192] 17 illustrates an imaging system 900 that includes a curved transmissive optical element 996, for example, having a wavelength-dependent reflective coating 998. The curved transmissive optical element 996 may be disposed distal to the waveguide 940 (on the environment side of the eyepiece 950). Thus, the curved transmissive optical element 996 may be disposed between the environment in front of the wearer and the waveguide 940 and / or the coupling optical element 944. Similarly, the waveguide 940 and / or the coupling optical element 944 may be disposed between the curved transmissive optical element 996 and the eye 210.

[0193] The wavelength-dependent reflective coating 998 may be configured to reflect light of a certain wavelength or range of wavelengths. In some implementations, for example, the wavelength-dependent reflective coating 998 may be configured to reflect invisible light (e.g., infrared light) within a certain range of wavelengths, while the wavelength-dependent reflective coating 998 may be configured to transmit visible light. The wavelength-dependent reflective coating 998 may, in some cases, be disposed on a surface of the curved transmissive optical element 996.

[0194] As discussed above, in various designs, the coupling optical element 944 is configured to redirect collimated light reflected from the eye 210 into the waveguide 940 to be guided therein. Thus, the reflective optical element 996 may be configured to collimate light reflected from the eye 210, such as an anterior portion of the eye (e.g., the cornea). The curved reflective optical element 996 may thus have a positive refractive power with respect to light incident on its proximal side that is reflected from the wavelength-dependent reflective coating 998. Notably, in various designs, the reflective optical element 994 may have a focal length equal or substantially equal to the distance from the reflective optical element 996 to the portion of the eye 210 to be imaged, e.g., the cornea, the iris, etc. Exemplary values ​​of the focal length may be, for example, 2 cm to 8 cm. In some implementations, the focal length is 4 cm to 6 cm. In some designs, the focal length is about 5 cm. The focal length may be within any range formed by any of these values, or may be outside such ranges in different designs.

[0195] In various implementations, the reflective optical element 996 is positioned on the distal side of the eyepiece 950 in front of the eyepiece. Thus, the reflective optical element 996 is positioned between the eyepiece 950 and the environment in front of the user. Similarly, the eyepiece 950 is positioned between the reflective optical element 996 and the eye 210.

[0196] The curved transmissive optical element 996 may have a curved reflective surface with any shape of curvature. In some implementations, the surface is rotationally symmetric. In some implementations, the surface may be spherical or aspheric (e.g., parabolic). Non-rotationally symmetric shapes are also possible. However, in various designs, the reflective surface has positive optical power. The reflective optical element 996 may, for example, comprise a concave mirror, at least for certain wavelengths and / or polarizations.

[0197] The curved transmissive optical element 996 may be configured to have negligible optical power in transmission. Similarly, the curved transmissive optical element 996 may be configured to transmit light without introducing convergence or divergence. In one example, the curved transmissive optical element 996 may have a curvature of an inner radius that is substantially the same as the curvature of an outer radius. A thin optical element 996 may, for example, reduce optical aberrations, may be lighter, and / or may be more compact with respect to light transmitted therethrough.

[0198] In various designs, the reflective optical element 996 includes a material that is transparent to visible light so that the user can see the environment in front of the wearer. In some cases, to improve transmission, the curved transmissive optical element 996 may be coated with an anti-reflective coating on an outer surface (e.g., a distal surface). The anti-reflective coating may be configured to reduce reflection of visible light, such as, for example, red, green, and / or blue light. However, the reflective optical element 996 may be configured to reflect a portion of the light scattered from the eye 210 to form an image of the eye. Thus, the reflective optical element 996 may act differently on different light. For example, the reflective optical element 996 may act differently on different wavelengths. The reflective optical element 996 may be configured to reflect infrared light and transmit visible light.

[0199] As discussed above, the one or more light sources 934 may be configured to illuminate the eye 210 with infrared light. The resulting light 988 reflected from the eye 210 (e.g., cornea) may diverge, as illustrated diagrammatically in FIG. 17. A curved transmissive optical element 996 may be positioned to receive this light 988 reflected from the eye (e.g., cornea, iris). A wavelength-dependent reflective coating 998 may be configured to reflect the light 988 reflected from the eye because the wavelength illumination used to illuminate the eye is the same wavelength (e.g., 850 nm) reflected by the reflective coating on the curved transmissive optical element 996. For example, the eye may be illuminated with infrared light (e.g., 850 nm) and the curved transmissive optical element 996 may be configured to reflect the infrared light (e.g., 850 nm) and pass visible light. The shape of the curved transmissive optical element 996 may also be configured to collimate the light 988 reflected from the eye and reflect the light to the coupling optical element 944, which redirects the collimated light into the waveguide 940 to be guided therein by TIR.

[0200] In FIG. 17, one or more light sources 934, such as LEDs or emitters, may be positioned relative to the eye 210 and guided by TIR through a waveguide 940 to direct light thereon, rather than being directed onto the eye 210, as in some other designs. In some implementations, the eyepiece 950 or the waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210. In some designs, multiple such light sources 934 may be arranged in a pattern (e.g., a circular or ring-like pattern) near and / or around the eye. In some designs, the pattern of light sources 934 may define an illumination axis parallel (e.g., coaxial) with the optical axis of one or more lenses 980, 990. The one or more light sources 934 may be similar to the one or more light sources 960 described above, and may be pulsed, for example. Similarly, the one or more light sources 934 may comprise an infrared light source, such as an infrared LED, or another type of invisible light. However, other types of light sources may also be used.

[0201] FIG. 18 illustrates another exemplary imaging system 900 configured to image a portion of the eye 210, such as an anterior portion of the eye (e.g., the cornea). In the implementation shown in FIG. 18, polarization selectivity is employed to help control the path of light reflected from the eye. In particular, in various designs, the combining optical element 944 is polarization selective. For example, light having a first polarization is transmitted through the combining optical element 944, while light of a second, different polarization is redirected by the combining optical element 944 into the waveguide 940 to be combined therein by TIR. Thus, in various implementations, the eye 210 is illuminated with polarized light, or a polarizer (not shown) is placed between the eye and the waveguide 940 such that light from the eye incident on the waveguide is polarized. For example, the emitter 934 may emit polarized light, or a polarizer may be placed in front of the emitter 934 such that the eye 210 is illuminated with polarized light. Thus, in various designs, the polarization of the polarized light received by the optical combining element 944, incident on the eye 210 and / or reflected therefrom may be a first polarization such that the light is directed to the reflector 996.

[0202] Similarly, in various implementations, the coupling optical element 944 (and / or the out-coupling optical element 952) are each configured to transmit light of a first polarization state, such as a first linear, circular, or elliptical polarization state (e.g., p-polarized, left-handed, or elliptical polarization, etc.) and redirect light of a second polarization state, such as a second linear, circular, or elliptical polarization state (e.g., s-polarized, right-handed, or elliptical polarization, etc.) into and / or out of the waveguide. In some implementations, the eye illuminator 934 may further include a polarization modifying element (e.g., a polarizer) that may emit only or primarily the first polarization (e.g., p-polarized) or that is configured to transmit only light of the first polarization state (e.g., p-polarized). In addition, the coupling optical element 944 and / or the out-coupling optical element 952 may each be configured to redirect light of a second polarization (e.g., s-polarized) into and / or out of the waveguide.

[0203] Similar to the imaging system 900 shown in FIG. 17, the curved reflector 998 shown in FIG. 17 comprises a curved transmissive optical element 996 having a wavelength-dependent reflective coating 998. The wavelength-dependent reflective coating 998 may be configured to reflect light of a certain wavelength or range of wavelengths. In some implementations, for example, the wavelength-dependent reflective coating 998 may be configured to reflect invisible light (e.g., infrared light) within a certain range of wavelengths, while the wavelength-dependent reflective coating 998 may be configured to transmit visible light. The wavelength-dependent reflective coating 998 may be disposed on a surface of the curved transmissive optical element 996 in some cases.

[0204] In various implementations, the curved transmissive optical element 996 is positioned on the distal side of the eyepiece 950 in front of the eyepiece. Thus, the reflective optical element 996 is positioned between the eyepiece 950 and the environment in front of the user. Similarly, the eyepiece 950 is positioned between the reflective optical element 996 and the eye 210.

[0205] Thus, light having a first polarization (e.g., p-polarized) from the eye 210 is incident on the combining optical element 944 and passes therethrough to the curved transmissive optical element 996. The imaging system 900 further includes a polarization modifying optical element 978, such as a retarder (e.g., a quarter-wave retarder). The retarder 978 is transmissive and imparts a quarter-wave phase difference on the light transmitted therethrough. The light is incident on the curved transmissive optical element 996 and is reflected therefrom. The wavelength-dependent reflective coating 998 may be configured to reflect certain wavelengths of light reflected from the eye. As a result, the light is reflected from the curved surface of the curved transmissive optical element 996 and collimated. The collimated light again passes through the retarder 978, thereby imparting another quarter-wave phase difference on the light transmitted therethrough. The phase difference (e.g., full-wave phase difference) introduced in these two passes through the retarder rotates the polarization. Thus, the first polarization (e.g., p-polarized) transmitted through the polarization-selective combining optical element 944 in the first pass is converted to the second polarization (s-polarized) and redirected into the waveguide 940 to be guided by TIR to the camera 920. As discussed above, in various designs, the combining optical element 944 is configured to redirect collimated light reflected from the eye 210 into the waveguide 940 to be guided therein. Thus, the reflective optical element 996 may be configured to collimate light reflected from the eye 210, such as the anterior portion of the eye (e.g., the cornea). The curved reflective optical element 996 may thus have a positive refractive power. In particular, in various designs, the reflective optical element 994 may have a focal length that is equal or substantially equal to the distance from the reflective optical element 996 to the portion of the eye 210 to be imaged, such as the cornea, iris, etc. Exemplary values ​​for the focal length may be, for example, 2 cm to 8 cm. In some implementations, the focal length is 4 cm to 6 cm. In some designs, the focal length is about 5 cm.

[0206] In various designs, the reflective optical element 996 may comprise a curved surface configured to reflect light. The curved surface may be spherical or rotationally symmetric in some cases. The reflective optical element 996 may comprise, for example, at least for certain wavelengths and / or polarizations, a concave mirror.

[0207] In various designs, the reflective optical element 996 includes a material that is transparent to visible light so that the user can see the environment in front of the wearer. The wavelength-dependent reflective coating 998 disposed on the surface of the curved transmissive optical element 996 may therefore be transparent to visible light or at least some wavelengths of visible light. The curved transmissive optical element 996 may also be coated with an anti-reflective coating on the outer surface (e.g., the distal surface). The anti-reflective coating may be configured to reduce reflection of red, green, and / or blue light. However, the reflective optical element 994 may be configured to reflect a portion of the light scattered from the eye 210 to form an image of the eye. Thus, the reflective optical element 996 may act differently on different light. For example, the reflective optical element 996 may act differently on light of different polarization states (and / or wavelengths). The reflective optical element 996 may be configured to transmit visible light and reflect infrared light.

[0208] As shown in FIG. 17, one or more light sources 934, such as LEDs or emitters in FIG. 18, may be positioned relative to the eye 210 and guided through a waveguide 940 by TIR to direct light onto the eye 210, rather than being directed onto it. Thus, in some implementations, the eyepiece 950 or the waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210. In some designs, multiple such light sources 934 may be arranged in a pattern (e.g., a circular or ring-like pattern) near and / or around the eye. The one or more light sources 934 may be similar to the one or more light sources 960 described above and may be pulsed, for example. Similarly, the one or more light sources 934 may comprise an infrared light source, such as an infrared LED, or another type of invisible light. Notably, in various implementations, the light source 934 may emit light that is reflected by a wavelength-dependent reflective coating 998 and / or a curved transmissive optical element 996. However, other types of light sources may also be used.

[0209] The polarization selective combining optical element 944 is configured to be polarization selective depending on the type of linearly polarized light incident thereon, although other polarization selective combining optical elements may be polarization selective for other types of polarization states, such as different types of circular or elliptical polarizations. The polarization selective combining optical element 944 may be configured such that, for example, a first polarization, such as a first circular or elliptical polarization (e.g., left-handed or LHP-polarized light), is transmitted through the polarization selective combining optical element 944 and a second polarization, such as a second circular or elliptical polarization (e.g., right-handed or RHP-polarized light), is redirected into the light guide, or vice versa. Such polarization selective combining optical elements 944 may comprise liquid crystals, such as cholesteric liquid crystals. Examples of some liquid crystal optical elements are discussed below in the section entitled "Cholesteric Liquid Crystal Mirrors," U.S. Patent Publication No. 2018 / 0164627, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL," filed December 7, 2017, U.S. Patent Publication No. 2018 / 0239147, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR," filed February 22, 2018, and U.S. Patent Publication No. 2018 / 0239177, entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION," filed February 22, 2018 (each of which is incorporated by reference herein in its entirety and for all purposes).

[0210] A polarization modifying element or retarder, such as a circular polarizer, may be placed between the eye and the polarization selective combining optical element 944 to convert light reflected from the eye to a first polarization (e.g., LHP). The LHP light will pass through the polarization selective combining optical element 944, reflect off the reflector 998, change polarization to RHP, and be redirected by the polarization selective combining optical element 944 into the waveguide to the camera.

[0211] In some implementations, the reflector 996 may be polarization selective in its reflectivity such that only light of one polarization state is reflected and / or light of a different polarization state is transmitted. Such optical elements may comprise liquid crystals, such as cholesteric liquid crystals. Examples of such optical elements are described below in the section entitled "Cholesteric Liquid Crystal Mirrors," in U.S. Patent Publication No. 2018 / 0164627, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL" filed on December 7, 2017, and in U.S. Patent Publication No. 2018 / 0239147, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR" filed on February 22, 2018, and in U.S. Patent Publication No. 2018 / 0239147, entitled "VARIABLE-FOCUS DIS ... Such optical elements may reflect light of a first polarization state, such as a first circular or elliptical polarization state (e.g., left-handed circular or elliptical polarization), and transmit light of a second polarization state, such as a second circular or elliptical polarization state (e.g., right-handed circular or elliptical polarization), or vice versa. In some embodiments, liquid crystals are disposed on the curved surface of the reflector 996 such that, in reflection, the reflector has a refractive power, such as a positive refractive power. In various other implementations, the liquid crystal optical element may be flat or planar. For example, the liquid crystals may be disposed on a flat or planar substrate or layer. Despite being flat, refractive power may be included within the liquid crystal optical element. Such an element may be referred to as a cholesteric liquid crystal reflective lens. Thus, light from the eye may be collimated and reflected to the coupling optical element 998. The reflector may, for example, reflect light of a first polarization state (e.g., left-handed or elliptically polarized) and transmit light of a second polarization (e.g., right-handed or elliptically polarized). Thus, the eye 210 is illuminated with left-handed circularly polarized light, or the light reflected from the eye is transmitted through a polarizer (e.g., a circular or elliptically polarizer) that transmits light with a first polarization (e.g., left-handed or elliptically polarized). The coupling optical element 944 may also be polarization selective, transmitting the LHP light and redirecting the RHP light into the waveguide. The LHP light from the eye passes through the coupling optical element 944. This transmitted LHP light also impinges on and is reflected from the wavelength-selective liquid crystal reflector 996. In one design, the wavelength-selective liquid crystal reflector 996 converts the first polarization state (e.g., LHP) to a second polarization state (e.g., RHP) upon reflection. This second polarization state light (e.g., RHP light) is directed to a combining optical element 944, which redirects the second polarization state light (RHP) into the waveguide 940 to the camera 920.

[0212] In some designs, the combining optical element 944 does not comprise a liquid crystal grating, but instead comprises, for example, a surface relief diffraction grating or a holographic grating. As discussed above, those combining optical elements 944 that do not comprise cholesteric liquid crystals may also comprise volume diffractive or holographic optical elements or gratings.

[0213] Thus, light scattered from the eye is reflected back to the waveguide 940 by the reflective optical element 996 for coupling into the waveguide by the coupling element 944. In contrast, however, a portion of the unpolarized light from the environment in front of the wearer, corresponding to a second polarization state (e.g., RHP), will be transmitted through the reflective optical element 996. Thus, the wearer may see objects through the reflective optical element 996.

[0214] However, in various designs, the reflective optical element 996 will have negligible optical power in transmission. For example, the reflective optical element 996 may have curved surfaces with the same curvature on both sides of the optical element, such that the total optical power of the optical element for light transmitted therethrough will be negligible.

[0215] As discussed above, in various implementations, the reflective optical element 996 may be a reflective optical element based on a reflective filter, as described below in the section entitled "Cholesteric Liquid Crystal Mirror," U.S. Patent Publication No. 2018 / 0164627, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL" and filed on December 7, 2017, and U.S. Patent Publication No. 2018 / 0239147, entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR" and filed on February 22, 2018, and entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION SYSTEMS." The optical element includes a cholesteric liquid crystal reflective lens, which is a cholesteric liquid crystal reflective element as discussed in U.S. Patent Publication No. 2018 / 0239177, entitled "CORRECTED EYE CONVERSION" and filed on February 22, 2018 (each of which is incorporated by reference in its entirety and for all purposes). Such optical elements may act on a specific wavelength or range of wavelengths. Thus, light such as infrared light reflected from the eye may be acted on by the cholesteric liquid crystal reflective element. However, light that is not within that wavelength range, such as visible light from the environment, may pass through the cholesteric liquid crystal reflective element without being acted on by the cholesteric liquid crystal reflective element. Thus, the cholesteric liquid crystal reflective element may have negligible refractive power for this visible light from the environment passing therethrough.

[0216] As discussed above, in some implementations, the illumination source 960 couples light redirected out of the waveguide into the waveguide 940 to illuminate the eye 210. In such embodiments, the combining optical element 944 may be polarization selective. For example, the combining optical element 944 may transmit light of a first polarization (p-polarized light) and transmit light of a second polarization (s-polarized light).

[0217] Thus, when light from illumination source 906 propagates through waveguide 940 and is redirected by coupling optical element 944, the illumination will be s-polarized. A polarization modifying optical element (e.g., a quarter-wave retarder) may be placed between waveguide 940 and eye 210 to cause a rotation of the polarization reflected from the eye. Light from light source 960 reflected from eye 210 passes twice through the quarter-wave retarder, so that the s-polarized light launched from the waveguide by coupling element 944 to illuminate the eye will be converted to p-polarized light.

[0218] This p-polarized light will be transmitted through the combining optical element 944 and the waveguide and will be incident on the reflective optical element 996 .

[0219] The imaging system 900 may further comprise a second polarization modifying element 978, which may comprise, for example, a retarder or a wave plate, as discussed above. The retarder may comprise, for example, a quarter-wave retarder. The second polarization modifying element 978 may be disposed distal to the waveguide 940, i.e., between the waveguide and the reflector 996. The second polarization modifying element 978 may also be disposed between the combining element light 944 and the reflector 996. The light from the eye 210 (p-polarized light) transmitted through the combining element light 944 passes through the second polarization modifying element 978 and is converted to circularly polarized light. If the reflector 996 reflects circularly polarized light, the light will again be reflected back to the waveguide 940 after passing through the polarization modifying element 978. Two passes through this polarization modifying element (e.g., a quarter-wave retarder) 978 will convert the light to s-polarized light, which will be redirected into the waveguide by the combining element 944, where it is directed to a camera (not shown).

[0220] As illustrated in FIG. 18 , the light 988 reflected from the eye 210 diverges. The light may be incident on and collimated by a curved or otherwise positively refractive reflector 996. A coupling optical element 944 configured to redirect the collimated light into a waveguide 940 would then direct the collimated light from the curved reflective optical element 996 towards an imaging device 920 (not shown). Thus, the light reflected from the eye 210 collimated by the curved reflective optical element 996 is coupled into the waveguide 940 and guided therein towards an outcoupling optical element 952. The outcoupling optical element 952 may be configured to direct the light out of the eyepiece 950 towards a camera (not shown).

[0221] Various variations are possible in the configuration of the imaging system. Different types of reflector 996 and combining element 944 may be employed. The reflector 996 and combining element 944 may be configured to act on, for example, linearly polarized light or circularly or elliptically polarized light. As discussed, the reflector 996 has refractive power. The reflector 996 and combining element 944 may comprise a cholesteric liquid crystal grating reflector and / or a lens, with or without refractive power. A polarization modifying element 978, such as a retarder, may be included between the combining element 944 and the reflector and / or between the combining element 944 and the eye. In some embodiments, a polarizer, such as a circular or linear polarizer, may be placed between the eye and the combining element 944. For example, if unpolarized light is reflected from the eye, a polarizer (e.g., a circular or linear polarizer) may be placed between the eye and the combining element 944. In some such cases, the combining element 944 is polarization selective.

[0222] In configurations such as those shown in FIGS. 17 and 18, where light reflected from the eye passes through the waveguide 940 to the curved reflective optical element 996, is collimated, and is redirected back to the waveguide, background noise is introduced. This background noise arises from light initially passing from the eye through the coupling optical element 944. As discussed above, the coupling optical element 944 may be configured to redirect collimated light into the waveguide 940 to be directed to the camera 920 in which the image is formed. However, the coupling optical element 944 will redirect some uncollimated light incident thereon. Thus, in its initial pass through the coupling optical element 944 and the waveguide 940 to the curved reflective optical element 996, some of the uncollimated (diverging) light reflected from the eye will be coupled into the waveguide by the coupling optical element 944 and will contribute background noise to the image of the eye formed by the camera 920. This noise will be superimposed on the image formed by collimated light retro-reflected by curved reflective optical element 996, which is coupled into the waveguide by coupling optical element 944 to be directed to camera 920.

[0223] In one design, this noise can be subtracted from the image. The process for subtracting the noise from the signal may involve (a) measuring the amount of light (referred to as N) that is coupled by the coupling optical element 944 in its initial pass through the coupling optical element 944 to the curved reflective optical element 996 where it is redirected and reaches the camera 920, and (b) measuring the total signal at the camera 920 as the light passes through the coupling optical element 944 and the waveguide 940 to the curved reflective optical element 996, is collimated, is reflected back to the coupling optical element, and is redirected to the camera. This total signal will also include some noise N because uncollimated light reflected from the eye passed through the coupling optical element 944 and reached the curved reflective optical element 996, and thus some of the uncollimated light will be redirected by the coupling optical element 944 to the camera 920. If the noise N can be measured separately from the total signal T, including the noise superimposed across the eye image, the noise N can be subtracted from the total signal T as represented by the following equation: I=TN where I represents the image from which the noise component N has been removed.

[0224] The above two measurements (a) and (b) can be obtained in a variety of ways. For example, as shown in Figure 19, a shutter 936 can be disposed between the curved reflective optical element 996 and the waveguide 940 and coupling optical element 944. This shutter 936 can be configured to block light when the shutter is in a first state and transmit light when the shutter is in a second state. This shutter 936 may comprise, for example, a liquid crystal shutter.

[0225] Thus, the noise component N can be measured when the shutter 936 is in a first state in which light reflected from the eye 210 is incident on and passes through the coupling optical element 944 towards the curved reflective optical element 996, but is prevented by the closed shutter from reaching the curved reflective optical element. As discussed above, a portion of the light reflected from the eye 210 is not largely collimated, but couples into the coupling optical element 944 and is redirected into the waveguide and therein directed to the camera 920. As referenced above, this light will not contribute to the formation of an image, but will be background noise. The camera 920 may record this noise N when the shutter 936 is closed.

[0226] The total signal T, including both the noise N and the image, can be measured when the shutter 936 is in a second state in which the shutter is open. The light reflected from the eye 210 again impinges on the coupling optical element 944. A portion of this light reflected from the eye 210 is not largely collimated, but couples into the coupling optical element 944 and is redirected into the waveguide where it is directed to the camera 920. However, most of the light reflected from the eye 210 passes through the coupling optical element 944, through the open shutter 936, to the curved reflective optical element 996. The curved reflective optical element 996 collimates the light and reflects at least a portion of it back to the coupling optical element 944, which redirects the collimated light into the waveguide 920 to be directed to the camera 920 to form an image of the eye 210. The camera 920 can capture the image of the eye 210.

[0227] Processing electronics (such as processing electronics 140) in communication with the camera 920 can receive the noise component N, measured when the shutter 936 is in a first closed state, and the total signal T, measured when the shutter is in a second open state, and can subtract (TN) the two. In this way, the noise N contributed by uncollimated light reflected from the eye 210 that is coupled into the combining optical element 944 on an initial pass therethrough can be subtracted from the total image signal T. The processing electronics may communicate with the camera 920 via wired electronic signals. Additionally or alternatively, the processing electronics may communicate with the camera 920 using one or more remote receivers. The processing electronics may reside remotely (e.g., a cloud computing device, a remote server, etc.).

[0228] Other methods may be employed to perform measurements (a) and (b), obtain N and T, and subtract N from T. For example, if the curved reflective optical element 996 is wavelength selective as shown in FIG. 18, the eye can be illuminated with different wavelengths of light at different times. For example, to perform measurement (a) and quantify the noise N, the eye can be illuminated with a wavelength that is not reflected by the curved reflective optical element 996. However, to perform measurement (b) and quantify the total signal T, the eye can be illuminated with a wavelength that is reflected by the curved reflective optical element 996. The noise N can then be subtracted from the total T (e.g., T N ) as discussed above.

[0229] 20-20E illustrate an example imaging system 900 configured to use wavelength modulation to make measurements and subtract noise components N as discussed above. The imaging system 900 in FIGS. 20A-20E includes a curved transmissive optical element 996 that is wavelength selective (such as that described with reference to FIGS. 17 and 18 above). For example, the curved transmissive optical element 996 has a wavelength-dependent reflective coating 998 on its curved surface. The imaging system 900 may also include one or more light sources or illumination sources (not shown) configured to illuminate the eye 210. The one or more light sources may be configured to emit infrared light. However, the one or more light sources can be configured to emit different colors or wavelengths of light at different times. Such wavelength modulation can enable separate measurements of N to be subtracted from the total signal T.

[0230] In various implementations, for example, one or more illumination sources 960, 934 may, in a first state, provide one or more wavelengths λ that are reflected by the curved reflective optical element. Reflect In the second state, one or more wavelengths λ are not reflected. Not Reflect In the second state, a negligible amount of wavelength λ is reflected by the curved reflective optical element. Reflect Similarly, in the first state, a negligible amount of wavelength λ is not reflected. Not Reflect is released.

[0231] In some embodiments, the reflected wavelength λ Reflect The reflected wavelength λ may be about 800 nm to 950 nm. Reflect The reflected wavelength λ may be about 835 nm to 915 nm. Reflect In some designs, the reflected wavelength λ Reflect is approximately 850 nm. Light emissions 928 from one or more light sources 960 may illuminate the eye.

[0232] As shown in FIG. 20B, the wavelength λ that is not reflected by the curved reflective optical element 944Not Reflect Light 988 having a length λ Reflect ) is reflected from a portion of the eye 210 (e.g., the cornea). This light is reflected from the wavelengths λ Not Reflect , light ray 916 is shown propagating through curved reflective optical element 996 into the environment in front of the user.

[0233] Although the light 988 incident on the combining optical element 944 is not collimated, the combining optical element nevertheless couples at least some of the light 914 into the waveguide 940 for guidance to the camera 920. Thus, the camera 920 can capture an image (Image #1) that corresponds to the noise component N resulting from the uncollimated light redirected by the combining optical element 944 on its initial pass to the curved reflective optical element 996. This image (Image #1) is background noise and is not a discernible image of the eye. The processing electronics 140 is shown to receive this first image (Image #1).

[0234] In FIGS. 20C-20E, an illumination source (not shown) emits one or more wavelengths λ that are reflected by a curved reflective optical element. Reflect and a negligible amount of wavelength λ that is not reflected Not Reflect This emits a wavelength of λ Reflect can be, for example, 850 nm.

[0235] As shown in FIG. 20C, on the first pass through the coupling optical element 944, a portion of the light 988 reflected from the eye 210 that is incident on the coupling optical element 944 is coupled by the coupling optical element 944 into the waveguide 940 (as in FIG. 20B) and directed towards the camera 920. In addition, the light 988 reflected from the eye 210 at wavelength λ ReflectA curved transmissive optical element 996, which selectively reflects light from the eye 210, reflects and collimates the non-intercoupled light 918 reflected from the eye 210 that is incident on the curved transmissive optical element. As illustrated in FIG. 20E, a coupling optical element 944 redirects and couples this collimated reflected light into a waveguide 940 towards the camera 920. FIG. 20E shows both components reaching the camera 920: light 988 that is incident on the coupling optical element 944 and reflected from the eye 210 on the first pass through the coupling optical element 944 that is coupled into the waveguide 940 by the coupling optical element, and light that is reflected and collimated by the curved transmissive optical element 996 that is coupled into the waveguide by the coupling optical element. The camera 920 can capture an image (Image #2) that corresponds to this total image component T. The processing electronics 140 is shown receiving this second image (Image #2).

[0236] As discussed above, the processing electronics may subtract noise from the image TN. In this example, image #1 may be subtracted from image #2. Thus, the processing electronics 140 may be configured to modify the second image based on the first image. However, other approaches are also possible. For example, the processing electronics 140 may be configured to create a new image that represents a version of the second image with reduced optical noise. The implementations for subtracting noise from the image may be used in the implementations described above. For example, the implementations shown in Figures 10, 11A-11E, and / or Figures 12A-12E may include a shutter 936 and / or a curved transmissive optical element 996 having a wavelength-dependent reflective coating 998 configured to selectively reflect the non-intercoupled light 912 and direct the light to the imaging device 920.

[0237] As discussed above, Image #1 is a filter that detects light at one or more wavelengths λ where light is not reflected by the curved reflective optical element. Not Reflect and a negligible amount of reflected wavelength λ ReflectImage #2 was taken for the case where light was illuminated at one or more wavelengths λ 2 as reflected by the curved reflective optical element. Reflect and a negligible amount of wavelength λ that is not reflected Not Reflect . Thus, one or more of the illumination sources 960, 934 may be configured to modulate wavelengths. For example, in one design, one or more of the illumination sources 960, 934 may modulate one or more wavelengths λ that are not reflected by the curved reflective optical element. Not Reflect and a negligible amount of reflected wavelength λ Reflect The one or more illumination sources may further comprise a first illumination source configured to output one or more wavelengths λ reflected by the curved reflective optical element. Reflect and a negligible amount of wavelength λ that is not reflected Not Reflect The illumination source may include a second illumination source configured to output λ . The intensities of the first and second illumination sources may alternatively be increased and decreased, turned on and off, attenuated and unattenuated, passed and blocked to provide modulation of the wavelength of light to illuminate the eye. For example, during a first time interval, the first illumination source may be blocked while the second illumination source is not blocked. During a subsequent second time interval, the second illumination source may be blocked while the first illumination source is not blocked. This process may be repeated to provide modulation of the wavelength of light to illuminate the eye. In other designs, the wavelength of the light source may be λ . Reflect and λ Not Reflect It may be tuned and detuned to shift the wavelength back and forth between. Other arrangements are possible.

[0238] As described above, the imaging system 900 may additionally be included in a head mounted display, such as an augmented reality display, that provides the ability to image the eye by collecting light with an eyepiece 950. Such imaging systems 900 may be used for eye tracking. Multiple images of the retina or anterior portion of the eye may be acquired. Eye movements and / or repositioning may be ascertained from these images to track eye position and / or orientation. These imaging systems may also be used for biometric imaging and / or to identify a user. For example, an image of a user's eye, such as the retina or iris, may be acquired and recorded. A subsequent image of the wearer's eye (e.g., retina, or iris) may be acquired at a later time. The two images may be compared to determine whether the wearer in the subsequent instance is the wearer in the first instance. However, other uses for the imaging system are also possible.

[0239] Although the illumination system may be described above as being waveguide-based and comprising one or more waveguides, other types of light redirecting optical elements may be employed in place of a waveguide. Such light redirecting optical elements may include redirecting features and eject light from the light redirecting optical elements, for example, onto a spatial light modulator. Thus, in any of the examples described herein and any of the examples below, any reference to a waveguide may be replaced with a light redirecting optical element, instead of a waveguide. Such light redirecting optical elements may comprise, for example, a polarizing beam splitter, such as a polarizing beam splitting prism.

[0240] As discussed above, the systems described herein can enable the collection of biometric data and / or biometric identification. For example, an eye or a portion thereof (e.g., a retina) can be imaged to provide such biometric data and / or biometric identification. Images of the eye, such as a retina, may be acquired at various times when the head mounted display system is worn by a user, perhaps the same user. Such collections of images can be recorded, for example, in a database. These images may be analyzed to collect biometric data. Such biometric data may be useful for monitoring the health or medical status of a user. Different medical parameters can be monitored by imaging a patient, for example, the patient's eye (e.g., a retina). The medical parameters can be recorded and compared with subsequent measurements acquired when the user is wearing the head mounted display system.

[0241] Additionally, if a person begins wearing the head mounted display system and an image of the user's eyes is captured that does not match the image stored in the database, a conclusion may be drawn that the person currently wearing the head mounted display system is different from the previous user. This may be useful in determining whether the headset is being worn by an intended user or by a new user. Such a feature may enable certain medical, security, and / or ease of use applications or functionality. For example, the head mounted display may be configured to identify the wearer based on characteristics of the wearer's eyes. For example, the system may be configured to determine an individual based on the wearer's retina (e.g., blood vessels), corneal features, or other ocular features. In some implementations, for example, a set of markers may be determined for a particular wearer. Based on the set of markers, the system may be able to determine that a previous user is wearing the headset, or alternatively, that another user is wearing the headset. The markers may include the shape or center of the user's cornea, the configuration of blood vessels in the user's retina, the intensity and / or location of light reflection from the cornea, the shape of the side of the eye, and / or any other biometric marker. In some implementations, a confusion matrix can be determined. As discussed above, for example, in the discussion of developing a retinal map using virtual / fixation targets at various locations (see, e.g., FIG. 13B), the system may have the user look at a set of predefined directions or eye poses and develop a matrix of properties of the eye or part of the eye (e.g., cornea, retina, etc.) associated with each direction or eye pose. Using such a matrix, the system can determine the identity of the individual. Other methods are also possible.

[0242] Similarly, various configurations of the system are possible, as discussed above. For example, FIG. 21 shows an example eyepiece 900 that may be used to project light into a user's eye while simultaneously imaging the user's eye. The eyepiece 900 shown includes an internal coupling optical element 2104, a light dispersing element 2108, a light focusing element 2116, and an external coupling optical element 2120 on the opposite side of the coupling optical element 2112. Each of these optical elements may be disposed within or on a waveguide 2102. The waveguide 2102 may correspond to, for example, one of the waveguides 670, 680, 690 described herein (see, e.g., FIGS. 9A-9C). The in-coupling optical element 2104 may correspond to one of the in-coupling optical elements 700, 710, 720 and / or the in-coupling optical element 942 (see, e.g., FIG. 10 ) described herein and may be configured to inject image content from a projector into the waveguide and / or inject illumination from the light source 960. The light dispersing element 2108 may correspond to one of the light dispersing elements 730, 740, 750 (see, e.g., FIG. 9A-9C ) described herein and may be used to diffuse light in a given direction and redirect light from the in-coupling optical element 2104 to the coupling optical element 2112. The coupling optical element 2112 may correspond to the coupling optical element 944 (see, e.g., FIG. 10 ) described herein. In some designs, the coupling optical element 2112 includes functionality described herein for the out-coupling optical elements 800, 810, 820 (see, e.g., FIG. 9A-9C ). The focusing element 2116 may be configured to reduce the lateral spatial extent of the light received from the coupling optical element 2112 and redirect the light towards the outcoupling optical element 2120. The outcoupling optical element 2120 may correspond to the outcoupling optical element 952 described herein (e.g., see FIG. 10 ).

[0243] The in-coupling optical element 2104 may be disposed within or on the waveguide 2102 to receive light, such as from a projector (e.g., image projector 930) and / or an illuminator (e.g., light source 960). The light may be passed through the waveguide 2102 to an associated light dispersive optical element 2108. Any of the in-coupling optical element 2104, the light dispersive optical element 2108, or the coupling optical element 2112 may be disposed on a major surface of the waveguide (e.g., on a top or bottom surface) or within the waveguide. Similarly, any one or combination of the light collection element 2116 and / or the out-coupling optical element 2120 may also be disposed on a major surface of the waveguide 2102 (e.g., on a top or both major surfaces) or within the waveguide.

[0244] The combining optical element 2112 may receive light from the light dispersing element 2108 (e.g., via TIR) and expand the light to enter the user's eye. The combining optical element 2112 may thus be positioned in front of the user's eye and project image content therein. Additionally or alternatively, the combining optical element 2112 may be configured to provide illumination light on and / or into the user's eye.

[0245] Light reflected from the eye (e.g., illumination light from an illumination source) may be reflected and captured by the coupling optical element 2112. Thus, in some embodiments, the coupling optical element 2112 may act to both outcouple light received from the light dispersive element 2108 and incouple light received from the eye into the waveguide 2102.

[0246] In some embodiments, the combining optical element 2112 may include one or more diffractive optical elements (DOEs) such that the combining optical element 2112 has dual functionality. A first DOE (e.g., a grating, a holographic region) may be configured to similarly outcouple light, and a second DOE may be configured to incouple light reflected from the eye into the waveguide 2102. In some embodiments, both the first and second DOEs are superimposed (e.g., occupy the same or nearly the same volume) within the waveguide 2102.

[0247] Alternatively, in some embodiments, the combining optical element 2112 includes at least two DOEs stacked over or in front of the other. For example, referring to FIG. 21, the first DOE of the combining optical element 2112 may be disposed over the second diffractive element, while the second diffractive element may be disposed below the first DOE. The order of each DOE may be reversed in other implementations.

[0248] Cholesteric Liquid Crystal Mirror Some liquid crystals are in a phase called chiral or cholesteric phase. In the cholesteric phase, the liquid crystal may exhibit a molecular twist along an axis perpendicular to the director, with the molecular axis being parallel to the director. As described herein, a cholesteric liquid crystal (CLC) layer comprises a plurality of liquid crystal molecules in a cholesteric phase that extend in a direction perpendicular to the director, such as the layer depth direction, and are continuously rotated or twisted in a rotation direction, such as clockwise or counterclockwise. The director of the liquid crystal molecules in the chiral structure may be characterized as a spiral with a helical pitch (p), which corresponds to a length in the layer depth direction that corresponds to the net rotation angle of the liquid crystal molecules in the chiral structure by one turn in the first rotation direction. In other words, the helical pitch refers to the distance over which the liquid crystal molecules undergo a full 360° twist. Liquid crystals exhibiting chirality may also be described as having a twist angle or rotation angle (φ), which may refer to, for example, the relative azimuthal rotation between successive liquid crystal molecules in the layer normal direction, and a net twist angle or net rotation angle, which may refer to, for example, the relative azimuthal rotation between the top and bottom liquid crystal molecules across a defined length, for example, the length of the chiral structure or the thickness of the liquid crystal layer. As described herein, a chiral structure refers to a plurality of liquid crystal molecules in a cholesteric phase that extend in a direction perpendicular to the director, for example, the layer depth direction, and are continuously rotated or twisted in a rotation direction, for example, clockwise or counterclockwise. In one aspect, the director of the liquid crystal molecules in the chiral structure may be characterized as a spiral having a helical pitch.

[0249] 22 illustrates a cross-sectional side view of a cholesteric liquid crystal (CLC) layer 1004 with multiple uniform chiral structures, according to an embodiment. In the CLC layer 1004, adjacent chiral structures in a lateral direction, e.g., in the x-direction, have liquid crystal molecules that are similarly aligned. In the illustrated embodiment, the chiral structures 1012-1, 1012-2, ... 1012-i are similarly configured such that liquid crystal molecules of different chiral structures at approximately the same depth, e.g., the liquid crystal molecules closest to the light incidence surface 1004S, have the same rotation angle, and the successive rotation angles of successive liquid crystal molecules at approximately the same depth, and the net rotation angle of the liquid crystal molecules of each chiral structure.

[0250] The CLC 1004 comprises a CLC layer 1008 comprising liquid crystal molecules arranged as a plurality of chiral structures 1012-1, 1012-2, ... 1012-i, each chiral structure comprising a plurality of liquid crystal molecules, where i is any suitable integer greater than 2. In operation, when incident light, comprising a combination of a light beam having left-handed circular polarization and a light beam having right-handed circular polarization, is incident on the surface 1004S of the CLC layer 1008 by Bragg reflection, light with one of the circular polarization handednesses is reflected by the CLC layer 1004, while light with the opposite polarization handedness is transmitted through the CLC layer 1008 without substantial interference. As described herein and throughout this disclosure, handedness is defined as viewed in the direction of propagation. According to an embodiment, when the direction of polarization or handedness of polarization of the light beams 1016-L, 1016-R is matched to have the same rotational direction as the liquid crystal molecules of the chiral structures 1012-1, 1012-2, ... 1012-i, the incident light is reflected. As shown, the incidence on the surface 1004S results in a light beam 1016-L with left-handed circular polarization and a light beam 1016-R with right-handed circular polarization. In the illustrated embodiment, the liquid crystal molecules of the chiral structures 1012-1, 1012-2, ... 1012-i are continuously rotated in the clockwise direction in which the incident light beams 1016-L, 1016-R travel, i.e., in the positive x-direction, which is the same rotational direction as the light team 1016-R with right-handed circular polarization. As a result, the light beam 1016 -R having right-handed circular polarization is substantially reflected, while the light beam 1016 -L having left-handed circular polarization is substantially transmitted through the CLC layer 1004 .

[0251] As described above, by matching the handedness of the polarization of incident elliptically or circularly polarized light with the rotation direction of the liquid crystal molecules of the chiral structure of the CLC layer, the CLC layer can be configured as a Bragg reflector. Furthermore, one or more CLC layers with different helical pitches can be configured as wavelength-selective Bragg reflectors with high bandwidth. Based on the concepts described herein with respect to various embodiments, the CLC layer can be configured as an off-axis or on-axis mirror configured to selectively reflect a first wavelength range, e.g., infrared wavelengths (e.g., near-infrared), while transmitting another wavelength range, e.g., visible wavelengths.

[0252] FIG. 23 illustrates an example of an eye tracking system 2300 employing a cholesteric liquid crystal reflector (CLCR), e.g., a wavelength-selective CLCR 1150, configured to image a viewer's eye 302, according to various embodiments. Unlike the CLC layer 1004 described above with respect to FIG. 22, chiral structures in the wavelength-selective CLCR 1150 adjacent in a lateral direction, e.g., in the x-direction, have liquid crystal molecules that are arranged differently. That is, the chiral structures are configured such that liquid crystal molecules of different chiral structures at approximately the same depth, e.g., the liquid crystal molecules closest to the light incidence surface 1004S, have different rotation angles. As a result, light incident on the CLCR 1150 rotates at an angle (θ R ) is reflected.

[0253] Eye tracking may be a useful feature in interactive vision or control systems, including wearable display systems described anywhere herein, for virtual / augmented / mixed reality display applications, among other applications. To achieve effective eye tracking, it may be desirable to capture images of the eye 302 at a low gaze angle, and in this regard, it may be desirable to position the eye tracking camera 702b near the center position of the viewer's eye. However, such a position of the camera 702b may interfere with the user's view. Alternatively, the eye tracking camera 702b may be positioned at a lower position or to the side. However, such a position of the camera may increase the difficulty of obtaining robust and accurate eye tracking, since the eye images are captured at a steeper angle. By configuring the CLCR 1150 to selectively reflect infrared (IR) light 2308 (e.g., having a wavelength of 850 nm) away from the eye 302 while transmitting visible light 2304 from the world, the camera 702b can be placed away from the user's view while capturing eye images at normal or low gaze angles. Such a configuration does not interfere with the user's view since visible light is not reflected. The same CLCR 1150 can also be configured as an IR illumination source 2320 by reflecting IR light from an IR source, e.g., an IR LED, into the eye 302 as shown. The low gaze angle of the IR illuminator can result in less occlusion from, for example, eyelashes, a configuration that allows for more robust detection of specular reflections and can be a useful feature in modern eye tracking systems.

[0254] 23, according to various embodiments, the CLCR 1150 comprises one or more cholesteric liquid crystal (CLC) layers each comprising a plurality of chiral structures, each chiral structure comprising a plurality of liquid crystal molecules extending in a layer depth direction (e.g., z-direction) and sequentially rotated in a first rotation direction, as described above. The alignment of the liquid crystal molecules in the chiral structures is such that the one or more CLC layers are arranged to provide a first wavelength (λ 1 ) while substantially Bragg reflecting a first incident light having a second wavelength (λ 2As explained above, each of the one or more CLC layers is configured to substantially Bragg reflect first and second incident light of elliptically or circularly polarized light having a handedness of polarization that matches the first rotation direction when viewed in the layer depth direction, while being configured to substantially transmit first and second incident light of elliptically or circularly polarized light having a handedness of polarization that is opposite to the first rotation direction when viewed in the layer depth direction. According to an embodiment, the alignment of the liquid crystal molecules, which varies periodically in the lateral direction, is arranged to have a period in the lateral direction such that the ratio between the first wavelength and the period is about 0.5 to about 2.0. According to an embodiment, the first wavelength is within the near infrared range of about 600 nm to about 1.4 μm, e.g., about 850 nm, and the second wavelength is within the visible range having one or more colors as described elsewhere herein. According to various embodiments, the liquid crystal molecules in the chiral structure are pretilted with respect to the direction normal to the layer depth direction. As configured, the one or more CLC layers are configured such that the first incident light is tilted at an angle of about 50° with respect to the layer depth direction. o , about 60 o , about 70 o , or about 80 o The angle (θ R ) is configured to be reflected by

[0255] The wavelength-selective CLCR 1150 thus configured comprises one or more cholesteric liquid crystal (CLC) layers each comprising a plurality of liquid crystal molecules extending in a layer depth direction and continuously rotated in a first rotation direction, and the arrangement of the chiral structured liquid crystal molecules varies periodically in a lateral direction perpendicular to the layer depth direction such that the one or more CLC layers are configured to substantially Bragg-reflect a first incident light having a first wavelength, e.g., an IR wavelength, while substantially transmitting a second incident light having a second wavelength, e.g., a visible wavelength.

[0256] Similar liquid crystal layers and structures may be used for the reflector 996 and coating 998 described above in connection with Figures 17-20E. The coating 998 may comprise, for example, a liquid crystal coating and, in some implementations, may be wavelength and / or polarization selective. However, other types of coatings 998 and reflectors 996 may be employed.

[0257] For example, as discussed above in connection with FIG. 16, a lens 980 may be used to modify (e.g., collimate) the propagation of light directed to the coupling optical element 944. This light may be light reflected from the user's eye, such as the anterior surface (e.g., corneal surface) of the user's eye. The distance from the eye, e.g., the anterior surface (e.g., corneal surface) to the coupling optical element 944 may be, for example, about 20 mm. A positive lens 980, such as a lens having a focal length of about 20 mm, may be configured to collimate light reflected from the eye 210, such as the anterior portion of the eye (e.g., cornea). Light reflected from the anterior surface of the eye may be coupled into the waveguide 940 and directed therein to the camera. With the focal length set to the distance to the anterior surface of the eye, the camera can image such a surface. Thus, in various implementations, the positive lens 980 may have a focal length that is therefore equal or substantially equal to the distance from the lens to the portion of the eye 210 to be imaged, such as the cornea.

[0258] Although a refractive optical element is shown, other types of lenses or optical elements with refractive power, such as positive refractive power, may be used. For example, the lens may comprise a diffractive optical element, such as a diffractive lens or a hologram. Such a lens may be disposed between the eye and the coupling optical element 944 in some implementations.

[0259] In various implementations, the combining optical element 944 may include refractive power. The combining optical element 944 may comprise, for example, a diffractive optical element having refractive power. The diffractive optical element may comprise, for example, a diffraction grating. The diffractive optical element may comprise a holographic optical element or a hologram. The diffractive optical element may have diffractive features, such as surface features, configured to both redirect light into the waveguide and provide refractive power. Other types of diffractive optical elements are also possible. In various implementations, the diffractive optical element may comprise a liquid crystal or may comprise a liquid crystal grating. The diffractive optical element may also comprise a polarization grating. In addition, the diffractive optical element may comprise a liquid crystal polarization grating. Some non-limiting examples of liquid crystal gratings, liquid crystal polarization gratings, and other liquid crystal optical elements are described in the following published applications, each of which is incorporated herein by reference in its entirety and for all purposes: "MULTILAYER LIQUID CRYSTAL ELECTRONICS INC., "U.S. Pat. Appln. KOKAI Publication No. 2011-013399, "Multilayer Liquid Crystal Optical Devices," and "U.S. Patent No. 5,993,3 ... each of which is incorporated herein by reference in its entirety and for all purposes. No. 2018 / 0143438, entitled "DIFFFRACTIVE GRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES," filed on November 16, 2017; U.S. Patent Publication No. 2018 / 0143485, entitled "SPATIALLY VARIABLE LIQUID CRYSTAL DIFFRACTION GRATINGS," filed on November 16, 2017; U.S. Patent Publication No. 2018 / 0143509, entitled "WAVEGUIDE LIGHT MULTIPLEXER USING CROSSED GRATINGS," filed on November 16, 2017; and U.S. Patent Publication No. 2018 / 0143509, entitled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON FLAT ... No. 2018 / 0239177, filed February 22, 2018, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL," and U.S. Patent Publication No. 2018 / 0164627, filed December 7, 2017, entitled "DIFFRACTIVE DEVICES BASED ON CHOLESTERIC LIQUID CRYSTAL,".

[0260] The diffractive optical element may have a refractive power that alters the propagation of light incident thereon. The diffractive optical element may collimate the reflected light, for example, as a surface having a distance from the diffractive optical element that corresponds to the focal length of the diffractive optical element. Such a distance may be, for example, about 15-20 mm (e.g., 20 mm or an approximation thereof). Such a focal length may provide collimation of light reflected from an anterior surface of the eye, such as the corneal surface (e.g., the cornea). Other distances are also possible. For example, the distance may be in the range of about 10-40 mm, 10-50 mm, 5-40 mm, or 5-50 mm, or any range between any of the distance values ​​herein. Values ​​outside these ranges are also possible.

[0261] FIG. 24 shows an example of a coupling optical element (e.g., a coupling grating) 2111 disposed on the waveguide 2102 of the eyepiece 950. The coupling optical element 2111 comprises a diffractive optical element configured to couple light incident thereon into the waveguide 2102. The diffractive optical element also includes refractive power. For example, the diffractive optical element includes diffractive features, such as surface diffractive features, that provide refractive power, for example configured to collimate light incident thereon from the anterior surface of the eye (e.g., the corneal surface). A focusing element 2116 and an out-coupling optical element 2120 are also shown disposed on the waveguide 2102. The focusing element 2116 is disposed to receive light coupled into the waveguide 2102 by the coupling element 2111. The focusing element 2116 is configured to redirect light incident thereon from the coupling optical element 2111 to the out-coupling optical element 2120. The light collecting element 2116 is configured to reduce the lateral spatial extent of the light (e.g., light beam) from the at least one coupling element prior to reaching the at least one out-coupling optical element, the out-coupling optical element 2120. In some configurations, fewer optical elements may be used, possibly, for example, to reduce cost and / or optical losses, or for other reasons. For example, the light collecting element 2116 may be omitted in some embodiments. In such embodiments, light may be in-coupled from the input coupling element 2111 (e.g., after being reflected from the eye 210) and directly coupled to the out-coupling optical element 2120. The light may propagate through the eyepiece 950 between the input coupling element 2111 and the out-coupling optical element 2120. Other configurations are also possible. A camera is positioned relative to the out-coupling optical element 2120 and receives light therefrom. The out-coupling optical element 2120 is configured to direct the light received from the light collecting element 2116 to the camera to capture an image.

[0262] 24 also shows an in-coupling optical element 2104, a light dispersing element 2108, and an image content out-coupling optical element 2110 disposed on the waveguide 2102. The in-coupling optical element 2104 may be configured to couple light received from an image projector into the waveguide 2102. The light distributing element 2108 may be configured to redirect light received from the in-coupling optical element 2104 to the out-coupling optical element 2110, and additionally increase the spatial extent of the light, as discussed above. The out-coupling optical element 2110 may be configured to couple light guided within the waveguide 2102 out of the waveguide and direct such light to the eye for viewing image content from the projector.

[0263] In various implementations, one or more of these optical elements 2111, 2116, 2120, 2110, 2108, 2104 may be disposed within or on the waveguide 2102. Similarly, as discussed above, one or more of these optical elements 2111, 2116, 2120, 2110, 2108, 2104 may comprise diffractive optical elements.

[0264] 24, a coupling optical element (e.g., a coupling grating) 2111 may be laterally displaced from the out-coupling optical element 2110 on the waveguide 2102. In the implementation shown, a space separates the coupling optical element 2111 laterally from the out-coupling optical element 2110.

[0265] 25 shows another similar implementation in which a coupling optical element (e.g., a coupling grating) 2111 is laterally displaced from the out-coupling optical element 2110 on the waveguide 2102. However, in the implementation shown in FIG. 25, a space does not laterally separate the coupling optical element 2111 from the out-coupling optical element 2110.

[0266] Displacing the coupling optical element (e.g., a coupling grating) 2111 laterally from the out-coupling optical element 2110 on the waveguide 2102 can enable the coupling optical element to include a refractive power, for example, to collimate light received from the anterior surface of the eye (e.g., the corneal surface) such that the refractive power does not affect the propagation of light from and / or through the out-coupling optical element 2110 to the eye. The image presented to the eye from the image projector and the view of the environment in front of the user and the head mounted display therefore need not be affected (e.g., distorted or defocused, etc.) by the refractive power of the coupling optical element 2111.

[0267] FIG. 26 illustrates an implementation of an imaging system 900 configured to image multiple portions of the eye. For example, the imaging system 900 as shown in FIG. 26 may be configured to image both the anterior surface of the eye (e.g., the corneal surface) and the retina. The imaging system 900 includes a pair of internal coupling optical elements, a first internal coupling optical element 2111a and a second internal coupling optical element, disposed on the waveguide 2102 of the eyepiece. The imaging system 900 additionally includes a pair of focusing elements, a first focusing optical element 2116a and a second focusing optical element 2116b, disposed on the waveguide 2102. In some configurations, fewer optical elements may be used, possibly, for example, to reduce cost and / or optical losses, or for other reasons. For example, the focusing elements 2116a, 2116b may be omitted in some embodiments. In such an embodiment, light may be in-coupled from the first and second input coupling elements 2111a, 2111b (e.g., after being reflected from a portion of the eye 210, such as the retina and / or cornea) and directly coupled to the corresponding first and second out-coupling optical elements 2120a, 2120b. The light may propagate through the eyepiece 950 (e.g., via a waveguide, such as the waveguide 2102) between the first and second input coupling elements 2111a, 2111b and the corresponding out-coupling optical elements 2120a, 2120b. Other configurations are also possible. In addition, the imaging system 900 additionally includes a pair of light out-coupling optical elements, i.e., the first out-coupling optical element 2120a and the second out-coupling optical element 2120b, disposed on the waveguide 2102.

[0268] The first coupling optical element 2111a is configured to couple light incident thereon into the waveguide 2102. The first light collecting element 2116a is positioned to receive light coupled into the waveguide 2102 by the first coupling element 2111a. The first light collecting element 2116a is configured to redirect light incident thereon from the first coupling optical element 2111a to the first out-coupling optical element 2120a. The first light collecting element 2116a is also configured to reduce the lateral spatial extent of light (e.g., a light beam) from the first coupling element 2111a prior to reaching the first out-coupling optical element 2120a. A camera is positioned relative to the first out-coupling optical element 2120a to receive light therefrom. The camera is not shown in FIG. 26, but an area corresponding to the detection area 2130 of the camera is shown. The first out-coupling optical element 2120a is configured to direct the light received from the first light collecting element 2116a to a camera, and in particular, to a detection area 2130 shown in FIG. 26, to capture an image.

[0269] Similarly, the second coupling optical element 2111b is configured to couple light incident thereon into the waveguide 2102. The second light collecting element 2116b is positioned to receive light coupled into the waveguide 2102 by the second coupling element 2111b. The second light collecting element 2116b is configured to redirect light incident thereon from the second coupling optical element 2111b to the second out-coupling optical element 2120b. The second light collecting element 2116b is also configured to reduce the lateral spatial extent of light (e.g., light beam) from the second coupling element 2111a prior to reaching the second out-coupling optical element 2120a. A camera is positioned relative to the second out-coupling optical element 2120b to receive light therefrom. The camera is not shown in FIG. 26, but an area corresponding to the detection area 2130 of the camera is shown. The second out-coupling optical element 2120b is configured to direct the light received from the second light collecting element 2116b to a camera, and in particular, to a detection area 2130 shown in FIG. 26, to capture an image.

[0270] 26 also shows an in-coupling optical element 2104, a light dispersing element 2108, and an image content out-coupling optical element 2110 disposed on the waveguide 2102. The in-coupling optical element 2104 may be configured to couple light received from an image projector into the waveguide 2102. The light distributing element 2108 may be configured to redirect light received from the in-coupling optical element 2104 to the out-coupling optical element 2110, and additionally increase the spatial extent of the light, as discussed above. The out-coupling optical element 2110 may be configured to couple light guided within the waveguide 2102 out of the waveguide and direct such light to the eye for viewing image content from the projector.

[0271] In the implementation illustrated in FIG. 26, the first and second coupling optical elements (e.g., coupling gratings) 2111a, 2111b are laterally displaced from the external coupling optical element 2110 on the waveguide 2102. In the implementation shown, a space laterally separates the first and second coupling optical elements 2111a, 2111b from the external coupling optical element 2110. Further, in the example of FIG. 26, the first and second coupling optical elements 2111a, 2111b are laterally displaced from one another on the waveguide 2102. In other implementations, two or more of the first coupling optical element (e.g., coupling grating) 2111a, the second coupling optical element 2111b, and the external coupling optical element 2110 need not be laterally displaced from one another on the waveguide 2102.

[0272] In various implementations, one or more of these optical elements 2111a, 2111b, 2116a, 2116b, 2120a, 2120b, 2110, 2108, 2104 may be disposed within or on the waveguide 2102. Similarly, as discussed above, one or more of these optical elements 2111a, 2111b, 2116a, 2116b, 2120a, 2120b, 2110, 2108, 2104 may comprise diffractive optical elements.

[0273] The imaging system 900 of FIG. 26 can be configured to image multiple portions of the eye. For example, the imaging system 900 may be configured to image both the anterior surface of the eye (e.g., the corneal surface) and the retina. The first internal coupling optical element 2111a may have, for example, a lens disposed in front of it having or with refractive power. The first coupling optical element 2111a may comprise, for example, a diffractive optical element configured to couple light into the waveguide 2102, but also configured to impart refractive power thereto. Additionally or alternatively, a lens having refractive power may be disposed in front of the first coupling optical element 2111a. The refractive power may be configured to modify the propagation of light received by the first internal coupling optical element 2111a such that a particular portion of the eye may be imaged. In some examples, the refractive power may be a positive refractive power. Additionally, the refractive power may be such that the anterior surface of the eye may be imaged. The optical power may correspond to a focal length of, for example, about 15-25 mm (e.g., about 20 mm). As a result, light reflected from the anterior surface of the eye, which may be about 15-25 mm (e.g., about 20 mm) from the first coupling optical element 2111a, may be collimated and coupled into the waveguide. Other focal lengths are also possible. For example, the focal length may be in the range of about 10-40 mm, or 10-50 mm, 5-40 mm, or 5-50 mm, or any range between any of the distance values ​​herein. Values ​​outside these ranges are also possible.

[0274] In contrast, in various implementations, the second coupling optical element 2111b may not have refractive power and may not include a lens disposed in front of it. The lack of refractive power associated with the second coupling optical element 2111b would result in light from an anterior portion of the eye (e.g., the corneal surface) not being collimated and imaged by the camera. However, light reflected from the retina may be collimated in response to passing through the natural lens of the user's eye. This light collimated by the natural lens of the eye may be coupled into the waveguide 2102 by the second coupling optical element 2111b and imaged by the camera. In this manner, the light collected by the first coupling optical element 2111a may image the anterior surface of the eye (e.g., the corneal surface of the eye), and the second coupling optical element 2111b may image the retina of the user's eye. In some implementations, such a configuration allows the light collected by the first combining optical element 2111a to form an image of the phosphene on the eye, for example, on the anterior surface of the eye (e.g., the corneal surface of the eye). The second combining optical element 2111b may image the retina of the user's eye, as discussed above. In some examples, the second combining optical element 2111b and / or a lens disposed in front of it may have an amount of net refractive power (e.g., a non-zero amount of net refractive power) that is weaker than that of the first combining optical element 2111a and / or a lens disposed in front of the first combining optical element 2111a. In particular, the second combining optical element 2112b may have a refractive power and / or a lens associated therewith, however, the total refractive power of the second combining optical element 2112b and / or any lenses associated with the second combining optical element is less than the refractive power of the first combining optical element and / or any lenses associated with the first combining optical element.

[0275] In some implementations, an image formed by light collected by the first combining optical element 2111a is adjacent to (e.g., not superimposed over) an image formed by light collected by the second combining optical element 2111b. For example, an image of the anterior surface of the user's eye (e.g., the cornea) may be formed adjacent to (e.g., not superimposed over) an image of the retina.

[0276] Polarization techniques can be used to attenuate or remove light from the anterior surface (e.g., the corneal surface) so that it does not affect the image formed by the light collected by the second coupling optical element 2111b. For example, the eye can be illuminated with polarized light having a first polarization, and the camera can form an image using light from the second out-coupling optical element 2120b using light of a second, different polarization. For example, the second out-coupling optical element 2111b can be a polarization-selective coupling element that selectively out-couples light of a second polarization different from the first polarization. Additionally or alternatively, a polarizer 2140 that filters out the first polarization (e.g., selectively transmits the second polarization) may be included between the second outcoupling optical element 2120b and the camera 920, as illustrated in FIG. 27, which is a cross section through the waveguides 950, 2102, the focusing optical elements 2116a, 2116b, and the outcoupling optical elements 2120a, 2120b shown in FIG. 26.

[0277] Such a configuration may be used to reduce undesired reflections (e.g., glints) from the cornea, etc., when imaging the retina. Reflections from the cornea will be specular. Thus, when light of a first polarization is incident on the cornea, the light reflected from the cornea will retain the first polarization. In contrast, the retina is diffuse. When light of a first polarization is incident on the retina, the light reflected from the retina will not simply retain the first polarization. Diffuse reflections are more likely to result in unpolarized light. Thus, a second polarization, different from the first polarization, will be present in the light reflected from the retina. As a result, by forming an image with light coupled out of the second out-coupling optical element 2120b using light of the second polarization, an image of the retina will be obtained, while images of the cornea or glints will be suppressed. Similarly, by illuminating with a first polarization and imaging with a second, different polarization, the retina can be imaged with reduced glare from the cornea.

[0278] Thus, in various implementations, polarization-specific optical filters or polarization-selective optical elements (e.g., coupling gratings) may be used to reduce unwanted light reflected from the eye 210 (e.g., the cornea). For example, unwanted light, glare, or flashes may be reflected from the cornea in a manner that may saturate the image captured by the camera. As discussed above, light reflected from the cornea may be specular and maintain its polarization. In contrast, light reflected from the retina may be more diffusely reflected and less homogeneously polarized. Similarly, a combination of polarizers may be used to remove some or most of the unwanted light reflected from the cornea. Initially, polarized light may be used to illuminate the user's eye. In some designs, a polarized illumination source (e.g., a light source) may be used. Additionally or alternatively, a first polarizer (e.g., a polarization-specific optical filter or polarization-selective optical coupling element that couples illumination light into the illumination waveguide) may be positioned at the beginning of the optical path of the illumination source to provide an initial polarization of light to the eye. A second polarizer (e.g., a polarization-specific optical filter or a polarization-selective combining element) may be positioned in the optical path before the light enters the camera. The second polarizer may be 90 degrees from the first polarizer. oThe light reflected from the retina may be rotated (e.g., the polarizer may be "crossed"). As a result, the eye will be illuminated with a first polarization with light of the first polarization reflected from the cornea. This light will not pass through a cross polarizer (which preferentially passes light of the second polarization) located proximal to the camera. However, the light reflected from the retina will contain the second polarization. Similarly, light diffusely reflected from the retina will pass through the polarizer 2140 proximal to the camera, allowing an image of the retina to be captured by the camera. Thus, in such a configuration, undesirable light received from the eye (e.g., the cornea) and entering the camera can be reduced or eliminated from the image captured using light from the second coupling optical element 2111b. Other configurations are also possible. For example, the polarization-selective coupling element 2111b and / or the polarization-selective out-coupling optical element 2120b may be used in addition to or as an alternative to a polarizer such as the polarizer 2140 proximal to the camera. Additionally or alternatively, a polarized light source may be used for illumination (e.g., illuminating the eye), the effect of which may again be to reduce or eliminate undesired light received from the eye (e.g., the cornea) before entering the imaging device 920.

[0279] As illustrated in FIG. 27, in various implementations, such a polarizer is not used in the optical path from the first coupling optical element 2111a and the camera or between the first out-coupling optical element 2120a and the camera. As a result, images of the cornea and the phosphene may be obtained from the light coupled into the waveguide by the first coupling optical element 2111a and / or the light coupled out of the waveguide from the first out-coupling optical element 2120a. As discussed above, this first coupling optical element 2111a may have refractive power or a lens associated therewith that is specifically used to image the cornea and / or the phosphene. Similarly, the polarization-selective coupling optical element 2111a or the polarization-selective out-coupling optical element 2120a that filters out light of the first polarization will not be used as the first coupling optical element 2111a and the out-coupling optical element 2120a, respectively. In addition, the polarizer 2140 between the coupling optical element 2111a and the camera 920 or between the external coupling optical element 2111a and the camera 920, which filters out light of the first polarization, would not be used.

[0280] Various variations are possible. For example, while the first and second outcoupling optical elements 2120a, 2120b are described above as combining light and forming an image on a single camera (e.g., a single detection area 2130), in other implementations the first and second outcoupling optical elements 2120a, 2120b can combine light and form an image on separate first and second cameras. Other variations are also possible.

[0281] Additionally, in some implementations, one or more of the eyepieces described above with reference to FIGS. 24-27 (e.g., eyepiece 950) may be a dedicated imaging eyepiece layer (e.g., omitting the in-coupling optical element 2104, the light dispersive element 2108, and / or the out-coupling element 2110). In such implementations, the imaging eyepiece layer may be included as a layer in the stack of waveguides. One or more other layers in the stack of waveguides may include the in-coupling optical element 2104, the light dispersive element 2108, and / or the out-coupling element 2110.

[0282] In some embodiments, the dedicated imaging eyepiece layer can be configured to capture an image of the environment. In some such embodiments, the imaging eyepiece layer may be positioned proximate to the environment when disposed in a head mounted display. In some such configurations, the dedicated imaging eyepiece layer may be the outermost layer in a waveguide stack, such that it is positioned between other layers (e.g., a waveguide) and the environment. In some embodiments, the dedicated imaging eyepiece layer can be configured to capture an image of the user's eye 210. In some such embodiments, the imaging eyepiece layer may be positioned proximate to the user when disposed in a head mounted display. In some implementations, the dedicated imaging eyepiece layer may be the innermost layer in a waveguide stack, such that it is positioned between other layers (e.g., a waveguide) and the user. Other configurations are also possible.

[0283] In an embodiment, the first and second coupling elements 2111a, 2111b may be laterally aligned but displaced in depth (e.g., along the z-axis, i.e., towards the other side of the page of FIG. 26). For example, in such an embodiment, the first and second coupling elements 2111a, 2111b may be positioned on opposite sides of the same waveguide. In an embodiment, the first input coupling element 2111a may be disposed on or in a first waveguide, and the second input coupling element 2111b may be disposed on or in a second waveguide. Additionally or alternatively, the first and second light collecting elements 2116a, 2116b and / or the first and second outcoupling optical elements 2120a, 2120b may be disposed on opposite sides of the same waveguide and / or may be disposed on or in corresponding separate waveguides.

[0284] One or more of the coupling optical elements 2111, 2111a, 2111b described above with reference to Figures 24-27 may be wavelength selective, such that the optical element is configured to interact only with a certain wavelength or band of wavelengths. The wavelength or band of wavelengths may include non-visible light (e.g., infrared light or a specific band thereof). In some implementations, the first input coupling element 2111a may be configured to act on lower wavelengths than those for the second input coupling element 2111b (or vice versa). For example, the first coupling optical element 2111a may be configured to interact with approximately 800 nm light, and / or the second coupling optical element 2111b may be configured to interact with approximately 950 nm light (or vice versa). Optical filters may also be used to provide similar wavelength selectivity. In various implementations, therefore, the light coupled into the waveguide by the first coupling optical element that reaches the at least one camera can be of a different wavelength than the light coupled into the waveguide by the second optical element that reaches the at least one camera. Other configurations are possible.

[0285] Various variations are possible in the structure and design of the coupling optical element 2120 and the out-coupling optical element 2012. For example, the size and shape of the coupling optical element 2120 for coupling light into the waveguide 2102 and the out-coupling optical element 2112 for coupling light out of the waveguide 2102 to the camera, in particular the coupling area of ​​the coupling optical element 2120 and the size and shape of the out-coupling optical element 2112, can be varied. Coupling area as used herein refers to the area of ​​an optical element configured to receive light and couple the light into or out of a waveguide for use by the system, for example, to image (e.g., an eye or an environment in front of a user). In some implementations, the coupling area may correspond to the area of ​​the optical element in its entirety or a portion of the area of ​​the optical element configured to be used in the system to couple light into or out of a waveguide for use by the system. For example, in some implementations, including a diffractive optical element used to couple light (from the eye or the environment in front of the user) into the waveguide, the coupling area of ​​the coupling element is the area of ​​the diffractive optical element used by the system to couple the light into the waveguide. The coupling optical element, e.g., a diffraction grating or diffractive optical element, may have a larger coupling area in some cases. For example, an opaque object, such as an opaque element or layer having a small aperture therein, may impede the propagation of light to a portion of the diffraction grating or diffractive optical element, thereby reducing the coupling area. Thus, the size and shape of the coupling area may be controlled by the size and perimeter of the optical element and other optical elements or other components or features that may block the light from reaching a portion of the optical element (e.g., the spatial extent of the diffractive optical element or grating). Other factors may also potentially reduce, modify, or otherwise affect the size and shape of the coupling area.

[0286] 28A and 28B show a coupling optical element 2120 for coupling light into the waveguide 2102, where the coupling optical element has a reduced size. In particular, the coupling optical element 2120 has a pinhole-sized coupling area. The size of the coupling area may be, for example, about 1.5 mm by 1.5 mm. As illustrated in FIG. 28A, for example, the coupling area of ​​the coupling optical element 2120 may have a length L and a thickness T. Both the length and thickness may be about 1.5 mm in some implementations. The reduced size of the coupling area eliminates multiple (e.g., residual) images from being collected by the coupling optical element 2120. The pinhole-sized coupling area of ​​the coupling optical element 2120 for coupling light from an object such as a user's eye or an object in the environment in front of the user and eyewear has an effect similar to that of a pinhole camera in terms of light collection and resulting imaging. The reduced coupling area size of the coupling optical element 2120 is similar to the reduced size of a pinhole camera coupling area. As a result, a large depth of focus or field is provided without the need for lenses. The combining optical element 2120 also need not have any optical power, nor need lenses be provided in the combining optical element 2120. Nevertheless, a large range of object distances are in focus with such a design.

[0287] The out-coupling optical element 2112 shown in Figures 28A and 28B for coupling light out of the waveguide 2102 to the camera is also reduced in size. In this particular implementation, the size and shape of the coupling optical element 2120 and the out-coupling optical element 2112 are similar or identical. The out-coupling optical element 2112 may also be 1.5 x 1.5 mm.

[0288] As discussed above, the combining optical element 2120 and the out-coupling optical element 2112 may comprise diffractive optical elements, such as diffraction gratings. The combining optical element 2120 and the out-coupling optical element 2112 may comprise hologram or holographic optical elements. As discussed herein, the combining optical element 2120 and the out-coupling optical element 2112 may comprise liquid crystals, liquid crystal gratings, polarization gratings, and / or liquid crystal polarization gratings. The combining optical element 2120 may be configured to receive light (e.g., from a user's eye or an environment in front of the user) and couple at least a portion of the light into the waveguide 2102. The combining optical element 2120 may also be configured to redirect and direct a portion of the light to the out-coupling optical element 2112. The out-coupling optical element 2112 may be configured to couple light received from the combining optical element 2120 and guided within the waveguide 2102 out of the waveguide, for example, to a camera. In the configuration shown, no focusing optical elements as described above are included.

[0289] The size and shape of the coupling areas of the coupling optical element 2120 and the external coupling optical element 2112 may vary. For example, while the coupling optical element 2120 and the external coupling optical element 2112 are shown as square, the shape of one or both may be different. In some implementations, the respective sizes and relative placement of the optical elements 2120, 2112 may be selected based at least in part on the desired distance from the user's eye (i.e., focal length), the wavelength of light to be captured, or both. The desired distance between the waveguide and the user's eye may be, for example, about 15 mm to 25 mm. Other configurations are possible. In some embodiments, the optical elements 2120, 2112 may be positioned, for example, 15 mm to 25 mm apart from each other on or within the waveguide. The shape of the coupling areas may be, for example, circular or otherwise rounded. The size of the coupling areas may also vary. For example, a dimension (e.g., length L or thickness T) along one direction of the bond area (e.g., average or maximum value) may be 0.1-3 mm, 0.1-0.3 mm, 0.1-0.5 mm, 0.5-3 mm, 0.5-2 mm, 1-3 mm, 1-2 mm, 0.5-2.5 mm, 1.0 mm-2.5 mm, 1.0 mm-1.5 mm, or 1.5 mm-2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. Similarly, a dimension (e.g., length L or thickness T) along another direction of the bond area (e.g., maximum or average value) may be 0.1-3 mm, 0.1-0.3 mm, 0.1-0.5 mm, 0.5-3 mm, 0.5-2 mm, 1-3 mm, 1-2 mm, 0.5-2.5 mm, 1.0-2.5 mm, 1.0 mm-1.5 mm, or 1.5 mm-2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. These two directions may be orthogonal in some implementations. The dimensions along the two directions need not be identical. Thus, the bond area may be symmetrical or asymmetrical.For example, the aspect ratio of the bond area (e.g., as measured by the ratio of length to thickness, such as maximum length to maximum thickness or average length to average thickness) may be 1 to 2, 1 to 1.75, 1 to 1.5, 1 to 1.3, 1 to 1.2, or 1 to 1.1, or any range formed by any of these values. Values ​​outside these ranges are also possible.

[0290] As discussed above, other shapes are possible. For example, Figures 29A and 29B show a coupling optical element 2120 for coupling light into the waveguide 2102 and an out-coupling optical element 2112 for coupling light out of the waveguide to a camera, where the coupling optical element 2120 has an arc-slit shaped coupling area. (The out-coupling optical element 2112 has a pinhole coupling area similar to that shown in Figures 28A and 28B.)

[0291] A slit may be advantageous in that it is larger than a pinhole, and therefore may collect more light. However, the narrowness of the slit provides a similar effect to a pinhole. For example, the reduced thickness T of the coupling area may eliminate multiple images from being collected by the coupling optical element 2120. Afterimages and / or blurring may thereby be reduced. As illustrated in FIG. 25A, the coupling area of ​​the coupling optical element 2120 may have a length L and a thickness T. The thickness T may be about 1.0 mm or 1.5 mm in some implementations. The pinhole-sized thickness T of the coupling optical element 2120 for coupling light from an object such as an eye or an object in the environment in front of the user and eyewear has an effect similar to that of a pinhole camera in terms of light collection and resulting imaging. The reduced size of the coupling area of ​​the coupling optical element 2120 is similar to the reduced size of a pinhole camera coupling area. As a result, a large depth of focus or depth of field is provided without the need for a lens. The combining optical element 2120 also need not have any refractive power, nor need a lens be provided in the combining optical element 2120. Nevertheless, a large range of object distances are in focus with such a design.

[0292] The out-coupling optical element 2112 shown in Figures 29A and 29B for coupling light out of the waveguide to the camera is also reduced in size. In this particular implementation, the size and shape of the out-coupling optical element 2112 is similar or identical to that described above in connection with Figures 28A and 28B. The out-coupling optical element 2112 may also be, for example, 1.5 x 1.5 mm.

[0293] As discussed above, the coupling optical element 2120 and the out-coupling optical element 2112 may comprise diffractive optical elements, such as diffraction gratings. The coupling optical element 2120 and the out-coupling optical element 2112 may comprise hologram or holographic optical elements. As discussed herein, the coupling optical element 2120 and the out-coupling optical element 2112 may comprise liquid crystals, liquid crystal gratings, polarization gratings, liquid crystal polarization gratings, or any combination thereof. The coupling optical element 2120 may be configured to receive light (e.g., from a user's eye or an environment in front of the user) and couple at least a portion of the light into the waveguide 2102. The coupling optical element 2120 may also be configured to redirect and direct a portion of the light to the out-coupling optical element 2112. The out-coupling optical element 2112 may be configured to couple light received from the coupling optical element 2120 and guided within the waveguide 2102 to the out-coupling optical element 2112 out of the waveguide, for example, to a camera. In the configuration shown, collection optics as described above are not included.

[0294] The shapes of the coupling areas of the coupling optical element 2120 and the out-coupling optical element 2112 may vary. In theory, a curved slit may potentially facilitate directing light from the coupling optical element 2120 to the out-coupling optical element. Furthermore, in some implementations, the arc-shaped slit may have a curvature described by a radius of curvature and a center of curvature. The curvature of the slit may be such that the out-coupling optical element 2112 is at the center of curvature of the arc-shaped slit. However, the curvature may be different, e.g., larger or smaller. Furthermore, other variations in shape are also possible. For example, the edges of the arc-shaped slit may be rounded.

[0295] The size of the bond area may also vary. For example, the smaller dimension along one direction of the bond area, i.e., thickness T, may be 0.1-0.3 mm, 0.1-0.5 mm, 0.3-0.5 mm, 0.2-0.5 mm, 0.1-1.0 mm, 0.3-1.0, 0.5-3 mm, 0.5-2 mm, 1-3 mm, 1-2 mm, 0.5-2.5 mm, 1.0-2.5 mm, 1.0 mm-1.5 mm, or 1.5 mm-2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. This dimension, i.e., thickness T, may be smaller than the other dimension, i.e., length L. The other dimension, i.e., length L, may correspond to the length from one end to the other or the resulting displacement. Alternatively, the path length P, in this case the arc length, may be used to provide a measure of the larger dimension of the bond area. In some implementations, the length L or path length or arc length P may be 5 mm to 40 mm, 10 to 40 mm, 10 to 30 mm, 15 to 30 mm, 15 to 25 mm, 1 to 5 mm, or 3 to 5 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. These two directions may be orthogonal in some implementations, however the directions need not be orthogonal. The slit is asymmetric. For example, the aspect ratio of the bonded area (e.g., measured by the ratio of the length L to the thickness, where the length can be either the resulting displacement from one end to the other or the path length, such as the arc length P) may be 5 to 100, 10 to 100, 15 to 100, 20 to 100, 10 to 40, or 10 to 50, or any range formed by any of these values. Values ​​outside these ranges are also possible. In some embodiments, a dimension along one direction of the coupling area of ​​the coupling optical element 2120 may be less than or equal to 2.5% of the distance between the center of the coupling optical element 2120 and the center of the external coupling optical element 2112. As an example, the center-to-center distance between the optical elements 2120, 2112 may be about 20 mm and the dimension (e.g., thickness T) may be less than or equal to about 0.5 mm (e.g., 2.5% of 20 mm). Other configurations or values ​​are also possible.For example, the dimension (e.g., thickness T) may be less than or equal to 2%, 1.5%, and / or 1% of the distance between the center of the coupling optical element 2120 and the center of the external coupling optical element 2112. The dimensions, referred to herein as thickness T, length L, path length P, etc., may be single measurements, average values, maximum values, or minimum values.

[0296] The shape and size of the external coupling optical element 2112 may also vary. For example, although the external coupling optical element 2112 is shown as a square, the shape may be different. The shape of the coupling area may be, for example, circular or otherwise rounded. The size of the coupling area may also vary. For example, the dimension along one direction of the coupling area, e.g., the thickness T, may be 0.1-0.3 mm, 0.1-0.5 mm, 0.3-0.5 mm, 0.5-3 mm, 0.5-2 mm, 1-3 mm, 1-2 mm, 0.5-2.5 mm, 1.0-2.5 mm, 1.0 mm-1.5 mm, or 1.5 mm-2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. Similarly, a dimension along another direction of the bond area, for example the length L, may be 0.1-0.3 mm, 0.1-0.5 mm, 0.3-0.5 mm, 0.5-3 mm, 0.5-2 mm, 1-3 mm, 1-2 mm, 0.5-2.5 mm, 1.0-2.5 mm, 1.0 mm-1.5 mm, or 1.5 mm-2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. These two directions may be orthogonal in some implementations. The dimensions along the two directions need not be identical. Thus, the bond area may be symmetrical or asymmetrical. For example, the aspect ratio of the bond area (e.g., measured by the ratio of length L to thickness T, where length can be either the resulting displacement from one end to the other or the path length, such as the arc length P) may be 1-2, 1-1.75, 1-1.5, 1-1.3, 1-1.2, or 1-1.1, or any range formed by any of these values. Values ​​outside these ranges are also possible. As noted above, dimensions referred to herein as thickness T, length L, path length P, etc., may be single measurements, averages, maximum values, or minimum values.

[0297] As discussed above, different shapes and sizes of coupling areas are also possible. Figures 30A and 30B show a coupling optical element 2120 for coupling light into the waveguide 2102 having a non-arcuate slit shaped coupling area. In particular, the coupling optical element 2120 has a straight slit coupling area. The slit is rectangular shaped. In contrast, the external coupling optical element 2112 comprises a pinhole size coupling area similar to that shown in Figures 28A and 28B and Figures 29A and 29B.

[0298] As discussed above, a slit is advantageous in that it is larger than a pinhole and therefore may collect more light. However, the narrowness of the slit provides a similar effect to a pinhole. For example, the reduced thickness T of the coupling area may eliminate multiple images from being collected by the coupling optical element 2120. Image remnants and / or blurring may thereby be reduced. As illustrated in FIG. 30A, the coupling area of ​​the coupling optical element 2120 may have a length L and a thickness T. The thickness T may be about 1.0 mm or 1.5 mm in some implementations. The pinhole-sized thickness T of the coupling optical element 2120 for coupling light from an object such as an eye or an object in the environment in front of the user and eyewear has an effect similar to that of a pinhole camera in terms of light collection and resulting imaging. The reduced size of the coupling area of ​​the coupling optical element 2120 is similar to the reduced size of a pinhole camera aperture. As a result, a large depth of focus or depth of field is provided without the need for a lens. The combining optical element 2120 also need not have any refractive power, nor need a lens be provided in the combining optical element 2120. Nevertheless, a large range of object distances are in focus with such a design.

[0299] The out-coupling optical element 2112 shown in Figures 30A and 30B for coupling light out of the waveguide to the camera is also reduced in size. In this particular implementation, the size and shape of the out-coupling optical element 2112 is similar or identical to that described above in connection with Figures 28A and 28B and 29A and 29B. The out-coupling optical element 2112 may also be, for example, 1.5 x 1.5 mm.

[0300] As discussed above, the coupling optical element 2120 and the out-coupling optical element 2112 may comprise diffractive optical elements, such as diffraction gratings. The coupling optical element 2120 and the out-coupling optical element 2112 may comprise hologram or holographic optical elements. As discussed herein, the coupling optical element 2120 and the out-coupling optical element 2112 may comprise liquid crystals, liquid crystal gratings, polarization gratings, liquid crystal polarization gratings, or any combination thereof. The coupling optical element 2120 may be configured to receive light (e.g., from a user's eye or an environment in front of the user) and couple at least a portion of the light into the waveguide 2102. The coupling optical element 2120 may also be configured to redirect and direct a portion of the light to the out-coupling optical element 2112. The out-coupling optical element 2112 may be configured to couple light received from the coupling optical element 2120 and guided within the waveguide 2102 to the out-coupling optical element 2112 out of the waveguide, for example, to a camera. In the configuration shown, collection optics as described above are not included.

[0301] As shown, the shapes of the coupling areas of the coupling optical element 2120 and the out-coupling optical element 2112 may vary. As discussed above, in theory, a curved slit could potentially facilitate directing light from the coupling optical element 2120 to the out-coupling optical element. However, a non-arcuate slit could also be used. A straight slit or a rectangular slit as shown in FIGS. 30A and 30B could direct enough light to the out-coupling optical element 2112 to capture an image using a camera. However, other variations in shape are also possible. For example, the edges of the slit could be rounded.

[0302] The size of the bond area may also vary. For example, the smaller dimension along one direction of the bond area, i.e., thickness T, may be 0.1-0.3 mm, 0.1-0.5 mm, 0.3-0.5 mm, 0.5-3 mm, 0.5-2 mm, 1-3 mm, 1-2 mm, 0.5-2.5 mm, 1.0-2.5 mm, 1.0 mm-1.5 mm, or 1.5 mm-2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. This dimension, i.e., thickness T, may be smaller than the other dimension, i.e., length L. The other dimension, i.e., length L, may correspond to the length from one end to the other or the resulting displacement. As discussed above, alternatively, the path length P may be used to provide a measure of the larger dimension of the bond area. In this example, if the slit is a straight line, the length L as measured by the displacement from one end to the other is the same as the path length P. In some implementations, the length L or path length or arc length P may be 1-5 mm, 1-3 mm, 5 mm-40 mm, 10-40 mm, 10-30 mm, 15-30 mm, or 15-25 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. As discussed, these two directions may be orthogonal in some implementations, such as for a straight or rectangular slit, however, the directions need not be orthogonal. The slit is asymmetric. For example, the aspect ratio of the bonded area (e.g., measured by the ratio of the length L to the thickness T, where the length can be either the resulting displacement from one end to the other or the path length, such as the arc length P) may be 5-100, 10-100, 15-100, 20-100, 10-40, or 10-50, or any range formed by any of these values. Values ​​outside these ranges are also possible. As noted above, dimensions, referred to herein as thickness T, length L, path length P, etc., may be single measurements, average values, maximum values, or minimum values.

[0303] The shape and size of the external coupling optical element 2112 may also vary. For example, while the external coupling optical element 2112 is shown as a square, one or both shapes may be different. The shape of the bonding area may be, for example, circular or otherwise rounded. The size of the bonding area may also vary. For example, the dimension along one direction of the bonding area, e.g., the thickness T, may be 0.1-0.3 mm, 0.1-0.5 mm, 0.3-0.5 mm, 0.5-3 mm, 0.5-2 mm, 1-3 mm, 1-2 mm, 0.5-2.5 mm, 1.0 mm-2.5 mm, 1.0 mm-1.5 mm, or 1.5 mm-2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. Similarly, a dimension along another direction of the bond area, for example the length L, may be 0.1-0.3 mm, 0.1-0.5 mm, 0.3-0.5 mm, 0.5-3 mm, 0.5-2 mm, 1-3 mm, 1-2 mm, 0.5-2.5 mm, 1.0-2.5 mm, 1.0 mm-1.5 mm, or 1.5 mm-2 mm, or any range formed by any of these values. Values ​​outside these ranges are also possible. These two directions may be orthogonal in some implementations. The dimensions along the two directions need not be identical. Thus, the bond area may be symmetrical or asymmetrical. For example, the aspect ratio of the bond area (e.g., measured by the ratio of length L to thickness T, where length can be either the resulting displacement from one end to the other or the path length, such as the arc length P) may be 1-2, 1-1.75, 1-1.5, 1-1.3, 1-1.2, 1-1.1, or any range formed by any of these values. Values ​​outside these ranges are also possible. Dimensions, referred to herein as thickness T, length L, path length P, etc., may be single measurements, averages, maximum values, or minimum values ​​(e.g., average thickness, maximum thickness, average length, maximum length).

[0304] FIG. 31 shows an eyepiece including a coupling optical element 2112 for coupling light into a waveguide 2102 and an out-coupling optical element 2120 for coupling light out of the waveguide 2102 to a camera, the coupling optical element 2112 having a non-arcuate (e.g., rectilinear rectangular) slit-shaped coupling area included with an image content in-coupling optical element 2104 for receiving light from an image projector, a light distribution element 2108 for directing light from the image content in-coupling optical element 2104 to the out-coupling optical element 2110, and the out-coupling optical element 2110 for coupling light guided in the waveguide to a user for viewing the image content. In some implementations, the coupling optical element 2112 has a lens associated therewith that includes or provides a refractive power. This refractive power may collimate light from an object at a particular distance, such as from the anterior surface of the eye (e.g., the cornea), to facilitate image capture of the object (e.g., a phosphene on the cornea). The optical power of the first coupling optical element 2112a and / or the lens associated therewith may correspond to a focal length in the range of approximately 15-25 mm, 10-40 mm, 10-50 mm, 5-40 mm, or 5-50 mm, or any range between any of the distance values ​​herein. Values ​​outside these ranges are also possible.

[0305] The slit coupling optical element 2112 is laterally displaced from the outcoupling optical element 2110. In other embodiments, such as that shown in Figures 30A and 30B, the system 900 need not be included on the eyepiece with such components for presenting an image to a user.

[0306] 32A and 32B show a first and a second coupling optical element 2112a, 2112b for coupling light into the waveguide 2102 and a first and a second out-coupling optical element 2120a, 2120b for coupling light out of the waveguide to a camera, where the coupling optical elements 2112a and 2112b have a non-arc (e.g., straight rectangular) slit-shaped coupling area. Such a configuration may be useful for imaging different objects simultaneously. In some implementations, different parts of the eye, such as the retina and the cornea (or the photinia thereon), can be imaged using a pair of coupling optical elements 2112a, 2112b, a pair of out-coupling optical elements 2120a, 2120b, and one or more cameras. In some implementations, the first combining optical element 2112a has a refractive power or is associated with a lens, while the second combining optical element 2112b does not have a similar refractive power or a similar lens (e.g., has no refractive power or lens associated with it). In some implementations, the second combining optical element 2112b may have a refractive power and / or a lens associated with it, however, the total refractive power of the second combining optical element 2112b and / or any lens associated with the second combining optical element is less than the refractive power of the first combining optical element and / or any lens associated with the first combining optical element. As discussed above, such a system 900 may be configured to image, for example, the anterior portion of the eye using the first combining optical element 2112a and the retina using the second combining optical element 2112b. As discussed above, in some implementations, the eye is illuminated with light having a first polarization and the second outcoupling optical element 2120b has a polarizer associated therewith that is polarization selective or configured to filter out light of the first polarization so as to remove reflections from the cornea, such as glints, so as not to degrade the image at the retina.

[0307] 33 shows first and second coupling optical elements 2112a, 2112b for coupling light into the waveguide 2102 and first and second out-coupling optical elements 2120a, 2120b for coupling light out of the waveguide 2102 to a camera, where the coupling optical elements 2112a and 2112b have a non-arc (e.g., straight rectangular) slit-shaped coupling area integrated on the eyepiece. The eyepiece further includes an image content in-coupling optical element 2104 for receiving light from an image projector and a light distribution element 2108 for directing light from the in-coupling optical element 2104 to an out-coupling optical element 2110 for coupling light guided in the waveguide 2102 to a user for viewing the image content. In other embodiments, such as that shown in Figures 32A and 32B, the system 900 need not be included on a waveguide with such components 2104, 2108, 2110 to present an image to a user.

[0308] 34A and 34B show a first and second coupling optical element 2112a, 2112b for coupling light into the waveguide 2102 and a first and second out-coupling optical element 2120a, 2120b for coupling light out of the waveguide to a camera, where the coupling optical element 2112a, 2112b has a non-arc (e.g., straight rectangular) slit-shaped coupling area. Such a configuration may be useful for simultaneously imaging different objects. In some implementations, different parts of the eye, such as the retina and cornea (or photinia thereon), can be imaged using a pair of coupling optical elements 2112a, 2112b, a pair of out-coupling optical elements 2120a, 2120b, and one or more cameras. In some implementations, the first combining optical element 2112a has a refractive power or is associated with a lens, while the second combining optical element 2112b does not have a similar refractive power or a similar lens (e.g., does not have a refractive power and a lens associated with it). In particular, as discussed above, in some implementations, the second combining optical element 2112b may have a refractive power and / or a lens associated with it, however, the total refractive power of the second combining optical element 2112b and / or any lens associated with the second combining optical element is less than the refractive power of the first combining optical element and / or any lens associated with the first combining optical element. The optical power of the first coupling optical element 2112a and / or the lens associated therewith may correspond to a focal length in the range of approximately 15-25 mm, 10-30 mm, 10-40 mm, 10-50 mm, 5-40 mm, or 5-50 mm, or any range between any of the distance values ​​herein. Values ​​outside these ranges are also possible.

[0309] As discussed above, such a system 900 may be configured, for example, to image the anterior portion of the eye using the first coupling optical element 2112a and the retina using the second coupling optical element 2112b. As discussed above, in some implementations, the eye is illuminated with light having a first polarization, and the second out-coupling optical element 2120b has a polarizer associated therewith that is configured to be polarization selective, i.e., to filter out light of the first polarization so as to remove reflections from the cornea, such as glints, so as not to degrade the image of the retina. The arrangements shown in Figures 34...

Claims

1. A head mounted display system configured to display augmented reality image content within a user's field of view by projecting light onto the user's eye, The head mounted display system comprises: a frame configured to be supported on a head of the user; and an image projector configured to display image content within a field of view of the user by projecting an image into the user's eye; a transparent layer supported on the frame, the transparent layer positioned at a location in front of the user's eye when the user is wearing the head mounted display system, the transparent layer providing a view of the user's environment in front of the user by transmitting light from the environment in front of the user to the user's eye, the transparent layer having front and back major surfaces surrounded by a plurality of edges, the transparent layer further comprising at least one sloped surface, the sloped surface being angled such that light guided within the transparent layer is reflected from the sloped surface and directed out of the transparent layer through the back major surface and towards the eye; at least one camera positioned to receive at least a portion of the light reflected from the user's eye to capture an image using the light reflected from the user's eye; A head mounted display system comprising:

2. The system of claim 1, wherein the inclined surface comprises one or more of a metallic coating or an IR reflective coating.

3. The system described in claim 1, wherein the inclined surface is angled at 35° to 55° relative to the front major surface and / or the back major surface of the transparent layer.

4. The system of claim 1, further comprising at least one light source positioned relative to the transparent layer, wherein light from the at least one light source is injected into the transparent layer, impinges on the inclined surface, and is reflected from the inclined surface toward the user's eye.

5. The system of claim 1, further comprising at least one light source positioned against at least one edge of the transparent layer, wherein light from the at least one light source is injected into the transparent layer, impinges on the inclined surface, and is reflected from the inclined surface toward the user's eye.

6. The system of claim 5, wherein the at least one light source comprises a plurality of spaced apart light sources or one or more infrared light sources.

7. The system described in claim 5, wherein the at least one light source creates a point source that illuminates the user's eye.

8. The system of claim 5, wherein the light source comprises one or more light emitting diodes (LEDs) or lasers.

9. The system of claim 5, wherein the light from the light source is guided within the transparent layer by total internal reflection from the front and back major surfaces of the transparent layer.

10. The system of claim 1, wherein the transparent layer comprises plastic.

11. The system described in claim 1, further comprising an eyepiece disposed on the frame, the eyepiece configured to display augmented reality image content in the user's field of view by directing light into the user's eye, at least a portion of the eyepiece being transparent and positioned in a location in front of the user's eye when the user is wearing the head-mounted display system, the transparent portion providing a view of the environment in front of the user by allowing light from the environment in front of the user to pass through to the user's eye.

12. The system described in claim 11, wherein the eyepiece is configured to receive light from the image projector and direct the light into the user's eye to display augmented reality image content in the user's field of view.

13. The system described in claim 11, wherein the eyepiece lens comprises at least one waveguide.

14. The system of claim 13, wherein the image projector is configured to direct light into the at least one waveguide.

15. The system of claim 13, wherein the at least one waveguide comprises a stack of the waveguides.

16. The system described in claim 15, wherein different waveguides of the waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eye.

17. The system of claim 15, wherein different waveguides of the waveguide stack are configured to output light with different colors.

18. The system described in claim 15, wherein the transparent layer comprises a layer within the eyepiece lens.

19. The system comprising: At least one waveguide; at least one coupling optical element, the at least one coupling optical element configured such that light output from the transparent layer that is reflected from the eye is coupled into and guided within the waveguide; at least one external coupling element configured to couple light guided within the waveguide out of the waveguide and direct the light to the camera to capture an image of the eye; The system of claim 1 further comprising:

20. The system of claim 1, wherein the system is configured to perform eye tracking based on an image of the eye captured by the camera.

Images