Eye imaging system with embedded prism

Abstract

An optics assembly includes an eyepiece operable to receive virtual image light and project the virtual image light toward an eye side of the optics assembly. The optics assembly includes a rear lens disposed between the eyepiece and the eye side of the optics assembly. The optics assembly includes one or more wedge prisms integrated with the rear lens along a periphery of the rear lens. The optics assembly includes an eye tracking camera disposed outside the periphery of the rear lens in alignment with the one or more wedge prisms and operable to capture light that propagates from the eye side of the optics assembly and is refracted through the one or more wedge prisms.

Description

EYE IMAGING SYSTEM WITH EMBEDDED PRISMBACKGROUND OF THE INVENTION

[0001] Modem computing and display technologies have facilitated the development of systems for so called "virtual reality" or "augmented reality" experiences, wherein digitally reproduced images or portions thereof are presented to a viewer in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or "VR," scenario typically involves the presentation of digital or virtual image information without transparency to other actual real -world visual input; an augmented reality, or "AR," scenario typically involves the presentation of digital or virtual image information as an augmentation to visualization of the actual world around the viewer.

[0002] Referring to FIG. 1, an augmented reality scene 100 is depicted. The user of an AR technology sees a real -world park-like setting 106 featuring people, trees, buildings in the background, and a real-world concrete platform 120. The user also perceives that he / she "sees" "virtual content" such as a robot statue 110 standing upon the real -world concrete platform 120, and a flying cartoon-like avatar character 102 which seems to be a personification of a bumble bee. The robot statue 110 and the flying cartoon-like avatar character 102 are "virtual" in that they do not exist in the real world. Because the human visual perception system is complex, it is challenging to produce AR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements.

[0003] Despite the progress made in these display technologies, there is a need in the art for improved methods and systems related to augmented reality systems, particularly, display systems.SUMMARY OF THE INVENTION

[0004] The present invention relates generally to methods and systems related to projection display systems including wearable displays. More particularly, embodiments of the present invention provide improved optics assemblies useful for eye tracking in augmented reality(AR) devices. Merely by way of example, some embodiments of the invention relate to the use of a wedge prism integrated in the rear Extended Depth Of Field (EDOF) lens of an optics assembly, allowing the body of the eye tracking camera to be shifted further out of the user’s field of view thereby minimizing the impact on the real -world view.

[0005] A summary of the various embodiments of the invention is provided below as a list of examples. As used below, any reference to a series of examples is to be understood as a reference to each of those examples disjunctively (e.g., "Examples 1-4" is to be understood as "Examples 1, 2, 3, or 4").

[0006] Example 1 is an optics assembly comprising: an eyepiece operable to receive virtual image light and project the virtual image light toward an eye side of the optics assembly; a rear lens disposed between the eyepiece and the eye side of the optics assembly; one or more wedge prisms integrated with the rear lens along a periphery of the rear lens; and an eye tracking camera disposed outside the periphery of the rear lens in alignment with the one or more wedge prisms and operable to capture light that propagates from the eye side of the optics assembly and is refracted through the one or more wedge prisms.

[0007] Example 2 is the optics assembly of example(s) 1, further comprising: an illumination source disposed along the periphery of the rear lens in alignment with the one or more wedge prisms and operable to emit illumination light that is refracted through the one or more wedge prisms toward the eye side of the optics assembly, wherein the light captured by the eye tracking camera includes the illumination light.

[0008] Example 3 is the optics assembly of example(s) 2, wherein one or more wedge prisms include: a first wedge prism operable to receive the illumination light from the illumination source and reflect the illumination light toward the eye side of the optics assembly; and a second wedge prism operable to receive the light that propagates from the eye side of the optics assembly and reflect the light toward the eye tracking camera.

[0009] Example 4 is the optics assembly of example(s) 3, further comprising: a third wedge prism integrated with the rear lens along the periphery of the rear lens; and a second eye tracking camera disposed outside the periphery of the rear lens in alignment with the third wedge prism and operable to capture light that propagates from the eye side of the optics assembly and is refracted through the third wedge prism, wherein the eye tracking camera is a first eye tracking camera.

[0010] Example 5 is the optics assembly of example(s) 4, further comprising: a fourth wedge prism integrated with the rear lens along the periphery of the rear lens; and a second illumination source disposed along the periphery of the rear lens in alignment with the fourth wedge prism and operable to emit second illumination light that is refracted through the fourth wedge prism toward the eye side of the optics assembly, wherein the illumination light is first illumination light, and wherein the light captured by the first eye tracking camera and the light captured by the second eye tracking camera include the first illumination light and the second illumination light.

[0011] Example 6 is the optics assembly of example(s) 1-5, wherein each wedge prism of one or more wedge prisms comprise: a first surface on an eye side of the wedge prism; a second surface on an inward radial direction of the wedge prism, the second surface being reflective, the second surface being adjacent to the first surface; and a third surface on an outward radial direction of the wedge prism, the third surface being adjacent to the first surface.

[0012] Example 7 is the optics assembly of example(s) 6, wherein the second surface is adjacent to the third surface such that the wedge prism has a triangular shape.

[0013] Example 8 is the optics assembly of example(s) 6, wherein each wedge prism of one or more wedge prisms further comprises: a fourth surface on a world side of the wedge prism, the fourth surface being adjacent to the second surface and the third surface.

[0014] Example 9 is the optics assembly of example(s) 6, wherein the second surface the second surface comprises a reflective coating.

[0015] Example 10 is the optics assembly of example(s) 6, wherein the second surface the second surface comprises a diffraction grating characterized by grating phase parameters.

[0016] Example 11 is the optics assembly of example(s) 1, wherein one or more wedge prisms comprise a ring wedge prism that is integrated with an entirety of the periphery of the rear lens.

[0017] Example 12 is the optics assembly of example(s) 1-11, wherein the rear lens is an Extended Depth Of Field (EDOF) lens.

[0018] Example 13 is an augmented reality (AR) system comprising: a frame; a set of projectors coupled to the frame and configured to generate virtual image light; a set of eyetracking cameras coupled to the frame; a set of optical stacks coupled to the frame, each of the set of optical stacks comprising: an eyepiece operable to receive the virtual image light and project the virtual image light toward an eye side of the AR system; a rear lens disposed between the eyepiece and the eye side of the AR system; a wedge prism integrated with the rear lens along a periphery of the rear lens and operable to receive light that propagates from the eye side of the AR system and reflect the light in an outward radial direction toward one of the set of eye tracking cameras.

[0019] Example 14 is the AR system of example(s) 13, wherein each of the set of optical stacks further comprises: an illumination source disposed along the periphery of the rear lens and operable to emit illumination light that is refracted through either the wedge prism or a second wedge prism toward the eye side of the AR system, wherein the light captured by one of the set of eye tracking cameras includes the illumination light.

[0020] Example 15 is the AR system of example(s) 13-14, wherein the wedge prism comprises: a first surface on an eye side of the wedge prism; a second surface on an inward radial direction of the wedge prism, the second surface being reflective, the second surface being adjacent to the first surface; and a third surface on an outward radial direction of the wedge prism, the third surface being adjacent to the first surface.

[0021] Example 16 is the AR system of example(s) 15, wherein the second surface is adjacent to the third surface such that the wedge prism has a triangular shape.

[0022] Example 17 is the AR system of example(s) 15, wherein the wedge prism further comprises: a fourth surface on a world side of the wedge prism, the fourth surface being adjacent to the second surface and the third surface.

[0023] Example 18 is a method of operating an optics assembly, the method comprising: receiving virtual image light at an eyepiece; projecting, by the eyepiece, the virtual image light toward an eye side of the optics assembly; passing the virtual image light through a rear lens disposed between the eyepiece and the eye side of the optics assembly; generating, by an illumination source disposed outside a periphery of the rear lens, illumination light that is coupled into one or more wedge prisms, wherein the one or more wedge prisms are integrated with the rear lens along the periphery of the rear lens; reflecting, by the one or more wedge prisms, the illumination light toward the eye side of the optics assembly; reflecting, by the one or more wedge prisms, the illumination light that propagates from the eye side of the optics assembly toward an outward radial direction; and capturing the illumination light by aneye tracking camera disposed outside the periphery of the rear lens in alignment with the one or more wedge prisms.

[0024] Example 19 is the method of example(s) 18, wherein one or more wedge prisms include a first wedge prism that receives the illumination light from the illumination source and reflects the illumination light toward the eye side of the optics assembly and a second wedge prism that receives the illumination light that propagates from the eye side of the optics assembly and reflects the illumination light toward the outward radial direction.

[0025] Example 20 is the method of example(s) 18-19, wherein one or more wedge prisms comprise a ring wedge prism that receives the illumination light from the illumination source, reflects the illumination light toward the eye side of the optics assembly, receives the illumination light that propagates from the eye side of the optics assembly, and reflects the illumination light toward the outward radial direction.BRIEF DESCRIPTION OF THE DRAWINGS

[0026] FIG. 1 illustrates a user's view of augmented reality (AR) through an AR device.

[0027] FIG. 2A illustrates a cross-sectional, side view of an example of a set of stacked waveguides that each includes an incoupling optical element.

[0028] FIG. 2B illustrates a perspective view of an example of one or more stacked waveguides of FIG. 2A.

[0029] FIG. 2C illustrates a top-down, plan view of an example of one or more stacked waveguides of FIGS. 2 A and 2B.

[0030] FIG. 3 is a simplified illustration of an eyepiece waveguide having a combined pupil expander.

[0031] FIG. 4 illustrates an example of wearable display system.

[0032] FIG. 5 shows a perspective view of a wearable device.

[0033] FIG. 6 is a simplified plan view of elements of an AR headset including eye tracking.

[0034] FIG. 7 illustrates a cross-sectional, side view of an example of a rear EDOF and ET structure.

[0035] FIG. 8 illustrates a cross-sectional, side view of an example of a rear EDOF and ET structure.

[0036] FIG. 9 illustrates a cross-sectional, side view of an example of a rear EDOF and ET structure.

[0037] FIG. 10 illustrates a perspective view of an example of a rear EDOF and ET structure.

[0038] FIG. 11 illustrates a perspective view of an example of a rear EDOF and ET structure.

[0039] FIG. 12 illustrates a perspective view of an example of a rear EDOF and ET structure.

[0040] FIG. 13 illustrates a cross-sectional, side view of an example of a rear EDOF and ET structure.

[0041] FIGS. 14A and 14B show simulated results demonstrating the reduction of the pinhole error by using the diffraction grating described in FIG. 13.

[0042] FIGS. 15A and 15B show simulated results demonstrating the improvement in image quality using the wedge prism.

[0043] FIG. 16 is a simplified flowchart illustrating a method of operating an optics assembly.

[0044] FIG. 17 is a simplified block diagram illustrating components of an AR system.DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

[0045] The present invention relates generally to methods and systems related to projection display systems including wearable displays. More particularly, embodiments of the present invention provide improved optics assemblies useful for eye tracking in augmented reality (AR) devices. In some embodiments, an optics assembly may include a wedge prism integrated in the rear Extended Depth Of Field (EDOF) lens. This allows the body of the eye tracking camera to be shifted into the frame and further out of the user’s field of view thereby minimizing the impact on the real-world view. Additional wedge prisms may be used by illumination sources (e.g., light emitting diodes (LEDs)) that illuminate the user’s eye with glints (the spot reflections from the LEDs from the cornea when imaged by the eye trackingcamera). Placing the illumination sources in the frame further increases the real-world visibility and also eliminates the need for small, custom-designed LEDs that are placed on the display in the direct line of sight of the user.

[0046] As AR technology moves toward increased immersiveness and large real-world apertures and virtual field of view, direct eye illumination and imaging from LEDs and cameras on the frame of the AR device will become increasingly difficult due to the steep angles of the LEDs and cameras towards the eye. Embodiments of the present invention solve these and other issues through the use of aligned wedge prisms integrated with the rear EDOF lens that reflect light passing there through, thereby enabling eye tracking in highly immersive AR glasses. Each wedge prism may include an eye-side surface (or first surface) that has the same optical prescription as the eye-side surface of the rear EDOF lens to form one smooth surface, a second surface on an inward radial direction of the wedge prism that is reflective due to, e.g., a near infrared reflective coating / film or diffractive mirror that reflects light from the radial direction to the axial direction (or vice-versa), and a third surface on an outward radial direction of the wedge prism that receives illumination light when aligned with an illumination source or outputs light when aligned with an eye tracking camera.

[0047] Reference will now be made to the drawings, in which like reference numerals refer to like parts throughout. Unless indicated otherwise, the drawings are schematic not necessarily drawn to scale.

[0048] With reference now to FIG. 2A, in some embodiments, light impinging on a waveguide may need to be redirected to incouple that light into the waveguide. An incoupling optical element may be used to redirect and incouple the light into its corresponding waveguide. Although referred to as "incoupling optical element" through the specification, the incoupling optical element need not be an optical element and may be a non-optical element. FIG. 2A illustrates a cross-sectional, side view of an example of a set of stacked waveguides 200 that each includes an incoupling optical element. The waveguides may each be configured to output light of one or more different wavelengths, or one or more different ranges of wavelengths. Light from a projector is injected into the set of stacked waveguides 200 and outcoupled to a user as described more fully below.

[0049] The illustrated set of stacked waveguides 200 includes waveguide 202, waveguide 204, and waveguide 206. Each waveguide includes an associated incoupling optical element (which may also be referred to as a light input area on the waveguide), with, e.g., incouplingoptical element 203 disposed on a major surface (e.g., an upper major surface) of waveguide 202, incoupling optical element 205 disposed on a major surface (e.g., an upper major surface) of waveguide 204, and incoupling optical element 207 disposed on a major surface (e.g., an upper major surface) of waveguide 206. In some embodiments, one or more of the incoupling optical elements may be disposed on the bottom major surface of the respective waveguide (particularly where one or more incoupling optical elements are reflective, deflecting optical elements). As illustrated, the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 may be disposed on the upper major surface of waveguide 202, waveguide 204, and waveguide 206, respectively (or the top of the next lower waveguide), particularly where those incoupling optical elements are transmissive, deflecting optical elements. In some embodiments, the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 may be disposed in the body of the waveguide 202, waveguide 204, and waveguide 206, respectively. In some embodiments, as discussed herein, the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 are wavelength- selective, such that they selectively redirect one or more wavelengths of light, while transmitting other wavelengths of light. While illustrated on one side or corner of waveguide 202, waveguide 204, and waveguide 206, respectively, it will be appreciated that the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 may be disposed in other areas of waveguide 202, waveguide 204, and waveguide 206, respectively, in some embodiments.

[0050] As illustrated, the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 may be laterally offset from one another. In some embodiments, each incoupling optical element may be offset such that it receives light without that light passing through another incoupling optical element. For example, each of the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 may be configured to receive light from a different projector and may be separated (e.g., laterally spaced apart) from other incoupling optical elements such that it substantially does not receive light from the other ones of the incoupling optical elements.

[0051] Each waveguide also includes associated light distributing elements, with, e.g., light distributing elements 210 disposed on a major surface (e.g., a top major surface) of waveguide 202, light distributing elements 212 disposed on a major surface (e.g., a top major surface) of waveguide 204, and light distributing elements 214 disposed on a major surface(e.g., a top major surface) of waveguide 206. In some other embodiments, the light distributing elements 210, the light distributing elements 212, and the light distributing elements 214 may be disposed on a bottom major surface of associated waveguide 202, waveguide 204, and waveguide 206, respectively. In some other embodiments, the light distributing elements 210, the light distributing elements 212, and the light distributing elements 214 may be disposed on both top and bottom major surfaces of associated waveguide 202, waveguide 204, and waveguide 206, respectively; or the light distributing elements 210, the light distributing elements 212, and the light distributing elements 214 may be disposed on different ones of the top and bottom major surfaces in different associated waveguide 202, waveguide 204, and waveguide 206, respectively.

[0052] Waveguide 202, waveguide 204, and waveguide 206 may be spaced apart and separated by, e.g., gas, liquid, and / or solid layers of material. For example, as illustrated in FIG. 2A, layer 208 may separate waveguide 202 and waveguide 204 and layer 209 may separate waveguide 204 and waveguide 206. In some embodiments, layer 208 and layer 209 are formed of low refractive index materials (that is, materials having a lower refractive index than the material forming the immediately adjacent one of waveguide 202, waveguide 204, or waveguide 206). Preferably, the refractive index of the material forming layer 208 and / or layer 209 is 0.05 or more, or 0.10 or less than the refractive index of the material forming the waveguide 202, the waveguide 204, or the waveguide 206. Advantageously, layer 208 and layer 209 having the lower refractive index may function as cladding layers that facilitate total internal reflection (TIR) of light through the waveguide 202, the waveguide 204, and the waveguide 206 (e.g., TIR between the top and bottom major surfaces of each waveguide). In some embodiments, the layer 208 and the layer 209 are formed of air. While not illustrated, it will be appreciated that the top and bottom of the illustrated set of stacked waveguides 200 may include immediately neighboring cladding layers.

[0053] Preferably, for ease of manufacturing and other considerations, the material forming the waveguide 202, the waveguide 204, and the waveguide 206 are similar or the same, and the material forming the layer 208 and the layer 209 are similar or the same. In some embodiments, the material forming the waveguide 202, the waveguide 204, and the waveguide 206 may be different between one or more waveguides, and / or the material forming the layer 208 and the layer 209 may be different, while still holding to the various refractive index relationships noted above.

[0054] With continued reference to FIG. 2 A, light ray 218, light ray 219, and light ray 220 are incident on the set of stacked waveguides 200. It will be appreciated that the light ray 218, the light ray 219, and the light ray 220 may be injected into the waveguide 202, the waveguide 204, and the waveguide 206 by one or more projectors (not shown).

[0055] In some embodiments, light ray 218, the light ray 219, and the light ray 220 have different properties, e.g., different wavelengths or different ranges of wavelengths, which may correspond to different colors. The incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 each deflect the incident light such that the light propagates through a respective one of the waveguide 202, the waveguide 204, or the waveguide 206 by TIR. In some embodiments, the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 each selectively deflect one or more particular wavelengths of light, while transmitting other wavelengths to an underlying waveguide and associated incoupling optical element.

[0056] For example, incoupling optical element 203 may be configured to deflect light ray 218, which has a first wavelength or range of wavelengths, while transmitting light ray 219 and light ray 220, which have different second and third wavelengths or ranges of wavelengths, respectively. The light ray 219 transmitted through the waveguide 202 impinges on and is deflected by the incoupling optical element 205, which is configured to deflect light of a second wavelength or range of wavelengths. The light ray 220 is deflected by the incoupling optical element 207, which is configured to selectively deflect light of third wavelength or range of wavelengths.

[0057] With continued reference to FIG. 2 A, the light ray 218, the light ray 219, and the light ray 220 are deflected such that they propagate through corresponding waveguide 202, waveguide 204, and waveguide 206, respectively; that is, the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 of each waveguide deflects the light into the corresponding waveguide 202, waveguide 204, or waveguide 206 to incouple light into that corresponding waveguide. The light ray 218, the light ray 219, and the light ray 220 are deflected at angles that cause the light to propagate through the respective waveguide 202, waveguide 204, and waveguide 206 by TIR. The light ray 218, the light ray 219, and the light ray 220 propagate through the respective waveguide 202, waveguide 204, and waveguide 206 by TIR until impinging on the waveguide's corresponding light distributing elements: the light distributing elements 210, the lightdistributing elements 212, and the light distributing elements 214, where they are outcoupled to provide out-coupled light rays 216.

[0058] With reference now to FIG. 2B, a perspective view of an example of the set of stacked waveguides 200 of FIG. 2A is illustrated. As noted above, the light ray 218, the light ray 219, and the light ray 220 are incoupled and deflected by the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207, respectively, and then propagate by TIR within the waveguide 202, the waveguide 204, and the waveguide 206, respectively. The light ray 218, the light ray 219, and the light ray 220 then impinge on the light distributing elements 210, the light distributing elements 212, and the light distributing elements 214, respectively. The light distributing elements 210, the light distributing elements 212, and the light distributing elements 214 deflect the light ray 218, the light ray 219, and the light ray 220 so that they propagate towards the outcoupling optical elements 222, the outcoupling optical elements 224, and the outcoupling optical elements 226, respectively.

[0059] In some embodiments, the light distributing elements 210, the light distributing elements 212, and the light distributing elements 214 are orthogonal pupil expanders (OPEs). In some embodiments, the OPEs deflect or distribute light to the outcoupling optical elements 222, the outcoupling optical elements 224, and the outcoupling optical elements 226 and, in some embodiments, may also increase the beam or spot size of this light as it propagates to the outcoupling optical elements. In some embodiments, the light distributing elements 210, the light distributing elements 212, and the light distributing elements 214 may be omitted and the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 may be configured to deflect light directly to the outcoupling optical elements 222, the outcoupling optical elements 224, and the outcoupling optical elements 226. For example, with reference to FIG. 2A, the light distributing elements 210, the light distributing elements 212, and the light distributing elements 214 may be replaced with the outcoupling optical elements 222, the outcoupling optical elements 224, and the outcoupling optical elements 226, respectively. In some embodiments, the outcoupling optical elements 222, the outcoupling optical elements 224, and the outcoupling optical elements 226 are exit pupils (EPs) or exit pupil expanders (EPEs) that direct light to the eye of the user. It will be appreciated that the OPEs may be configured to increase the dimensions of the eye box in at least one axis and the EPEs may be configured to increase the eye box in an axis crossing, e.g., orthogonal to, the axis of the OPEs. For example, each OPE may beconfigured to redirect a portion of the light striking the OPE to an EPE of the same waveguide, while allowing the remaining portion of the light to continue to propagate down the waveguide. Upon impinging on the OPE again, another portion of the remaining light is redirected to the EPE, and the remaining portion of that portion continues to propagate further down the waveguide, and so on. Similarly, upon striking the EPE, a portion of the impinging light is directed out of the waveguide towards the user, and a remaining portion of that light continues to propagate through the waveguide until it strikes the EPE again, at which time another portion of the impinging light is directed out of the waveguide, and so on. Consequently, a single beam of incoupled light may be "replicated" each time a portion of that light is redirected by an OPE or EPE, thereby forming a field of cloned beams of light. In some embodiments, the OPE and / or EPE may be configured to modify a size of the beams of light. In some embodiments, the functionality of the light distributing elements 210, the light distributing elements 212, and the light distributing elements 214 and the outcoupling optical elements 222, the outcoupling optical elements 224, and the outcoupling optical elements 226 are combined in a combined pupil expander as discussed in relation to FIG. 2E.

[0060] Accordingly, with reference to FIGS. 2 A and 2B, in some embodiments, the set of stacked waveguides 200 includes the waveguide 202, the waveguide 204, and the waveguide 206; the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207; the light distributing elements 210, the light distributing elements 212, and the light distributing elements 214 (e.g., OPEs); and the outcoupling optical elements 222, the outcoupling optical elements 224, and the outcoupling optical elements 226 (e.g., EPs) for each component color. The waveguide 202, the waveguide 204, and the waveguide 206 may be stacked with an air gap / cladding layer between each one. The incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 redirect or deflect incident light (with different incoupling optical elements receiving light of different wavelengths) into its waveguide. The light then propagates at an angle which will result in TIR within the waveguide 202, the waveguide 204, and the waveguide 206, respectively. In the example shown, light ray 218 (e.g., blue light) is deflected by the incoupling optical element 203, and then continues to bounce down the waveguide, interacting with the light distributing element 210 (e.g., OPEs) and then the outcoupling optical element 222 (e.g., EPs), in a manner described earlier. The light ray 219 and the light ray 220 (e.g., green and red light, respectively) will pass through the waveguide 202, with light ray 219 impinging on and being deflected by incoupling optical element 205.The light ray 219 then bounces down the waveguide 204 via TIR, proceeding on to its light distributing element 212 (e.g., OPEs) and then the outcoupling optical element 224 (e.g., EPs). Finally, light ray 220 (e.g., red light) passes through the waveguide 206 to impinge on the incoupling optical element 207 of the waveguide 206. The incoupling optical element 207 deflects the light ray 220 such that the light ray propagates to light distributing element 214 (e.g., OPEs) by TIR, and then to the outcoupling optical element 226 (e.g., EPs) by TIR. The outcoupling optical element 226 then finally out-couples the light ray 220 to the viewer, who also receives the outcoupled light from the other waveguides: the waveguide 202 and the waveguide 204.

[0061] FIG. 2C illustrates a top-down, plan view of an example of the set of stacked waveguides 200 of FIGS. 2A and 2B. As illustrated, the waveguide 202, the waveguide 204, and the waveguide 206, along with each waveguide's associated light distributing element: the light distributing element 210, light distributing element 212, and light distributing element 214 and the associated outcoupling optical elements: the outcoupling optical elements 222, the outcoupling optical elements 224, and the outcoupling optical elements 226, may be vertically aligned. However, as discussed herein, the incoupling optical element 203, the incoupling optical element 205, and the incoupling optical element 207 are not vertically aligned; rather, the incoupling optical elements are preferably nonoverlapping (e.g., laterally spaced apart as seen in the top-down or plan view). As discussed further herein, this nonoverlapping spatial arrangement facilitates the injection of light from different resources into different waveguides on a one-to-one basis, thereby allowing a specific light source to be uniquely coupled to a specific waveguide. In some embodiments, arrangements including nonoverlapping spatially separated incoupling optical elements may be referred to as a shifted pupil system, and the incoupling optical elements within these arrangements may correspond to sub pupils.

[0062] FIG. 3 is a simplified illustration of an eyepiece waveguide having a combined pupil expander according to an embodiment of the present invention. In the example illustrated in FIG. 3, the eyepiece 304 utilizes a combined OPEZEPE region in a single-side configuration. Referring to FIG. 3, the eyepiece 304 includes a substrate 320 in which incoupling optical element 322 and a combined OPEZEPE region 324, also referred to as a combined pupil expander (CPE), are provided. Incident light ray 330 is incoupled via the incoupling optical element 322 and outcoupled as output light rays 332 via the combined OPEZEPE region 324.

[0063] The combined OPEZEPE region 324 includes gratings corresponding to both an OPE and an EPE that spatially overlap in the x-direction and the y-direction. In some embodiments, the gratings corresponding to both the OPE and the EPE are located on the same side of a substrate 320 such that either the OPE gratings are superimposed onto the EPE gratings or the EPE gratings are superimposed onto the OPE gratings (or both). In other embodiments, the OPE gratings are located on the opposite side of the substrate 320 from the EPE gratings such that the gratings spatially overlap in the x-direction and the y-direction but are separated from each other in the z-direction (i.e., in different planes). Thus, the combined OPEZEPE region 324 can be implemented in either a single-sided configuration or in a two- sided configuration.

[0064] FIG. 4 illustrates an example of wearable display system 430 into which the various waveguides and related systems disclosed herein may be integrated. With reference to FIG. 4, the wearable display system 430 includes a display 432, and various mechanical and electronic modules and systems to support the functioning of the display 432. The display 432 may be coupled to a frame 434, which is wearable by a user 440 (also referred to as a viewer or a display system user) and which is configured to position the display 432 in front of the eyes of the user 440. The display 432 may be considered eyewear in some embodiments. In some embodiments, a speaker 436 is coupled to the frame 434 and configured to be positioned adjacent to the ear canal of the user 440 (in some embodiments, another speaker, not shown, may optionally be positioned adjacent to the other ear canal of the user to provide stereo / shapeable sound control). The wearable display system 430 may also include one or more microphones or other devices to detect sound. In some embodiments, the microphone is configured to allow the user to provide inputs or commands to the wearable display system 430 (e.g., the selection of voice menu commands, natural language questions), and / or may allow audio communication with other persons (e.g., with other users of similar display systems). The microphone may further be configured as a peripheral sensor to collect audio data (e.g., sounds from the user and / or environment). In some embodiments, the wearable display system 430 may further include one or more outwardly directed environmental sensors configured to detect objects, stimuli, people, animals, locations, or other aspects of the world around the user. For example, environmental sensors may include one or more cameras, which may be located, for example, facing outward so as to capture images similar to at least a portion of an ordinary field of view of the user 440. In some embodiments, the wearable display system may also include a peripheralsensor, which may be separate from the frame 434 and attached to the body of the user 440 (e.g., on the head, torso, an extremity, of the user 440). The peripheral sensor may be configured to acquire data characterizing a physiological state of the user 440 in some embodiments. For example, the sensor may be an electrode.

[0065] The display 432 is operatively coupled by a communications link, such as by a wired lead or wireless connectivity, to a local data processing module which may be mounted in a variety of configurations, such as fixedly attached to the frame 434, fixedly attached to a helmet or hat worn by the user, embedded in headphones, or otherwise removably attached to the user 440 (e.g., in a backpack-style configuration, in a belt-coupling style configuration). Similarly, the sensor may be operatively coupled by a communications link, e.g., a wired lead or wireless connectivity, to the local processor and data module. The local processing and data module may comprise a hardware processor, as well as digital memory, such as nonvolatile memory (e.g., flash memory or hard disk drives), both of which may be utilized to assist in the processing, caching, and storage of data. Optionally, the local processor and data module may include one or more central processing units (CPUs), graphics processing units (GPUs), dedicated processing hardware, and so on. The data may include data a) captured from sensors (which may be, e.g., operatively coupled to the frame 434 or otherwise attached to the user 440), such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, radio devices, gyros, and / or other sensors disclosed herein; and / or b) acquired and / or processed using remote processing module 452 and / or remote data repository 454 (including data relating to virtual content), possibly for passage to the display 432 after such processing or retrieval. The local processing and data module may be operatively coupled by communication links 438 such as via wired or wireless communication links, to the remote processing and data module 450, which can include the remote processing module 452, the remote data repository 454, and a battery 460. The remote processing module 452 and the remote data repository 454 can be coupled by communication links 456 and communication links 458 to remote processing and data module 450 such that these remote modules are operatively coupled to each other and available as resources to the remote processing and data module 450. In some embodiments, the remote processing and data module 450 may include one or more of the image capture devices, microphones, inertial measurement units, accelerometers, compasses, GPS units, radio devices, and / or gyros. In some other embodiments, one or more of these sensors may beattached to the frame 434, or may be standalone structures that communicate with the remote processing and data module 450 by wired or wireless communication pathways.

[0066] With continued reference to FIG. 4, in some embodiments, the remote processing and data module 450 may comprise one or more processors configured to analyze and process data and / or image information, for instance including one or more central processing units (CPUs), graphics processing units (GPUs), dedicated processing hardware, and so on. In some embodiments, the remote data repository 454 may comprise a digital data storage facility, which may be available through the internet or other networking configuration in a "cloud" resource configuration. In some embodiments, the remote data repository 454 may include one or more remote servers, which provide information, e.g., information for generating augmented reality content, to the local processing and data module and / or the remote processing and data module 450. In some embodiments, all data is stored and all computations are performed in the local processing and data module, allowing fully autonomous use from a remote module. Optionally, an outside system (e.g., a system of one or more processors, one or more computers) that includes CPUs, GPUs, and so on, may perform at least a portion of processing (e.g., generating image information, processing data) and provide information to, and receive information from, the illustrated modules, for instance, via wireless or wired connections.

[0067] FIG. 5 shows a perspective view of a wearable device 500 according to an embodiment of the present invention. Wearable device 500 includes a frame 502 configured to support one or more projectors 523 at various positions along an interior-facing surface of frame 502, as illustrated. In some embodiments, projectors 523 can be attached at positions near temples 506. Alternatively, or in addition, another projector could be placed in position 508. Such projectors may, for instance, include or operate in conjunction with one or more liquid crystal on silicon (LCoS) modules, micro-LED displays, or fiber scanning devices. In some embodiments, light from projectors 523 or projectors disposed in position 508 could be guided into eyepieces 504 for display to eyes of a user. Projectors placed at positions 512 can be somewhat smaller on account of the close proximity this gives the projectors to the waveguide system. The closer proximity can reduce the amount of light lost as the waveguide system guides light from the projectors to eyepiece 504. In some embodiments, the projectors at positions 512 can be utilized in conjunction with projectors 523 or projectors disposed in position 508. While not depicted, in some embodiments, projectors could also be located at positions beneath eyepieces 504. Wearable device 500 is also depicted including sensors 514and sensors 516. Sensors 514 and sensors 516 can take the form of forward-facing and lateral-facing optical sensors configured to characterize the real-world environment surrounding wearable device 500.

[0068] Embodiments of the present invention utilize an eye tracking system to determine the eye gaze location of the user and utilize the eye gaze location for image compression processes. Referring to FIG. 5, eye tracking cameras 505 are located on the frame 502 and can be utilized to track the eye gaze location of the user using the wearable device 500. In other embodiments, other eye tracking systems are utilized to determine the eye gaze location and the eye tracking cameras 505 illustrated in FIG. 5 are merely exemplary. As described more fully herein, the image compression processes utilized to compress and decompress virtual content for storage in memory, internal communications, and display, among other functions, can be modified depending on the eye gaze location, for example, portions of an image or video stream corresponding to the eye gaze location can be compressed using a higher quality compression process compared to other portions of the image or video stream that are located more distant from the eye gaze location. Since these more distant portions of the image or video stream are in the user's peripheral vision, any impact on the user experience resulting from the reduction in compression quality can be less than the benefits achieved in terms of memory and processing efficiency and / or requirements. One of ordinary skill in the art would recognize many variations, modifications, and alternatives.

[0069] FIG. 6 is a simplified plan view of elements of an AR headset including eye tracking according to an embodiment of the present invention. In FIG. 6, elements of an optical stack 600 are illustrated. The AR headset may include a pair of optical stacks, one for each eye of the user. The optical stack 600 includes a front Extended Depth Of Field (EDOF) lens 610 and front optics 612 that receive world light propagating from right to left toward the eye 660 of the user. The optical stack 600 also includes a dimmer 602, an eyepiece 604, a rear EDOF lens 652, an eye tracking (ET) structure, and an optional prescription lens insert 662. The eye tracking structure may include one or more eye tracking cameras 605 disposed along / outside a periphery of the rear EDOF lens 652, one or more wedge prisms 636 integrated with the periphery of the rear EDOF lens 652, and one or more illumination sources 613 disposed along / outside the periphery of the rear EDOF lens 652 or within an illumination layer disposed between the rear EDOF lens 652 and the eyepiece 604.

[0070] Dimmer 602 includes a world side linear polarizer 620, a first quarter waveplate 622, a liquid crystal panel 624, a second quarter waveplate 626, and an eye side linear polarizer 628, for example, a hard coated linear polarizer (HC-LP) with a surface open to air hard coated for handling purposes. Eyepiece 604 includes three eyepiece waveguide layers: blue active layer 630, green active layer 632, and red active layer 634. Although a three-layer eyepiece (i.e., an eyepiece including three eyepiece waveguide layers) is illustrated in FIG. 6, this is not required and in other embodiments, a six-layer eyepiece structure can be utilized with, for example, two depth planes.

[0071] The eye tracking system and the rear EDOF lens 652 collectively form a rear EDOF and ET structure 606. The wedge prisms 636 may be integrated with the rear EDOF lens 652 such that the curvature of the wedge prisms 636 conforms to the curvature of the rear EDOF lens 652. Each of the wedge prisms 636 may be aligned with at least one eye tracking camera 605 or at least one illumination source 613. Each of the wedge prisms 636 may be operable to reflect light from an outward radial direction of the optical stack 600 to an eye side axial direction of the optical stack 600, or from a world side axial direction to an outward radial direction. For example, each of the wedge prisms 636 aligned with illumination sources may reflect illumination light from the inward radial direction to the eye side axial direction, and each of the wedge prisms 636 aligned with eye tracking cameras may reflect illumination light from the world side axial direction to the outward radial direction.

[0072] FIG. 7 illustrates a cross-sectional, side view of an example of a rear EDOF and ET structure 706 according to an embodiment of the present invention. In FIG. 7, the rear EDOF and ET structure 706 includes a rear EDOF lens 752, a wedge prism 736 integrated along a periphery of the rear EDOF lens 752, and an eye tracking camera 705 disposed outside the periphery of the rear EDOF lens 752 in alignment with the wedge prism 736. During operation, the eye tracking camera 705 may capture an image of an eye 760 of a user using light that is refracted through the wedge prism 736 and reflected toward the eye tracking camera 705. For example, light having reflected off the eye 760 of the user may propagate in the world side axial direction (and partially in the outward radial direction) toward the wedge prism 736. The light may enter the wedge prism 736 and be reflected in the outward radial direction toward the eye tracking camera 705, which captures the light and generates an image of the eye 760 of the user based on the captured light.

[0073] An inset in FIG. 7 shows an expanded view of various surfaces 738 of the wedge prism 736. In various embodiments, the wedge prism 736 includes a first surface 738-1, a second surface 738-2, a third surface 738-3, and optionally a fourth surface 738-4. The first surface 738-1 is located on the eye side of the wedge prism 736 and acts as the input surface at which light is received when the wedge prism 736 is aligned with an eye tracking camera or as the output surface at which illumination light is outputted when the wedge prism 736 is aligned with an illumination source. The second surface 738-2 is located on the inward radial direction of the wedge prism 736 and acts as the reflective surface at which light passing through the wedge prism 736 is reflected. The second surface 738-2 is adjacent to the first surface 738-1 in that the second surface 738-2 shares an edge with the first surface 738-1.

[0074] The third surface 738-3 is located on the outward radial direction of the wedge prism 736 and acts as the output surface at which light is outputted when the wedge prism 736 is aligned with an eye tracking camera or as the input surface at which illumination light is received when the wedge prism 736 is aligned with an illumination source. The third surface 738-3 is adjacent to the first surface 738-1 in that the third surface 738-3 shares an edge with the first surface 738-1. In some embodiments, the second surface 738-2 may be adjacent to the third surface 738-3 in that the second surface 738-2 shares an edge with the third surface 738-3, such that the wedge prism 736 has a triangular shape. In some embodiments, the wedge prism 736 may include a fourth surface 738-4 that is adjacent to the second surface 738-2 and the third surface 738-3 in that the fourth surface 738-4 shares an edge with the second surface 738-2 and the third surface 738-3.

[0075] In some examples, the wedge prism 736 includes a reflective coating 742 on the second surface 738-2 that reflects infrared light and passes visible light. The reflective coating 742 may be a dielectric coating, an Indium Tin Oxide (ITO) coating, among other possibilities. In some embodiments, the second surface 738-2 may include a diffraction grating characterized by a set of grating phase parameters. The diffraction grating may have slit or groove spacings to match the wavelengths of infrared light so that the grating can be used to reflect infrared light. By making the spacings smaller than the wavelengths of visible light, the visible light will not be efficiently diffracted and will mostly be transmitted through the second surface 738-2. In some examples, the diffraction grating may comprise a liquid crystal polarization grating (LCPG) or a holographic optical element (HOE) where the diffractive phase profile is optimized to reduce a pinhole error associated with the image captured by the eye tracking camera 705.

[0076] FIG. 8 illustrates a cross-sectional, side view of an example of a rear EDOF and ET structure 806 according to an embodiment of the present invention. In FIG. 8, the rear EDOF and ET structure 806 includes a rear EDOF lens 852, a first wedge prism 836-1 integrated along a periphery of the rear EDOF lens 852, an illumination source 813 disposed outside the periphery of the rear EDOF lens 852 in alignment with the first wedge prism 836-1, a second wedge prism 836-2 integrated along the periphery of the rear EDOF lens 852, and an eye tracking camera 805 disposed outside the periphery of the rear EDOF lens 852 in alignment with the second wedge prism 836-2. In the illustrated example, each of the first wedge prism 836-1 and the second wedge prism 836-2 have four surfaces in the cross-sectional side view.

[0077] During operation, the illumination source 813 generates and transmits illumination light in the inward radial direction into the first wedge prism 836-1, which reflects the illumination light to the eye side axial direction (and partially in the inward radial direction) toward an eye 860 of the user. The illumination light reflects off the eye 860 of the user and propagates in the world side axial direction (and partially in the outward radial direction) toward the second wedge prism 836-2. The illumination light enters the second wedge prism 836-2, which reflects the illumination light to the outward radial direction toward the eye tracking camera 805. The eye tracking camera 805 captures the illumination light along with other light and generates an image of the eye 860 of the user based on the captured light.

[0078] FIG. 9 illustrates a cross-sectional, side view of an example of a rear EDOF and ET structure 906 according to an embodiment of the present invention. In FIG. 9, the rear EDOF and ET structure 906 includes a rear EDOF lens 952, a first wedge prism 936-1 integrated along a periphery of the rear EDOF lens 952, an illumination source 913 disposed outside the periphery of the rear EDOF lens 952 in alignment with the first wedge prism 936-1, a second wedge prism 936-2 integrated along the periphery of the rear EDOF lens 952, and an eye tracking camera 905 disposed outside the periphery of the rear EDOF lens 952 in alignment with the second wedge prism 936-2. The rear EDOF and ET structure 906 differs from the rear EDOF and ET structure 806 in FIG. 8 in that each of the first wedge prism 936-1 and the second wedge prism 936-2 have three surfaces in the cross-sectional side view forming a triangular shape.

[0079] FIG. 10 illustrates a perspective view of an example of a rear EDOF and ET structure 1006 according to an embodiment of the present invention. In FIG. 10, the rear EDOF and ET structure 1006 includes a rear EDOF lens 1052, a first wedge prism 1036-1integrated along a periphery of the rear EDOF lens 1052, a first eye tracking camera 1005-1 disposed outside the periphery of the rear EDOF lens 1052 in alignment with the first wedge prism 1036-1, a second wedge prism 1036-2 integrated along the periphery of the rear EDOF lens 1052, and a second eye tracking camera 1005-2 disposed outside the periphery of the rear EDOF lens 1052 in alignment with the second wedge prism 1036-2.

[0080] FIG. 11 illustrates a perspective view of an example of a rear EDOF and ET structure 1106 according to an embodiment of the present invention. In FIG. 11, the rear EDOF and ET structure 1106 includes a rear EDOF lens 1152, a first wedge prism 1136-1 integrated along a periphery of the rear EDOF lens 1152, a first eye tracking camera 1105-1 disposed outside the periphery of the rear EDOF lens 1152 in alignment with the first wedge prism 1136-1, a second wedge prism 1136-2 integrated along the periphery of the rear EDOF lens 1152, and a second eye tracking camera 1105-2 disposed outside the periphery of the rear EDOF lens 1152 in alignment with the second wedge prism 1136-2. The rear EDOF and ET structure 1106 further includes a third wedge prism 1136-3, a fourth wedge prism 1136-4, a fifth wedge prism 1136-5, a sixth wedge prism 1136-6, and a seventh wedge prism 1136-7 integrated along the periphery of the rear EDOF lens 1152 and respectively aligned with a first illumination source 1113-1, a second illumination source 1113-2, a third illumination source 1113-3, a fourth illumination source 1113-4, and a fifth illumination source 1113-5.

[0081] FIG. 12 illustrates a perspective view of an example of a rear EDOF and ET structure 1206 according to an embodiment of the present invention. In FIG. 12, the rear EDOF and ET structure 1206 includes a rear EDOF lens 1252 and a wedge prism 1236 integrated along an entirety of a periphery of the rear EDOF lens 1252, forming a wedge ring that completely surrounds the rear EDOF lens 1252. The rear EDOF and ET structure 1206 further includes a first eye tracking camera 1205-1, a second eye tracking camera 1205-2, a first illumination source 1213-1, a second illumination source 1213-2, a third illumination source 1213-3, a fourth illumination source 1213-4, and a fifth illumination source 1213-5 disposed outside the periphery of the rear EDOF lens 1252 in alignment with the wedge prism 1236.

[0082] FIG. 13 illustrates a cross-sectional, side view of an example of a rear EDOF and ET structure 1306 and example grating parameters according to an embodiment of the present invention. In FIG. 13, the rear EDOF and ET structure 1306 includes a rear EDOF lens 1352 and a wedge prism 1336 integrated along a periphery of the rear EDOF lens 1352.In various embodiments, the wedge prism 1336 includes a first surface 1338-1, a second surface 1338-2, a third surface 1338-3, and optionally a fourth surface 1338-4 as described herein. In FIG. 13, the wedge prism 1336 includes a diffraction grating at the second surface 1338-2 characterized by a set of grating phase parameters. In some examples, the eye tracking algorithm employed assumes a pinhole imaging system where the rays from the eye converge to a point (pinhole). Imaging through the wedge prism 1336 creates a virtual pinhole, i.e., a virtual image of the camera pinhole through the wedge prism. Due to aberration introduced by the wedge prism 1336, the rays from the eye may have a certain distribution at the virtual pinhole plane. The example grating parameters for the second surface 1338-2 shown in FIG. 13 reduce the pinhole error, improving the eye tracking accuracy. Alternatively or additionally, in some embodiments the third surface 1338-3 may include a transmissive diffractive grating or a refractive power can be applied to the third surface 1338-3 to help reduce the pinhole error and improve the image quality.

[0083] FIGS. 14A and 14B show simulated results demonstrating the reduction of the pinhole error by using the diffraction grating described in FIG. 13. FIG. 14A shows the ray distribution at the entrance pupil plane before pinhole error compensation, using a reflective coating on the second surface 1338-2. FIG. 14B shows the ray distribution at the entrance pupil plane after pinhole error compensation, using a diffractive grating on the second surface 1338-2. The root-mean-square (RMS) radius of ray distribution (chief rays only) improves from FIGS. 14A to 14B from 0.194 mm to 0.059 mm.

[0084] FIGS. 15A and 15B show simulated results demonstrating the improvement in image quality using the wedge prism when a high power Rx lens is present. In the plots shown in FIGS. 15A and 15B, the y-axis corresponds to modulation transfer function (MTF) and the x-axis corresponds to spatial frequency in cycles / mm. The plot in FIG. 15A shows simulated MTF values for imaging through a -10 diopter Rx lens with the camera positioned at a virtual pinhole location. The plot in FIG. 15B shows simulated MTF values for imaging through a -10 diopter Rx lens and the wedge prism. In the illustrated example, 5 field points at the object / eye space were used for the MTF evaluation. To compensate for the aberration introduced by Rx lenses with large negative power (e.g. -7.5D to -10D), an acute angle is preferred between the third and fourth surfaces of the wedge prism, ranging from 50° to 90°. In this example, the angle between the third and fourth surfaces was 75°.

[0085] FIG. 16 is a simplified flowchart illustrating a method 1600 of operating an optics assembly according to an embodiment of the present invention. In some examples, the method 1600 may be a method of performing eye tracking at an AR system. The method 1600 includes, at step 1610, receiving virtual image light at an eyepiece (e.g., eyepieces 304, 504, 604). The virtual image light may be generated by a projector (e.g., projectors 523) and coupled into the eyepiece.

[0086] The method 1600 further includes, at step 1612, projecting, by the eyepiece, the virtual image light toward an eye side of the optics assembly.

[0087] The method 1600 further includes, at step 1614, passing the virtual image light through a rear lens (e.g., rear EDOF lenses 652, 752, 852, 952, 1052, 1152, 1252, 1352) disposed between the eyepiece and the eye side of the optics assembly. The virtual image light may reach an eye (e.g., eyes 660, 760, 860) of a user, causing the user to perceive virtual content. The virtual content may be perceived by the user along with real-world imagery elements.

[0088] The method 1600 further includes, at step 1616, generating, by an illumination source (e.g., illumination sources 613, 813, 913, 1113, 1213) disposed outside a periphery of the rear lens, illumination light that is coupled into one or more wedge prisms (wedge prisms 636, 736, 836, 936, 1036, 1136, 1236, 1336). The one or more wedge prisms may be integrated with the rear lens along the periphery of the rear lens.

[0089] The method 1600 further includes, at step 1618, reflecting, by the one or more wedge prisms, the illumination light toward the eye side of the optics assembly. For example, the illumination light may be reflected by a first wedge prism of the one or more wedge prisms. The first wedge prism may include a first surface (e.g., surface 738-1) on an eye side of the wedge prism, a second surface (e.g., surface 738-2) on an inward radial direction of the wedge prism, and a third surface (e.g., surface 738-3) on an outward radial direction of the wedge prism. The second surface may be reflective. The illumination light that is reflected toward the eye side of the optics assembly may reflect off the eye of the user and propagate from the eye side of the optics assembly toward the world side of the optics assembly in the world side axial direction.

[0090] The method 1600 further includes, at step 1620, reflecting, by the one or more wedge prisms, the illumination light that propagates from the eye side of the optics assembly in the world side axial direction toward an outward radial direction. For example, theillumination light propagating from the eye side of the optics assembly may be reflected by a second wedge prism of the one or more wedge prisms. The second wedge prism may include a first surface (e.g., surface 738-1) on an eye side of the wedge prism, a second surface (e.g., surface 738-2) on an inward radial direction of the wedge prism, and a third surface (e.g., surface 738-3) on an outward radial direction of the wedge prism. The second surface may be reflective. In some examples, the first wedge prism and the second wedge prism may be a same wedge prism that is a ring wedge prism (e.g., wedge prism 1236) that is integrated with an entirety of the periphery of the rear lens.

[0091] The method 1600 further includes, at step 1622, capturing the illumination light propagating in the outward radial direction by an eye tracking camera (eye tracking cameras 505, 605, 705, 805, 905, 1005, 1105, 1205) disposed outside the periphery of the rear lens in alignment with the one or more wedge prisms. The illumination light captured by the eye tracking camera may be used to determine the gaze location of the eye of the user.

[0092] It should be appreciated that the specific steps illustrated in FIG. 16 provides a particular method of operating an optics assembly according to an embodiment of the present invention. Other sequences of steps may also be performed according to alternative embodiments. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Moreover, the individual steps illustrated in FIG.16 may include multiple sub-steps that may be performed in various sequences as appropriate to the individual step. Furthermore, additional steps may be added or removed depending on the particular applications. One of ordinary skill in the art would recognize many variations, modifications, and alternatives.

[0093] FIG. 17 is a simplified block diagram illustrating components of an AR system according to an embodiment of the present invention. AR system 1700 as illustrated in FIG.17 may be incorporated into the AR devices as described herein. FIG. 17 provides a schematic illustration of one embodiment of AR system 1700 that can perform some or all of the steps of the methods provided by various embodiments. It should be noted that FIG. 17 is meant only to provide a generalized illustration of various components, any or all of which may be utilized as appropriate. FIG. 17, therefore, broadly illustrates how individual system elements may be implemented in a relatively separated or relatively more integrated manner.

[0094] AR system 1700 is shown comprising hardware elements that can be electrically coupled via a bus 1705, or may otherwise be in communication, as appropriate. The hardwareelements may include one or more processors 1710, including without limitation one or more general-purpose processors and / or one or more special-purpose processors such as digital signal processing chips, graphics acceleration processors, and / or the like; one or more input devices 1730, which can include without limitation a mouse, a keyboard, a camera, and / or the like; and one or more output devices 1740, which can include without limitation a display device, a printer, and / or the like. Additionally, AR system 1700 includes an eye tracking system 1770 that can provide the user's eye gaze location to the AR system. Utilizing one or more processors 1710, the eye tracking techniques discussed herein can be implemented.

[0095] AR system 1700 may further include and / or be in communication with storage device(s) 1720 (e.g., one or more non-transitory storage devices), which can comprise, without limitation, local and / or network accessible storage, and / or can include, without limitation, a disk drive, a drive array, an optical storage device, a solid-state storage device, such as a random access memory (RAM), and / or a read-only memory (ROM), which can be programmable, flash-updateable, and / or the like. Such storage devices may be configured to implement any appropriate data stores, including without limitation, various file systems, database structures, and / or the like.

[0096] AR system 1700 might also include a communications subsystem 1750, which can include without limitation a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device, and / or a chipset such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, cellular communication facilities, etc., and / or the like. Communications subsystem 1750 may include one or more input and / or output communication interfaces to permit data to be exchanged with a network such as the network described below to name one example, other computer systems, television, and / or any other devices described herein. Depending on the desired functionality and / or other implementation concerns, a portable electronic device or similar device may communicate an image and / or other information via communications subsystem 1750. In other embodiments, a portable electronic device, e.g., the first electronic device, may be incorporated into AR system 1700, e.g., an electronic device as an input device 1730. In some embodiments, AR system 1700 will further comprise a working memory 1760, which can include a RAM or ROM device, as described above.

[0097] AR system 1700 also can include software elements, shown as being currently located within working memory 1760, including an operating system 1762, device drivers,executable libraries, and / or other code, such as one or more application programs 1764, which may comprise computer programs provided by various embodiments, and / or may be designed to implement methods, and / or configure systems, provided by other embodiments, as described herein. Merely by way of example, one or more procedures described with respect to the methods discussed above might be implemented as code and / or instructions executable by a computer and / or a processor within a computer; in an aspect, then, such code and / or instructions can be used to configure and / or adapt a general purpose computer or other device to perform one or more operations in accordance with the described methods.

[0098] A set of these instructions and / or code may be stored on a non-transitory computer- readable storage medium, such as storage device(s) 1720 described above. In some cases, the storage medium might be incorporated within a computer system, such as AR system 1700. In other embodiments, the storage medium might be separate from a computer system e.g., a removable medium, such as a compact disc, and / or provided in an installation package, such that the storage medium can be used to program, configure, and / or adapt a general purpose computer with the instructions / code stored thereon. These instructions might take the form of executable code, which is executable by AR system 1700 and / or might take the form of source and / or installable code, which, upon compilation and / or installation on AR system 1700, e.g., using any of a variety of generally available compilers, installation programs, compression / decompression utilities, then takes the form of executable code.

[0099] It will be apparent to those skilled in the art that substantial variations may be made in accordance with specific requirements. For example, customized hardware might also be used, and / or particular elements might be implemented in hardware, software including portable software, such as applets, etc., or both. Further, connection to other computing devices such as network input / output devices may be employed.

[0100] As mentioned above, in one aspect, some embodiments may employ a computer system such as AR system 1700 to perform methods in accordance with various embodiments of the technology. According to a set of embodiments, some or all of the procedures of such methods are performed by AR system 1700 in response to one or more processors 1710 executing one or more sequences of one or more instructions, which might be incorporated into operating system 1762 and / or other code, such as an application program 1764, contained in working memory 1760. Such instructions may be read into working memory 1760 from another computer-readable medium, such as one or more of storagedevice(s) 1720. Merely by way of example, execution of the sequences of instructions contained in working memory 1760 might cause one or more processors 1710 to perform one or more procedures of the methods described herein. Additionally or alternatively, portions of the methods described herein may be executed through specialized hardware.

[0101] The terms machine-readable medium and computer-readable medium, as used herein, refer to any medium that participates in providing data that causes a machine to operate in a specific fashion. In an embodiment implemented using AR system 1700, various computer-readable media might be involved in providing instructions / code to one or more processors 1710 for execution and / or might be used to store and / or carry such instructions / code. In many implementations, a computer-readable medium is a physical and / or tangible storage medium. Such a medium may take the form of a non-volatile media or volatile media. Non-volatile media include, for example, optical and / or magnetic disks, such as storage device(s) 1720. Volatile media include, without limitation, dynamic memory, such as working memory 1760.

[0102] Common forms of physical and / or tangible computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read instructions and / or code.

[0103] Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to one or more processors 1710 for execution. Merely by way of example, the instructions may initially be carried on a magnetic disk and / or optical disc of a remote computer. A remote computer might load the instructions into its dynamic memory and send the instructions as signals over a transmission medium to be received and / or executed by AR system 1700.

[0104] Communications subsystem 1750 and / or components thereof generally will receive signals, and bus 1705 then might carry the signals and / or the data, instructions, etc. carried by the signals to working memory 1760, from which one or more processors 1710 retrieves and executes the instructions. The instructions received by working memory 1760 may optionally be stored on storage device(s) 1720, e.g., a non-transitory storage device, either before or after execution by one or more processors 1710.

[0105] In the foregoing specification, the disclosure has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure. The specification and drawings are, accordingly, to be regarded in an illustrative rather than restrictive sense.

[0106] Indeed, it will be appreciated that the systems and methods of the disclosure each have several innovative aspects, no single one of which is solely responsible or required for the desirable attributes disclosed herein. The various features and processes described above may be used independently of one another, or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of this disclosure.

[0107] Certain features that are described in this specification in the context of separate embodiments also may be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment also may be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination. No single feature or group of features is necessary or indispensable to each and every embodiment.

[0108] It will be appreciated that conditional language used herein, such as, among others, "can," "could," "might," "may," "e.g.," and the like, unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements and / or steps. Thus, such conditional language is not generally intended to imply that features, elements and / or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without author input or prompting, whether these features, elements and / or steps are included or are to be performed in any particular embodiment. The terms "comprising," "including," "having," and the like are synonymous and are used inclusively, in an open-ended fashion, and do not exclude additional elements, features, acts, operations, and so forth. Also, the term "or" is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term "or" means one, some, or all of the elements in the list. In addition,the articles "a," "an," and "the" as used in this application and the appended claims are to be construed to mean "one or more" or "at least one" unless specified otherwise. Similarly, while operations may be depicted in the drawings in a particular order, it is to be recognized that such operations need not be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Further, the drawings may schematically depict one or more example processes in the form of a flowchart. However, other operations that are not depicted may be incorporated in the example methods and processes that are schematically illustrated. For example, one or more additional operations may be performed before, after, simultaneously, or between any of the illustrated operations. Additionally, the operations may be rearranged or reordered in other embodiments. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged into multiple software products. Additionally, other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve desirable results.

[0109] Accordingly, the claims are not intended to be limited to the embodiments shown herein but are to be accorded the widest scope consistent with this disclosure, the principles, and the novel features disclosed herein. Thus, it is also understood that the examples and embodiments described herein are for illustrative purposes only and that various modifications or changes in light thereof will be suggested to persons skilled in the art and are to be included within the spirit and purview of this application and scope of the appended claims.

Claims

WHAT IS CLAIMED IS:

1. An optics assembly comprising: an eyepiece operable to receive virtual image light and project the virtual image light toward an eye side of the optics assembly; a rear lens disposed between the eyepiece and the eye side of the optics assembly; one or more wedge prisms integrated with the rear lens along a periphery of the rear lens; and an eye tracking camera disposed outside the periphery of the rear lens in alignment with the one or more wedge prisms and operable to capture light that propagates from the eye side of the optics assembly and is refracted through the one or more wedge prisms.

2. The optics assembly of claim 1, further comprising: an illumination source disposed along the periphery of the rear lens in alignment with the one or more wedge prisms and operable to emit illumination light that is refracted through the one or more wedge prisms toward the eye side of the optics assembly, wherein the light captured by the eye tracking camera includes the illumination light.

3. The optics assembly of claim 2, wherein the one or more wedge prisms include: a first wedge prism operable to receive the illumination light from the illumination source and reflect the illumination light toward the eye side of the optics assembly; and a second wedge prism operable to receive the light that propagates from the eye side of the optics assembly and reflect the light toward the eye tracking camera.

4. The optics assembly of claim 3, further comprising: a third wedge prism integrated with the rear lens along the periphery of the rear lens; and a second eye tracking camera disposed outside the periphery of the rear lens in alignment with the third wedge prism and operable to capture light that propagates from the eye side of the optics assembly and is refracted through the third wedge prism, wherein the eye tracking camera is a first eye tracking camera.

5. The optics assembly of claim 4, further comprising: a fourth wedge prism integrated with the rear lens along the periphery of the rear lens; and a second illumination source disposed along the periphery of the rear lens in alignment with the fourth wedge prism and operable to emit second illumination light that is refracted through the fourth wedge prism toward the eye side of the optics assembly, wherein the illumination light is first illumination light, and wherein the light captured by the first eye tracking camera and the light captured by the second eye tracking camera include the first illumination light and the second illumination light.

6. The optics assembly of claim 1, wherein each wedge prism of the one or more wedge prisms comprise: a first surface on an eye side of the wedge prism; a second surface on an inward radial direction of the wedge prism, the second surface being reflective, the second surface being adjacent to the first surface; and a third surface on an outward radial direction of the wedge prism, the third surface being adjacent to the first surface.

7. The optics assembly of claim 6, wherein the second surface is adjacent to the third surface such that the wedge prism has a triangular shape.

8. The optics assembly of claim 6, wherein each wedge prism of the one or more wedge prisms further comprises: a fourth surface on a world side of the wedge prism, the fourth surface being adjacent to the second surface and the third surface.

9. The optics assembly of claim 6, wherein the second surface comprises a reflective coating.

10. The optics assembly of claim 6, wherein the second surface comprises a diffraction grating characterized by grating phase parameters.

11. The optics assembly of claim 1, wherein the one or more wedge prisms comprise a ring wedge prism that is integrated with an entirety of the periphery of the rear lens.

12. The optics assembly of claim 1, wherein the rear lens is an Extended Depth Of Field (EDOF) lens.

13. An augmented reality (AR) system comprising: a frame; a set of projectors coupled to the frame and configured to generate virtual image light; a set of eye tracking cameras coupled to the frame; and a set of optical stacks coupled to the frame, each of the set of optical stacks comprising: an eyepiece operable to receive the virtual image light and project the virtual image light toward an eye side of the AR system; a rear lens disposed between the eyepiece and the eye side of the AR system; and a wedge prism integrated with the rear lens along a periphery of the rear lens and operable to receive light that propagates from the eye side of the AR system and reflect the light in an outward radial direction toward one of the set of eye tracking cameras.

14. The AR system of claim 13, wherein each of the set of optical stacks further comprises: an illumination source disposed along the periphery of the rear lens and operable to emit illumination light that is refracted through either the wedge prism or a second wedge prism toward the eye side of the AR system, wherein the light captured by one of the set of eye tracking cameras includes the illumination light.

15. The AR system of claim 13, wherein the wedge prism comprises: a first surface on an eye side of the wedge prism; a second surface on an inward radial direction of the wedge prism, the second surface being reflective, the second surface being adjacent to the first surface; and a third surface on an outward radial direction of the wedge prism, the third surface being adjacent to the first surface.

16. The AR system of claim 15, wherein the second surface is adjacent to the third surface such that the wedge prism has a triangular shape.

17. The AR system of claim 15, wherein the wedge prism further comprises: a fourth surface on a world side of the wedge prism, the fourth surface being adjacent to the second surface and the third surface.

18. A method of operating an optics assembly, the method comprising: receiving virtual image light at an eyepiece; projecting, by the eyepiece, the virtual image light toward an eye side of the optics assembly; passing the virtual image light through a rear lens disposed between the eyepiece and the eye side of the optics assembly; generating, by an illumination source disposed outside a periphery of the rear lens, illumination light that is coupled into one or more wedge prisms, wherein the one or more wedge prisms are integrated with the rear lens along the periphery of the rear lens; reflecting, by the one or more wedge prisms, the illumination light toward the eye side of the optics assembly; reflecting, by the one or more wedge prisms, the illumination light that propagates from the eye side of the optics assembly toward an outward radial direction; and capturing the illumination light by an eye tracking camera disposed outside the periphery of the rear lens in alignment with the one or more wedge prisms.

19. The method of claim 18, wherein the one or more wedge prisms include a first wedge prism that receives the illumination light from the illumination source and reflects the illumination light toward the eye side of the optics assembly and a second wedge prism that receives the illumination light that propagates from the eye side of the optics assembly and reflects the illumination light toward the outward radial direction.

20. The method of claim 18, wherein the one or more wedge prisms comprise a ring wedge prism that receives the illumination light from the illumination source, reflects the illumination light toward the eye side of the optics assembly, receives the illumination light that propagates from the eye side of the optics assembly, and reflects the illumination light toward the outward radial direction.

Images