Optical combiner aberration correction in eye-tracking imaging

Abstract

An optical assembly for an eye-tracking camera includes an aperture stop, a first optical surface, and a second optical surface. The optical assembly is configured to receive non-visible light reflected or scattered by an eye and direct the non-visible light along an optical path to an image sensor, wherein the non-visible light is received from an optical combiner of an eye-tracking system. The first optical surface is disposed on the optical path and is configured to correct for field-independent optical aberrations caused by the optical combiner. The second optical surface is disposed on the optical path and is configured to correct for field-dependent optical aberrations caused by the optical combiner.

Description

TECHNICAL FIELD

[0001] Aspects of the present disclosure generally relate to smart devices, such as smart displays, head-mounted displays (HMDs), head-mounted devices, and the like. In particular, aspects of the present disclosure relate to, but are not limited to, aberration correction in eye tracking systems. BACKGROUND

[0002] Smart devices are electronic devices that typically communicate with other devices or networks. In some cases, smart devices can be configured to interactively operate with a user. Smart devices can be designed to support various form factors, such as head-mounted devices, head-mounted displays (HMDs), or smart displays, to name a few examples.

[0003] Smart devices can include one or more electronic components for various applications, such as gaming, aviation, engineering, medicine, entertainment, video / audio chat, activity tracking, and the like. In some examples, smart devices can perform eye tracking, which can enhance a user’s viewing experience. In some cases, eye tracking can be aided by illuminating a user’s eyes. Accordingly, some smart devices can include an eye tracking system that includes an illumination source and a camera for tracking movement of a user’s eyes. However, various environments can present challenges to capturing images of the eyes sufficient for analysis. In particular, various optical elements included in an eye tracking system can induce optical aberrations that distort captured images. SUMMARY

[0004] In one aspect of the present disclosure, an eye tracking system is provided, comprising: an eye tracking camera comprising an image sensor; and an optical combiner configured to receive non-visible light reflected or scattered by an eye and direct the non-visible light to the eye tracking camera, wherein the eye tracking camera further comprises: an optical assembly to receive the non-visible light from the optical combiner and direct the non-visible light along an optical path to the image sensor, wherein the optical assembly comprises: an aperture stop disposed on the optical path; a first optical surface disposed on the optical path, wherein the first optical surface is configured to correct for a field-independent optical aberration induced by the optical combiner; and a second optical surface disposed on the optical path, wherein the second optical surface is configured to correct for a field-dependent optical aberration induced by the optical combiner.

[0005] The first optical surface can comprise a surface profile that can be one or more of rotationally symmetric, aspheric, freeform, or anamorphic.

[0006] The second optical surface can be a freeform optical surface.

[0007] The freeform optical surface can be a Zernike-polynomial optical surface, an XY-polynomial optical surface, a Chebyshev-polynomial optical surface, or an off-axis segment of a deformed optical surface.

[0008] The optical assembly can further include a first lens disposed on the optical path, wherein the first optical surface can be a surface of the first lens, and a second lens disposed on the optical path, wherein the second optical surface can be a surface of the second lens.

[0009] The optical assembly can further include a prism disposed on the optical path between the aperture stop and the image sensor, wherein the prism can be configured to provide both the first optical surface and the second optical surface.

[0010] The second optical surface can be configured to reflect the non-visible light to the image sensor.

[0011] The field-independent optical aberrations caused by the optical combiner can include a vignetting aberration that is substantially uniform across a field of view of the eye movement tracking camera, and wherein the field-dependent optical aberrations caused by the optical combiner can include a vignetting aberration that varies across the field of view of the eye movement tracking camera.

[0012] The optical combiner can include a diffractive optical element (DOE) or a volume hologram.

[0013] The non-visible light can include infrared light or near-infrared light.

[0014] In one aspect of the application, there is provided an eye movement tracking camera, comprising: an image sensor configured to capture an image; and an optical assembly for receiving non-visible light reflected or scattered by an eye and for directing the non-visible light along an optical path to the image sensor, the non-visible light being received from an optical combiner of the eye movement tracking system, wherein the optical assembly comprises: an aperture stop disposed on the optical path; a first optical surface disposed on the optical path proximate to the aperture stop, wherein the first optical surface is configured to correct for field-independent optical aberrations caused by the optical combiner; and a second optical surface disposed on the optical path, wherein the second optical surface is configured to correct for field-dependent optical aberrations caused by the optical combiner.

[0015] The optical assembly can further include a first lens disposed on the optical path, wherein the first optical surface can be a rotationally symmetric, aspherical, and deformed surface of the first lens, and a second lens disposed on the optical path, wherein the second optical surface can be a freeform optical surface of the second lens.

[0016] The optical assembly can further include a prism disposed on the optical path between the aperture stop and the image sensor, wherein the prism can be configured to provide both the first optical surface and the second optical surface.

[0017] The second optical surface can be configured to reflect the non-visible light to the image sensor.

[0018] The field-independent optical aberration caused by the optical combiner can include a substantially uniform astigmatism aberration across a field of view of the eye tracking camera, and wherein the field-dependent optical aberration caused by the optical combiner can include a varying astigmatism aberration across the field of view of the eye tracking camera.

[0019] The non-visible light can include infrared light or near-infrared light.

[0020] In one aspect of the disclosure, an optical assembly for an eye tracking camera is provided, the optical assembly comprising: an aperture stop to receive non-visible light reflected by an eye, the non-visible light received from an optical combiner of an eye tracking system; a first optical surface disposed on an optical path between the aperture stop and an image sensor of the eye tracking system, wherein the first optical surface is configured to correct a field-independent optical aberration caused by the optical combiner; and a second optical surface disposed on the optical path between the first optical surface and the image sensor, wherein the second optical surface is configured to correct a field-dependent optical aberration caused by the optical combiner.

[0021] The optical assembly can further comprise: a lens disposed on the optical path between the aperture stop and the second optical surface, wherein the first optical surface can be a rotationally symmetric, aspheric, and anamorphic surface of the first lens; and a second lens disposed on the optical path between the first lens and the image sensor, wherein the second optical surface can be a freeform optical surface of the second lens.

[0022] The optical assembly can further comprise: a prism disposed on the optical path between the aperture stop and the image sensor, wherein the prism can be configured to provide the first optical surface and the second optical surface, and wherein the second optical surface can be configured to reflect the non-visible light to the image sensor.

[0023] The field-independent optical aberration caused by the optical combiner includes a substantially uniform astigmatism aberration across a field of view of the eye tracking camera, and wherein the field-dependent optical aberration caused by the optical combiner includes a varying astigmatism aberration across the field of view of the eye tracking camera. BRIEF DESCRIPTION OF DRAWINGS

[0024] Non-limiting and non-exhaustive embodiments of the present disclosure are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views.

[0025] Figure 1 An example head-mounted device is illustrated in accordance with aspects of the present disclosure.

[0026] Figure 2An example eye tracking system is illustrated in accordance with aspects of the disclosure.

[0027] Figure 3 An example front view of an eye passing through an example illumination layer is illustrated in accordance with aspects of the disclosure.

[0028] Figure 4 An example optical path of reflected invisible light is illustrated in accordance with aspects of the disclosure.

[0029] Figure 5A And Figure 5B An example optical assembly of an eye tracking camera is shown in accordance with aspects of the disclosure.

[0030] Figure 6 Another example optical assembly of an eye tracking camera is illustrated in accordance with aspects of the disclosure.

[0031] Figure 7 And Figure 8 is an example astigmatism plot image illustrating optical aberrations that can be caused by an optical combiner in accordance with aspects of the disclosure.

[0032] Figure 9 An example eye tracking camera is illustrated in accordance with aspects of the disclosure. DETAILED DESCRIPTION

[0033] Embodiments of optical assemblies, eye tracking cameras, eye tracking systems, and smart devices including optical combiner aberration correction in eye tracking imaging are described herein. In the following description, numerous specific details are set forth to provide a thorough understanding of the embodiments. One skilled in the relevant art will recognize, however, that the technology described herein can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.

[0034] Reference throughout the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Thus, the appearance of the phrases “in one embodiment” or “in an embodiment” in various places throughout the specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0035] In some implementations of the disclosure, the term “near-eye” can be defined to include elements configured to be placed within 50 mm of a user’s eye when utilizing the near-eye device. Thus, a “near-eye optical element” or “near-eye system” would include one or more elements configured to be placed within 50 mm of a user’s eye.

[0036] In aspects of the present disclosure, visible light can be defined as having a wavelength range of about 380-700 nm. Non-visible light can be defined as light having a wavelength outside the visible light range, such as ultraviolet light and infrared light. Infrared light has a wavelength range of about 700 nm-1 mm, which includes near-infrared light. In aspects of the present disclosure, near-infrared light can be defined as having a wavelength range of about 700 nm-1.4 pm.

[0037] An eye tracking system can include illuminating the eye with non-visible light, where the non-visible light reflected or scattered by the eye is received by one or more optical combiners. The optical combiners can be configured to then direct the non-visible light to an eye tracking camera for imaging. In various embodiments, the optical combiners can include a diffractive optical element (DOE) (e.g., a uniform grating, a Bragg grating, a blazed grating, a volume hologram, etc.). In operation, the optical combiners can be configured to diffract (reflect) certain wavelengths and / or polarizations of incident light while passing other wavelengths / polarizations of light. For example, the optical combiners can be configured to diffract the reflected non-visible light received from the eye toward the eye tracking camera while allowing visible light to pass. In some examples, the optical combiner is one layer in a surface stack that can have a coating that blocks non-visible light from the world side from entering the system. The eye tracking camera itself can include a bandpass filter to pass non-visible light to the image sensor while blocking / rejecting unwanted light from angles / wavelengths.

[0038] As described above, the optical combiners direct non-visible light to the eye tracking camera for imaging the eye. However, the optical combiners can cause optical aberrations when diffracting the non-visible light.

[0039] One type of optical aberration caused by the optical combiners can include a field-independent optical aberration, such as a uniform astigmatism aberration. In some examples, the field-independent optical aberration is substantially uniform across the field of view of the eye tracking camera. Another type of optical aberration caused by the optical combiners can include a field-dependent optical aberration, such as a non-uniform astigmatism aberration. This field-dependent optical aberration can be caused by the optical combiners as a result of the diffractive effects that change depending on the angle at which light is incident on the optical combiners. Thus, the diffractive effects at one point in the field of view can be different than the diffractive effects at another point in the field. Both the field-independent and field-dependent optical aberrations can distort the images captured by the eye tracking camera, hindering image analysis for eye tracking operations if not distorted.

[0040] Accordingly, aspects of the present disclosure include incorporating multiple optical surfaces into the optical assembly of an eye-tracking camera to correct for optical aberrations caused by the optical combiner used to image the eye. The optical assembly can include a first optical surface disposed on the optical path proximate the aperture stop to correct for field-independent optical aberrations (e.g., uniform astigmatism) caused by the optical combiner. The optical assembly can also include a second optical surface disposed on the optical path to correct for field-dependent optical aberrations (e.g., non-uniform astigmatism) also caused by the optical combiner. In one embodiment, the first and second optical surfaces are provided by respective first and second lenses. In another embodiment, the first and second optical surfaces are provided by a single monolithic prism that reflects non-visible light to the image sensor. These and other aspects are described in greater detail below.

[0041] Figure 1 An example head-mounted device 100 is illustrated in accordance with aspects of the present disclosure. A head-mounted device, such as head-mounted device 100, is a type of smart device that is typically worn on a user’s head to provide artificial reality content to the user. Artificial reality is a form of reality that has been adjusted in some manner before presentation to the user, which can include, e.g., a virtual reality (VR), an augmented reality (AR), a mixed reality (MR), a hybrid reality, or some combination and / or derivatives thereof.

[0042] The illustrated example of head-mounted device 100 includes a frame 102, temples 104A and 104B, and near-eye optical elements 110A and 110B. Eye-tracking cameras 108A and 108B are shown coupled to temples 104A and 104B, respectively. Figure 1 An exploded view of an example of near-eye optical element 110A is also illustrated. Near-eye optical element 110A is shown as including an optically transparent layer 120A, an illumination layer 130A, an optical combiner layer 140A, and a display layer 150A. Illumination layer 130A is shown as including a plurality of in- field light sources 126. In-field light sources 126 can be configured to emit non-visible light for eye-tracking purposes, for example. While Figure 1 Head-mounted device 100 is illustrated as including in-field light sources 126 disposed within the field of view, but in other examples, head-mounted device 100 can alternatively include light sources disposed outside the field of view, such as around the periphery of near-eye optical element 110A (e.g., incorporated within or near the edges of frame 102). In some examples, illumination layer 130A includes a blocking layer to block / prevent non-visible light received from back face 111. Display layer 150A can include a waveguide 158A configured to direct virtual images to the eye of a user of head-mounted device 100.

[0043] As Figure 1As shown, the frame 102 is coupled to the temples 104A and 104B for securing the headset 100 to a user’s head. The example headset 100 can also include support hardware incorporated into the frame 102 and / or the temples 104A and 104B. The hardware of the headset 100 can include any of processing logic, wired and / or wireless data interfaces for transmitting and receiving data, a graphics processor, and one or more memories for storing data and computer-executable instructions. In one example, the headset 100 can be configured to receive wired power and / or can be configured to be powered by one or more batteries. Further, the headset 100 can be configured to receive wired and / or wireless data including video data.

[0044] Figure 1 Near-eye optical elements 110A and 110B are illustrated, which are configured to be mounted to the frame 102. In some examples, the near-eye optical elements 110A and 110B can appear transparent to the user to facilitate augmented reality or mixed reality, such that the user can view visible scene light 191 from the environment while also receiving display light directed to the user’s eye(s) through the display layer 150A. In further examples, some or all of the near-eye optical elements 110A and 110B can be incorporated into a virtual reality headset, where the transparent nature of the near-eye optical elements 110A and 110B allow the user to view an electronic display (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a microLED display, etc.) incorporated in the virtual reality headset.

[0045] As Figure 1 The illumination layer 130A includes a plurality of in-field light sources 126, as shown. Each in-field light source 126 can be disposed on a transparent substrate and can be configured to emit non-visible light toward the eye-side 109 of the near-eye optical element 110A. In some aspects of the present disclosure, the in-field light sources 126 are configured to emit near-infrared light (e.g., 700 nm - 1.4 pm). Each in-field light source 126 can be a micro light-emitting diode (microLED), an edge-emitting LED, a vertical-cavity surface-emitting laser (VCSEL) diode, or a superluminescent diode (SLED). In other embodiments, each in-field light source 126 can be an exit feature of a waveguide included in the illumination layer 130A, where the waveguide is configured to direct non-visible light emitted from a VCSEL diode or other light source incorporated into the rim of the frame 102 to the exit feature for illuminating the user’s eye.

[0046] As noted above, in some examples, the head-mounted device 100 can provide light sources disposed around the periphery of the lenses. However, it can be advantageous to place the light sources within the field of view of the eye for the reflection of the mirror or "glint" that can be imaged by a camera, such as the eye tracking camera 108A, which is positioned to image the eye of the wearer of the head-mounted device 100.

[0047] While the in-field light sources 126 can introduce a small occlusion into the near-eye optical elements 110A, the in-field light sources 126, and their corresponding wiring, can be small enough to be unobtrusive or unnoticeable to the wearer of the head-mounted device 100. Additionally, any occlusion from the in-field light sources 126 will be placed so close to the eye that the human eye cannot perceive, and thus, helps the in-field light sources 126 to be unobtrusive or unnoticeable. In some embodiments, each in-field light source 126 has a footprint (or size) that is less than about 200 x 200 microns.

[0048] As described above, the in-field light source 126 of the illumination layer 130A can be configured to emit non-visible light toward the eye-ward side 109 of the near-eye optical element 110A to illuminate the user’s eye. The near-eye optical element 110A is shown to include an optical combiner layer 140A disposed between the illumination layer 130A and the back side 111 of the near-eye optical element 110A. In some aspects, the optical combiner layer 140A is configured to receive reflected non-visible light reflected by the user’s eye and direct the reflected non-visible light toward the eye movement tracking camera 108A. In examples in which the in-field light source 126 emits infrared light, the eye movement tracking camera 108A can be an infrared camera configured to image the user’s eye based on the received reflected infrared light. In some aspects, the optical combiner layer 140A can be transmissive to visible light, such as scene light 191 incident on the back side 111 of the near-eye optical element 110A. In some examples, the optical combiner layer 140A can be configured as a volume hologram and / or can include one or more diffractive gratings (e.g., Bragg gratings, blazed gratings, uniform gratings, etc.) for directing the reflected non-visible light to the eye movement tracking camera 108A. In some examples, the optical combiner layer 140A includes a polarization-selective hologram (also known as a polarization volume hologram) that diffracts a particular polarization orientation of incident light while passing through other polarization orientations. In other examples, the optical combiner layer 140A includes one or more Fresnel optical elements configured to direct the reflected non-visible light to the camera while also allowing visible light to propagate through the near-eye optical element 110A. As an example, the Fresnel optical elements included in the optical combiner layer 140A can include an active surface selectively coated with a “hot mirror” layer (reflecting non-visible light and transmitting visible light) to direct non-visible light to the camera while also transmitting visible light for viewing by a wearer of the head-mounted device 100.

[0049] Depending on the design of the head-mounted device 100, the display layer 150A can include one or more other optical elements. For example, the display layer 150A can include a waveguide 158A to direct display light generated by an electronic display to the user’s eye. In some implementations, at least a portion of the electronic display is included in the frame 102 of the head-mounted device 100. The electronic display can include an LCD, an organic light-emitting diode (OLED) display, a micro-LED display, a micro-projector, or a liquid crystal on silicon (LCOS) display for generating display light.

[0050] An optically clear layer 120A is shown disposed between an illumination layer 130A and the eye-facing side 109 of a near-eye optical element 110A. The optically clear layer 120A can receive invisible light emitted by the illumination layer 130A and allow the invisible illumination light to pass through to illuminate the user's eye. As described above, the optically clear layer 120A can also be transparent to visible light (such as scene light 191 received from the environment and / or display light received from the display layer 150A). In some examples, the optically clear layer 120A has a curvature for focusing light (e.g., display light and / or scene light) onto the user's eye. Therefore, in some examples, the optically clear layer 120A can be referred to as a lens. In some aspects, the optically clear layer 120A has a thickness and / or curvature corresponding to user specifications. In other words, the optically clear layer 120A can be a prescription lens. However, in other examples, the optically clear layer 120A can be a non-prescription lens. In some examples, the back side 111 of the optically transparent layer 120A may have optical power, wherein the eye-facing side 109 of the optically transparent layer 120A may include a curved surface for counteracting the power caused by the back side surface.

[0051] Figure 2 An example eye-tracking system 200 according to various aspects of this disclosure is illustrated. The illustrated example of the eye-tracking system 200 is shown as including an eye-tracking camera 108A and a near-eye optics element 210. The near-eye optics element 210 is... Figure 1 A possible example of near-eye optical elements 110A and / or 110B. Figure 2 The near-eye optical element 210 is shown to include a transparent layer 220, an illumination layer 230, an optical combiner layer 240, and a display layer 250. An in-field light source 237 is configured to emit invisible light 239 toward an ocular region 207 on the eye-side 109 of the near-eye optical element 210 to illuminate the eye 206. In some examples, different in-field light sources 237 of the illumination layer 230 may direct the invisible light 239 to the eye 206 at different angles depending on the position of the in-field light source 237 relative to the eye 206. For example, each in-field light source 237 may include a corresponding beamforming element that directs the invisible light 239 to the eye 206 at a different angle than other in-field light sources 237 included in the illumination layer 230. As described above, the in-field light source 237 may be a VCSEL or an SLED, and therefore the invisible light 239 may be narrowband infrared illumination light (e.g., a linewidth of 1-10 nm).

[0052] The illumination layer 230 may include a transparent material 232 that encapsulates the field light source 237. The transparent material 232 and the refractive material 222 may be configured to transmit visible light (e.g., 400 nm-700 nm) and near-infrared light (e.g., 700 nm-1.4 μm).

[0053] Figure 3 A front view of the eye 206 through the example illumination layer 230 is illustrated in accordance with aspects of the disclosure. In the illustrated embodiment, the in- field light sources 237 are arranged in a grid within the field of view of the eye 206. However, in other examples, the in-field light sources 237 can be arranged in various patterns, such as a sparse pattern or concentric rings.

[0054] Figure 4 An example light path 460 of the invisible light 239 of the eye tracking system 200, for example, is illustrated in accordance with aspects of the disclosure. As shown, the in- field light source 237A generates the invisible light 239, which is then directed along the light path 460(1) to the eyebox region 207. Figure 4 Figure 4 The invisible light 239 is also illustrated reflecting off of the eye 206, and then returning to the near-eye optical element 210 along the light path 460(2). The reflected invisible light then passes through the transparent layer 420, through the illumination layer 430, and then is incident on the optical combiner layer 440. As shown, the optical combiner layer 440 then directs the reflected invisible light 239 along the light path 460(3) to the eye tracking camera 108A on the to-eye side 109.

[0055] The eye tracking camera 108A is configured to capture images of the eye 206 for eye tracking operations. In some examples, the eye tracking camera 108A includes a bandpass filter having a center wavelength corresponding to the wavelength of the invisible light 239. Thus, the bandpass filter can pass the wavelength of the invisible light 239 emitted by the in-field light source, and block other light from being incident on the image sensor of the eye tracking camera 108A. The eye tracking camera 108A can include a complementary metal-oxide-semiconductor (CMOS) image sensor.

[0056] Figure 4 It is also shown that scene light 191 from the external environment can propagate through the display layer 250, the optical combiner layer 240, the illumination layer 230, and the transparent layer 220 to be incident on the eye 206, such that the user can view the scene of the external environment.

[0057] As described above, the optical combiner of the eye tracking system (e.g., the optical combiner layer 240) is configured to direct the invisible light (e.g., by diffraction) to the eye tracking camera 108A for imaging, which can cause field-independent and field-dependent optical aberrations, which can distort the images captured by the eye tracking camera 108A. Thus, the eye tracking camera 108A can include optical components that match the optical aberrations caused by the optical combiner to correct for these optical aberrations.

[0058] As an example, Figure 5A and Figure 5B ​An optical assembly 500 of an eye-tracking camera is illustrated in accordance with aspects of the disclosure. Figure 5A and Figure 5B The example optical assembly 500 is illustrated as including an aperture stop 508, a first lens 506, and a second lens 504. Figure 5A and Figure 5B The image sensor 502 is also shown in FIG. 5. The optical assembly 500 and the image sensor 502 together are one possible implementation of the eye-tracking camera 108A of Figure 1 Although the optical assembly 500 is shown as including only two lenses, in some embodiments, the optical assembly 500 can include any number of lenses, including two or more.

[0059] As shown in FIG. 5, the aperture stop 508 is configured to receive the non-visible light 239 from an optical combiner (e.g., the optical combiner layer 240 of Figure 5A and Figure 5B The aperture stop 508 includes an aperture or opening through which the non-visible light 239 passes and can be utilized within the optical assembly 500 to control the cone angle, depth of field, stray light, etc. In some embodiments, the aperture stop 508 can be an opaque sticker or decal having an aperture placed on or near the first lens 506. In another example, the aperture stop 508 is ink, blackened aluminum, black copper, or other coating applied to the first lens 506. Figure 2 The first lens 506 is shown as being disposed in the optical path of the non-visible light 239 between the aperture stop 508 and the image sensor 502. In particular, the first lens 506 can be disposed between the aperture stop 508 and the second lens 504. The first lens 506 can be a plastic or glass lens. As shown in FIG. 5, the first lens 506 includes optical surfaces 503 and 505. One or both of the optical surfaces 503 / 505 are configured to correct for field-independent optical aberrations (e.g., uniform astigmatic aberrations) caused by the optical combiner. One or more of the optical surfaces 503 and 505 can include a rotationally symmetric, freeform, aspheric, and / or anamorphic surface profile. For example, in one embodiment, the optical surface 503 is an anamorphic optical surface, while the optical surface 503 is an aspheric optical surface.

[0060] Figure 5A As used herein, an aspheric optical surface is a surface profile that is not a portion of a sphere or cylinder. Further, an anamorphic optical surface is an optical surface that optically distorts an image (e.g., has different optical power in at least two directions). If the imaging properties of an optical surface are not invariant with respect to any rotation about an axis (e.g., about the optical axis of the optical surface), then the optical surface is anamorphic.

[0061] As used herein, an aspheric optical surface is a surface profile that is not a portion of a sphere or cylinder. Further, an anamorphic optical surface is an optical surface that optically distorts an image (e.g., has different optical power in at least two directions). If the imaging properties of an optical surface are not invariant with respect to any rotation about an axis (e.g., about the optical axis of the optical surface), then the optical surface is anamorphic. Figure 5A ​An optical surface can be considered "rotationally symmetric" if it has rotational symmetry about an axis (e.g., the x-axis) that changes as a function of position on the surface. A "freeform optical surface" can include an optical surface that has minimal bilateral symmetry and can not have a plane of symmetry.

[0062] The second lens 504 is shown disposed in the optical path of the non-visible light 239 between the optical surface 503 / 505 of the first lens 506 and the image sensor 502. The second lens 504 can be a plastic or glass lens. As shown, the second lens 504 includes optical surfaces 507 and 509. One or both of the optical surfaces 507 / 509 are configured to correct for field-dependent optical aberrations (e.g., non-uniform astigmatism aberrations) caused by the optical combiner. One or more of the optical surfaces 507 and 509 can include a rotationally asymmetric, aspheric, and / or freeform surface profile. For example, in one embodiment, the optical surface 507 is a planar optical surface and the optical surface 509 is a rotationally asymmetric freeform optical surface. In another embodiment, the optical surface 507 is an aspheric optical surface. Figure 5A

[0063] As used herein, a freeform optical surface is a surface profile that has no translational or rotational symmetry. Further, a freeform optical surface can include a surface profile that can be mathematically described by one or more polynomials. In some examples, a freeform optical surface can be designed to correct for field-dependent aberrations caused by a particular optical combiner used in an eye-tracking system. Figure 5A In the illustrated example, the optical surface 509 is shown as a freeform optical surface that is a Zernike-polynomial optical surface. However, the optical surface 509 can be designed as other freeform optical surfaces, including an XY-polynomial optical surface, a Chebyshev-polynomial optical surface, etc. Alternatively, the optical surface 509 can be an off-axis segment of a metamorphic optical surface. The optical surface 509 is configured to correct for field-dependent aberrations caused by the particular optical combiner used in the eye-tracking system.

[0064] Figure 5B An orthogonal rotated view (about the x-axis) of the optical assembly 500 is illustrated. Figure 5A Figure 5B The rotational asymmetry of the optical surface 509 about the x-axis is illustrated.

[0065] In some examples, to reduce the size of the optical assembly 500, the optical surface 509 is disposed near the image plane proximate the image sensor 502, where the footprint of the field points are separated from one another (e.g., as shown in the example of FIG. 6A). However, in other examples, the optical surface 509 can be disposed at an appropriate location in front of the aperture stop 508 to perform the field-dependent correction. Figure 5A and Figure 5B In some examples, to reduce the size of the optical assembly 500, the optical surface 509 is disposed near the image plane proximate the image sensor 502, where the footprint of the field points are separated from one another (e.g., as shown in the example of FIG. 6A). However, in other examples, the optical surface 509 can be disposed at an appropriate location in front of the aperture stop 508 to perform the field-dependent correction.

[0066] ​​In some implementations, the first lens 506 includes a uniform diffractive element for correcting chromatic aberration due to dispersion from the optical combiner. For higher order aberrations, the uniform diffractive element can further extend to a freeform diffractive element pattern. Even so, the first lens 506 can include a freeform diffractive element for correcting monochromatic aberrations, here assuming no chromatic aberration to correct. In this example, such a freeform diffractive element can handle non-rotationally symmetric aberrations, while the rest of the system can be constructed from rotationally symmetric elements.

[0067] Further, although Figure 5A and Figure 5B described above as including two optical surfaces 503 / 505 for providing field independent correction and field dependent correction, respectively, in other examples, additional surfaces can be configured to provide focusing power and aberration correction. For example, as Figure 5A and Figure 5B illustrated, there is a rotationally symmetric surface that performs most of the focusing power of the lens assembly. However, the surface 507 on the second lens 504 can alternatively include a free surface to assist other free surfaces (e.g., surface 509) in correcting field dependent aberrations.

[0068] Figure 6 FIG. illustrates another example optical assembly 600 of an eye tracking camera in accordance with aspects of the disclosure. Figure 6 The example optical assembly 600 is illustrated as including an aperture stop 508 and a prism 604. Together, the optical assembly 600 and the image sensor 502 are Figure 1 an eye tracking camera 108A of FIG. 1 in accordance with aspects of the disclosure. In some embodiments, the optical assembly 600 can include one or more additional optical elements (e.g., lenses, filters, etc.) not explicitly shown in FIG. 1. Figure 6

[0069] The prism 604 can be a single monolithic piece of transparent material (such as plastic or glass) disposed in the optical path of the non-visible light 239 between the aperture stop 508 and the image sensor 502. As Figure 6 illustrated, the prism includes optical surfaces 603 and 605. One or both of the optical surfaces 603 / 605 are configured to correct for field dependent optical aberrations (e.g., non-uniform astigmatism aberrations) caused by the optical combiner. One or more of the optical surfaces 603 and 605 can include a rotationally asymmetric, aspheric, and / or freeform surface profile. For example, in one embodiment, the optical surface 603 is an aspheric optical surface, while the optical surface 605 is a rotationally non-symmetric freeform optical surface. In Figure 6 ​In the illustrated example, optical surface 603 is configured to direct non-visible light 239 to optical surface 605 by refraction, while optical surface 605 is configured to direct non-visible light 239 to image sensor 502 by reflection. Thus, in some examples, prism 604 can include a reflective coating 606 (e.g., metal) disposed on optical surface 605.

[0070] Figure 7 and Figure 8 are example astigmatic line images 700 / 800 that illustrate optical aberrations that can be caused by an optical combiner. As described above, an optical combiner, such as optical combiner 140A of Figure 1 , can cause both field-independent and field-dependent optical aberrations for imaging. One type of optical aberration includes an astigmatic aberration. Figure 7 and Figure 8 illustrate example test images 700 / 800 of a set of uniformly distributed points imaged via an optical combiner. That is, an “ideal” image would simply illustrate an array of points. However, the optical combiner causes an astigmatic aberration that causes the points to be “stretched” along one or more of the x and y axes. Thus, Figure 7 and Figure 8 the length of each line illustrated in FIGS. 700 / 800 represents the magnitude of the astigmatic aberration caused at the field point, with longer lines representing greater amounts of astigmatic distortion.

[0071] Figure 7 represents an astigmatic line image 700 obtained via an optical combiner without optical aberration correction (e.g., non-visible light 239 received at aperture stop 508 of optical assembly 500 of Figure 5A ). As illustrated, astigmatic line image 700 illustrates a dominant astigmatic optical aberration generally along the y-axis. As further illustrated in Figure 7 , the astigmatic optical aberration along the y-axis is substantially uniform across the field of view, and thus is referred to herein as a field-independent optical aberration. Accordingly, a first optical surface of the optical assembly (e.g., optical surface 505 of Figure 5A ) can be configured to correct for these field-independent optical aberrations.

[0072] Figure 8 represents an astigmatic line image 800 obtained after correction for the field-independent optical aberrations (e.g., non-visible light 239 received at second lens 504 of Figure 5A ). As illustrated in Figure 8 , some residual astigmatic distortion remains in the image. However, the astigmatic optical aberration remaining in image 800 varies in magnitude and in direction across the field of view, and thus can be referred to herein as a field-dependent optical aberration. Accordingly, a second optical surface of the optical assembly, such as optical surface 507 of Figure 5A or optical surface 605 of Figure 6optical surface 605 can be configured to correct these field-dependent optical aberrations.

[0073] Figure 9 An example eye-tracking camera 900 is illustrated in accordance with aspects of the present disclosure. The eye-tracking camera 900 is one possible implementation of the eye-tracking camera 108A Figure 1 of FIG. 1. The illustrated example of the eye-tracking camera 900 is shown to include a housing 902, a first lens 904, a second lens 906, an image sensor 908, and a printed circuit board (PCB) 910. The first lens 904 is shown to include a first optical surface 905, and the second lens 906 is shown to include a second optical surface 907. The first lens 904 and the second lens 906 can be collectively referred to herein as optical components of the eye-tracking camera 900. Figure 9 The optical axis 909 of the optical components of the eye-tracking camera 900 is also shown in FIG. 9, where an image plane of the image sensor 908 is perpendicular to the optical axis 909.

[0074] In some embodiments, the housing 902 is plastic or metal and is configured to house the first lens 904 and the second lens 906, as well as the image sensor 908. In some examples, the housing 902 includes one or more registration features and / or spacers (not shown) that in turn provide alignment (e.g., centering) and spacing (e.g., axial positioning) of the various optical components relative to one another. The housing 902 can also be configured to block and / or absorb stray light. The illustrated example of the housing 902 is also shown to provide an aperture stop through an aperture 912. As shown, the aperture stop is configured (e.g., aligned) to receive the non-visible light 239 from the optical combiner of the eye-tracking system. Figure 9

[0075] In the illustrated example, the first lens 904 and the second lens 906 are configured to direct the received non-visible light 239 to the image sensor 908 for imaging of the eye, where the first optical surface 905 is configured to correct for field-independent optical aberrations and the second optical surface 907 is configured to correct for field-dependent optical aberrations. As described above, the first optical surface 905 and the second optical surface 907 are configured to match the particular optical aberrations caused by the particular optical combiner used to direct the non-visible light 239 to the eye-tracking camera 900. Further, in some examples, a centerline 911 of the image sensor 908 can be offset from the optical axis 909 to account for the angle (e.g., optical CANT) at which the eye-tracking camera 900 is positioned relative to the optical combiner (e.g., see FIG. 8). Figure 1

[0076] ​​Embodiments of the application can include or be implemented in conjunction with an artificial reality system. Artificial reality is a form of reality that has been adjusted in some manner before presentation to a user, which can include, e.g., a virtual reality (VR), an augmented reality (AR), a mixed reality (MR), a hybrid reality, or some combination and / or derivatives thereof. Artificial reality content can include completely generated content or generated content combined with captured (e.g., real-world) content. Artificial reality content can include video, audio, haptic feedback, or some combination thereof, and can be presented in a single channel or in multiple channels (such as stereo videos that produce a three-dimensional effect to the viewer). Additionally, in some embodiments, artificial reality can also be associated with applications, products, accessories, services, or some combination thereof, that are used to create content for or in an artificial reality (e.g., performing activities in an artificial reality) or that are used in in conjunction with the artificial reality (e.g., applications for use in an artificial reality). The artificial reality system that provides the artificial reality content can be implemented on various platforms, including a head-mounted display (HMD) connected to a host computer system, a standalone HMD, a mobile device or computing system, or any other hardware platform capable of providing artificial reality content to one or more viewers.

[0077] The above description of illustrated embodiments of the application, including what is described in the abstract, is not intended to be exhaustive or to limit the application to the precise forms disclosed. While specific embodiments of, and examples for, the application are described herein for illustrative purposes, various modifications are possible within the scope of the application, as those skilled in the relevant art will recognize.

[0078] These modifications can be made in light of the above detailed description of the application. The terms used in the following claims should not be construed to limit the application to the specific embodiments disclosed in the specification and the drawings. Rather, the scope of the application is to be determined entirely by the following claims, which are to be construed in accordance with the principles of claim interpretation.

Claims

1. An eye-tracking system, comprising: Eye-tracking camera, including an image sensor; as well as An optical combiner is configured to receive invisible light reflected or scattered by the eye and guide the invisible light from a layer included in a stack of near-eye optical elements to the eye-tracking camera, wherein the eye-tracking camera further includes: An optical component for receiving the invisible light from the optical combiner and guiding the invisible light along an optical path to the image sensor, wherein the optical component includes: An aperture stop is disposed in the optical path; A first optical surface is disposed in the optical path, wherein the first optical surface is configured to correct field-independent optical aberrations caused by the optical combiner; and A second optical surface is disposed in the optical path, wherein the second optical surface is configured to correct field-related optical aberrations caused by the optical combiner.

2. The eye-tracking system of claim 1, wherein the first optical surface comprises one or more of the following surface profiles: rotationally symmetric, aspherical, free-form, or deformable.

3. The eye-tracking system according to claim 1, wherein the second optical surface is a freeform optical surface.

4. The eye-tracking system according to claim 3, wherein the freeform optical surface is a Zernike-polynomial optical surface, an XY-polynomial optical surface, a Chebyshev-polynomial optical surface, or an off-axis segment of a deformable optical surface.

5. The eye-tracking system of claim 1, wherein the optical component further comprises: A first lens is disposed in the optical path, wherein the first optical surface is the surface of the first lens; as well as A second lens is disposed in the optical path, wherein the second optical surface is the surface of the second lens.

6. The eye-tracking system of claim 1, wherein the optical component further comprises: A prism is disposed in the optical path between the aperture stop and the image sensor, wherein the prism is configured to provide both the first optical surface and the second optical surface.

7. The eye-tracking system of claim 6, wherein the second optical surface is configured to reflect the invisible light to the image sensor.

8. The eye-tracking system of claim 1, wherein the field-independent optical aberrations caused by the optical combiner include astigmatic aberrations that are substantially uniform across the field of view of the eye-tracking camera, and wherein the field-related optical aberrations caused by the optical combiner include astigmatic aberrations that vary across the field of view of the eye-tracking camera.

9. The eye-tracking system of claim 1, wherein the optical combiner comprises a diffractive optical element (DOE) or a volumetric hologram.

10. The eye-tracking system of claim 1, wherein the invisible light includes infrared light or near-infrared light.

11. An eye-tracking camera, comprising: An image sensor is configured to capture images; as well as Optical components for receiving invisible light reflected or scattered by the eye and for guiding the invisible light along an optical path to the image sensor, the invisible light being received from an optical combiner included in a layer of a stack of near-eye optics of the eye-tracking system, wherein the optical components include: An aperture stop is disposed in the optical path; A first optical surface is disposed in the optical path near the aperture stop, wherein the first optical surface is configured to correct field-independent optical aberrations caused by the optical combiner; and A second optical surface is disposed in the optical path, wherein the second optical surface is configured to correct field-related optical aberrations caused by the optical combiner.

12. The eye-tracking camera of claim 11, wherein the optical components further include: A first lens is disposed on the optical path, wherein the first optical surface is a rotationally symmetric, aspherical, and deformable surface of the first lens; as well as A second lens is disposed in the optical path, wherein the second optical surface is a freeform optical surface of the second lens.

13. The eye-tracking camera of claim 11, wherein the optical component further comprises a prism disposed in the optical path between the aperture stop and the image sensor, wherein the prism is configured to provide both the first optical surface and the second optical surface.

14. The eye-tracking camera of claim 13, wherein the second optical surface is configured to reflect the invisible light onto the image sensor.

15. The eye-tracking camera of claim 11, wherein the field-independent optical aberrations caused by the optical combiner include astigmatic aberrations that are substantially uniform across the field of view of the eye-tracking camera, and wherein the field-related optical aberrations caused by the optical combiner include astigmatic aberrations that vary across the field of view of the eye-tracking camera.

16. The eye-tracking camera of claim 11, wherein the invisible light includes infrared light or near-infrared light.

17. An optical component for an eye-tracking camera, the optical component comprising: An aperture stop is used to receive invisible light reflected by the eye, which is received from an optical combiner included in the layers of a stack of near-eye optics of the eye-tracking system. A first optical surface is disposed in the optical path between the aperture stop and the image sensor of the eye-tracking system, wherein the first optical surface is configured to correct field-independent optical aberrations caused by the optical combiner; as well as A second optical surface is disposed in the optical path between the first optical surface and the image sensor, wherein the second optical surface is configured to correct field-related optical aberrations caused by the optical combiner.

18. The optical component of claim 17, further comprising: A first lens is disposed in the optical path between the aperture stop and the second optical surface, wherein the first optical surface is a rotationally symmetric, aspherical, and deformable surface of the first lens; as well as A second lens is disposed in the optical path between the first lens and the image sensor, wherein the second optical surface is a freeform optical surface of the second lens.

19. The optical component of claim 17, further comprising: A prism is disposed in the optical path between the aperture stop and the image sensor, wherein the prism is configured to provide a first optical surface and a second optical surface, and wherein the second optical surface is configured to reflect the invisible light to the image sensor.

20. The optical assembly of claim 17, wherein the field-independent optical aberrations caused by the optical combiner include astigmatic aberrations that are substantially uniform across the field of view of the eye-tracking camera, and wherein the field-related optical aberrations caused by the optical combiner include astigmatic aberrations that vary across the field of view of the eye-tracking camera.

Images