Eye tracking using camera lens aligned retinal illumination

Abstract

Various implementations disclosed herein include devices, systems, and methods of capturing an image of an illuminated retina and performing eye tracking using the image. For example, a newly captured image may be compared to a previously captured retinal image or model to determine a three-dimensional (3D) position or orientation of the eye relative to the camera / tracking system. Diffused light is directed toward the retina to produce reflections captured by the camera. The diffused light is directed from a position that is better aligned with the camera than existing retinal imaging techniques. For example, at least some of the diffused light may be directed toward the retina from one or more locations that are less than a camera lens aperture radius distance from the camera lens optical axis.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application Serial No. 63 / 342,322 filed on May 16, 2022 and U.S. Patent Application Serial No. 18 / 143,213 filed on May 4, 2023, each of which is incorporated herein in its entirety. Technical Field

[0003] The present disclosure relates generally to electronic devices, and particularly to systems, methods, and devices for tracking eye characteristics of users of electronic devices. Background Art

[0004] Some existing eye tracking technologies generate light that is reflected from a user's eye (e.g., typically the user's cornea) as one or more flashes that are captured in an image via an image sensor. The flash patterns in the image can be analyzed to determine the position or orientation of the user's eye. Existing tracking systems may lack efficiency, accuracy, or other characteristics desired for various eye tracking applications. Summary of the invention

[0005] Various embodiments disclosed herein include devices, systems, and methods that capture an image of an illuminated retina and use the image to perform eye tracking. For example, a newly captured image can be compared to a previously captured retinal image or model to determine the three-dimensional (3D) position or orientation of the eye relative to a camera / tracking system or the surrounding environment. The light is intended to illuminate the retina, which is then imaged by the camera. The diffuse light can be directed from a position that is better aligned with the camera than existing eye tracking technologies. For example, at least some of the diffuse light can be directed toward the retina from inside the working-Numerical-Aperture of the lens, meaning that the illumination is positioned close to the optical axis, inside the clear aperture of the lens, or directly in front of the clear aperture. Some embodiments provide eye tracking capabilities using one or more modular camera attachment enabled optical (MCO) devices that are small enough to be used on head-mounted devices and other devices that are sensitive to size constraints.

[0006] Some specific implementations relate to a retinal imaging device having a camera, a light source, and a scattering optic for generating diffuse light toward the retina of an eye. The camera has a lens having an optical axis and a clear aperture radius (the radius of the entrance pupil of the lens). At least some of the diffuse light is directed toward the retina from a location that is less than the lens aperture radius distance from the lens optical axis, and the diffuse light is thus better aligned with the optical axis of the camera. The scattering optic can be small to avoid / limit interference with the light captured by the camera and to avoid significantly increasing the size of the device to accommodate the generation of the diffuse light. The diffuse light can be generated from a location closer to the optical axis of the lens, thereby providing better retinal imaging, especially when the pupil is constricted, without significantly increasing the size of the device. The light can also be polarized to reduce / avoid glare / ghost corneal reflections.

[0007] Some embodiments provide a device including a camera having a chamber with an aperture, the aperture being equipped with a lens, through which captured light is received to form an image, which is projected onto a surface for recording or conversion into electrical pulses. The camera lens has a lens optical axis and a lens aperture radius. These exemplary devices may include a light source and a scattering optical device. The light source may be configured to generate light directed toward the scattering optical device. The scattering optical device may be positioned and configured to generate diffuse light by scattering light generated by the light source, wherein at least some of the diffuse light is directed from a position around the optical axis and closer to the optical axis than the aperture radius (e.g., within a lens aperture radius distance from the optical axis) and directed toward the retina of the eye. The captured light includes reflections of the diffuse light off the retina. The device may also include one or more processors configured to track the eye based on the image.

[0008] Some embodiments provide a device including a camera and an outward light source configured and positioned to generate light such that at least some of the generated light is directed toward a retina. The light source may be configured to avoid / limit interference with light captured by the camera. For example, a device may include a camera having a chamber with an aperture, the aperture being equipped with a lens, through which captured light is received to form an image, the image being projected onto a surface for recording or conversion into electrical impulses, the camera lens having a lens optical axis and a lens aperture radius. The device may include a light source configured to generate diffuse light, wherein at least some of the diffuse light is generated from the light source at a distance from the lens optical axis that is less than the lens aperture radius and is directed toward a retina of an eye. The captured light includes reflections of the diffuse light off the retina. The device may include one or more processors configured to track the eye based on the image.

[0009] Some embodiments provide an eye tracking method. The method may involve generating diffuse light directed toward a retina of an eye. The method may also involve generating an image of the retina using a camera including a lens having a lens optical axis and a lens aperture radius distance, wherein the image is generated by capturing reflections of the diffuse light off the retina of the eye. At least some of the diffuse light may be directed from a location less than the lens aperture radius distance from the lens optical axis. The method may track the eye (e.g., 3D position, orientation, retinal properties, etc. of the eye) based on the image.

[0010] According to some specific implementations, a non-transitory computer-readable storage medium stores instructions, which are computer-executable to perform or cause the performance of any method described herein. According to some specific implementations, a device includes one or more processors, a non-transitory memory, and one or more programs; the one or more programs are stored in the non-transitory memory and are configured to be executed by one or more processors, and the one or more programs include instructions for performing or causing the performance of any of the methods described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] So that the present disclosure can be understood by those of ordinary skill in the art, a more detailed description may be obtained with reference to aspects of some exemplary implementations, some of which are illustrated in the accompanying drawings.

[0012] Figure 1 Example devices according to some implementations are shown.

[0013] FIG. 2A to FIG. 2B FIG. 1 shows an example of performing eye tracking according to some implementations. Figure 1 An exemplary device of.

[0014] FIG. 3A to FIG. 3B Shown are portions of an eye captured using off-axis illumination given different pupil sizes.

[0015] FIG. 4A to FIG. 4B An exemplary eye tracking device according to some implementations is shown.

[0016] Figure 5A , Figure 5B , Figure 5C Additional example eye tracking devices are shown according to some implementations.

[0017] Figure 6 An exemplary eye tracking device according to some implementations is shown.

[0018] Figure 7 According to some specific implementations Figure 6 Light diffusion for eye tracking devices.

[0019] Figure 8 An exemplary eye tracking device according to some implementations is shown.

[0020] Fig. 9 An exemplary eye tracking device according to some implementations is shown.

[0021] Fig.10 An exemplary eye tracking device according to some implementations is shown.

[0022] Fig.11 An exemplary eye tracking device according to some implementations is shown.

[0023] FIG. 12A to FIG. 12B Exemplary lens configurations are shown according to some implementations.

[0024] Fig.13 An exemplary eye tracking device according to some implementations is shown.

[0025] Fig.14 According to some specific implementations Fig.13 Attachment of a light source in an exemplary eye tracking device.

[0026] Fig.15 An exemplary eye tracking device according to some implementations is shown.

[0027] Fig.16 is a flowchart representation of a method for tracking eye characteristics according to some specific implementations.

[0028] Fig.17 is a block diagram of an example electronic device according to some implementations.

[0029] As is common practice, the various features shown in the drawings may not be drawn to scale. Therefore, the sizes of the various features may be arbitrarily expanded or reduced for clarity. In addition, some of the drawings may not depict all of the components of a given system, method, or device. Finally, throughout the specification and drawings, similar reference numerals may be used to represent similar features. DETAILED DESCRIPTION

[0030] Many details are described in order to provide a thorough understanding of the example implementations shown in the accompanying drawings. However, the accompanying drawings only illustrate some example aspects of the present disclosure and should not be considered limiting. One of ordinary skill in the art will appreciate that other effective aspects or variations do not include all of the specific details described herein. In addition, well-known systems, methods, components, devices, and circuits are not described in detail in order to avoid obscuring more relevant aspects of the example implementations described herein.

[0031] Figure 1An example environment 100 is shown that includes a device 120. In some implementations, the device 120 displays content to a user 110. For example, the content may include a user interface or a portion thereof, such as a button, a user interface icon, a text box, a graphic, etc., and in some implementations, the content may occupy the entire display area of ​​a display of the device 120. The device 120 may obtain image data, motion data, and / or physiological data from the user 110 via one or more sensors. For example, the device 120 may obtain eye characteristic data via an eye tracking module. Such an eye tracking module may include one or more illumination components (e.g., a light source, a scattering optical device, etc.) and a camera component (e.g., a light sensor, a lens, a polarizer, etc.).

[0032] Although this example and other examples discussed herein show a single device 120, the techniques disclosed herein may also utilize multiple devices. For example, the eye tracking functionality of device 120 may be performed by multiple devices, e.g., with cameras, light sources, and / or scattering optics on each respective device, or divided among them in any combination.

[0033] In some specific implementations, such as Figure 1 As shown, device 120 is a handheld electronic device (e.g., a smartphone or tablet). In some implementations, device 120 is a laptop or desktop computer. In some implementations, device 120 has a trackpad, and in some implementations, device 120 has a touch-sensitive display (also referred to as a "touch screen" or "touch screen display"). In some implementations, device 10 is a wearable device such as a head mounted device (HMD).

[0034] In some implementations, the device 120 includes an eye tracking system for detecting eye characteristics such as eye position and eye movement. For example, the eye tracking system may include an eye tracking camera (e.g., an IR camera or a near infrared (NIR) camera) and an illumination source (e.g., an IR light source or a NIR light source) that emits light toward the eyes of the user 110. The illumination source of the device 120 may emit light that is directed (e.g., via scattering optics) to illuminate the retina of the eye of the user 110, and the camera may capture an image of the retina by capturing reflections of the light off the retina. In some implementations, the images captured by the eye tracking system may be analyzed to detect the position and movement of the eyes of the user 110, or to detect other information about the eyes, such as medical information, such as retinal health, retinal changes, cholesterol status, etc. In addition, in some implementations, retinal imaging is used to determine the 3D orientation of one or both eyes, which can be used to determine gaze direction, identify objects that the user 110 is looking at, identify changes in gaze, determine gaze speed, etc.

[0035] In some implementations, the device 120 has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or instruction sets stored in the memory for performing multiple functions. In some implementations, the user 110 interacts with the GUI by providing input, for example, via gestures and / or gaze-based input. In some implementations, these functions include image editing, drawing, presentation, word processing, web page creation, disk editing, spreadsheet creation, playing games, making and receiving calls, video conferencing, sending and receiving emails, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playback, and / or digital video playback. Executable instructions for performing these functions may be included in a computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0036] FIG. 2A to FIG. 2B 206 of the eye 205 when the eye 205 is in different orientations. Figure 1 The device 120 includes a display 210 and an eye tracking system 220. The eye tracking system 220 uses a light source and / or scattering optics to direct diffuse light 250a to 250c through the pupil 207 and onto the retina 206. Additionally, the eye tracking system 220 includes a camera (e.g., an image sensor) to observe the light 250a to 250c after it is reflected off the retina 206 of the eye 205 in order to acquire one or more images of the retina 206. The image can depict blood vessels and other structures and characteristics of the retina 206. Figure 2A and Figure 2B A comparison of shows how the portion of the retina that is illuminated and captured in the image will depend on the orientation of the eye 205 and the size of the pupil 207 opening.

[0037] It is desirable to capture a relatively large portion of the central portion of the retina 206 in the image. Capturing a relatively large portion of the central portion of the retina 206 may improve the efficiency, accuracy, or other attributes of a function that uses the retinal image. For example, tracking the position / orientation of the eye 205 based on the retinal image may be more efficient, more accurate, and may be used to track a wider range of eye orientations and / or pupil opening sizes given an image of a relatively large portion of the central portion of the retina 206. Obtaining an image of a relatively large portion of the central portion of the retina 206 may require aligning the camera with the location from which diffuse light is directed toward the eye 205.

[0038] Images obtained based on diffuse light that is not so aligned with the optical axis of the camera may not produce an adequate image of the retina. FIG. 3A to FIG. 3BThe portion of the eye / retina that is illuminated relative to the portion of the eye / retina captured by the camera is shown given that the four light sources are not adequately aligned with the optical axis of the camera. Figure 3A In the example, the pupil opening is relatively large and the eye is aligned toward the image tracking system (e.g., Figure 2A In this case, the illuminated eye / retina portions 310a to 310d substantially coincide with the captured eye / retina portion 305. Figure 3B In the example, the pupil opening is relatively small and the eye is not aligned toward the image tracking system (e.g., Figure 2B ). In this case, the illuminated eye / retina portion 310a to 310d does not substantially coincide with the captured eye / retina portion 305. This illustrates a significant disadvantage of using a retinal imaging system in which the illumination is not aligned both spatially and angularly with the optical axis of the camera.

[0039] In some direct retinal imaging applications, it is desirable or necessary to employ a camera-aligned (e.g., coaxially aligned) illumination system to ensure greater overlap of the illuminated area of ​​the retina with the captured field of view. This is particularly true for applications where small pupil openings are expected and where non-aligned illumination would provide little or no overlap with the camera field of view.

[0040] Implementations disclosed herein provide devices and techniques that enable retinal imaging in which illumination and camera capture are better aligned than existing systems and are therefore better suited to capture retinal images of illuminated portions of the retina. The devices and techniques disclosed herein can enable more efficient and accurate retinal imaging over a wider range of situations (e.g., for a wider range of pupil opening sizes and / or eye orientations). According to some aspects, improved alignment can improve accuracy with respect to determining eye position / orientation and / or autofocus depth / distance of the eye. The devices and techniques disclosed herein can provide retinal imaging on devices subject to size, power, and / or processing constraints. Implementations disclosed herein can be well suited for eye tracking applications on mobile and / or head mounted devices (HMDs).

[0041] Implementations that provide eye tracking may do so based on previously obtained information about the eye (e.g., such as a previous retinal image, or a representation of the retina generated based on a previous retinal image from a registration process). In some implementations, the representation of the retina provides a mapping that distinguishes retinal features so that later acquired images can be matched to the mapping. Based on such matching, the position / orientation of the retina and, therefore, the position / orientation of the eye as a whole can be determined. Similarly, since the eyepiece can be focused at different depths, which will cause the captured retinal image content to change (e.g., shrink or magnify), comparing the retinal image to a previously acquired retinal map (associated with a given autofocus level) can provide information about the current autofocus of the retina (i.e., at the time of the captured retinal image content).

[0042] Some implementations are additionally configured to reduce or illuminate specular reflections / flares off the cornea of ​​the eye that appear in retinal images. For example, the illumination emitted toward the eye can have a particular polarization, and the camera can utilize perpendicular polarization. Such cross-polarization can reduce or eliminate corneal reflections / flares that appear in the captured image.

[0043] FIG. 4A to FIG. 4B An exemplary eye tracking device 400 is shown. Eye tracking device 400 includes a housing 402 that at least partially encloses an image sensor 401, a camera lens 405 within an aperture, a polarizer 410, a light source 420, and a scattering optical device 430. Image sensor 401 can include any type of sensor capable of capturing an image based on receiving light, such as a CMOS sensor configured to convert charges from light-sensitive pixels into voltages at individual pixel sites, which are recorded as an image with pixel values ​​in rows and columns. Image sensor 401 can be configured to capture light of the same type as light emitted by light source 420 (e.g., IR light, light within a specific wavelength range, etc.). Polarizer 410 can be configured to be perpendicular to the illumination polarization.

[0044] The lens 405 may be configured to focus light onto the image sensor 401. The lens 405 has an optical axis and an aperture diameter 407 (twice the distance of the lens aperture radius).

[0045] Light source 420 emits directed light and / or polarized light toward scattering optics 430. For example, the light source may be a collimated polarized light emitting diode (LED). The scattering optics may be positioned to direct the received light toward eye 205. In this example, scattering optics 430 is a reflective diffuser at a 45 degree angle relative to the light source and at a 45 degree angle relative to the lens optical axis 406. Figure 4A and Figure 4BAs shown, some implementations provide a device in which a light source 420 provides collimated light from a side of an eye tracking device 400 toward a scattering optic 430 that redirects and diffuses the light from a position aligned with an image sensor 401 and / or lens 405 toward the retina of the eye 205. Such a scattering optic 430 may have properties that make it at least partially reflective and configured to produce diffuse light 440. In an alternative configuration, the scattering optic is an optical element with divergent optical power, e.g., it is not necessary to have each point spread light differently. Any type of light diffusion or spreading component may be used.

[0046] like Figure 4A and Figure 4B 4, the scattering optics 430 are aligned with the lens 405. In this example, the scattering optics 430 are coaxially aligned with the lens 405, i.e., the center of the scattering optics is positioned along the optical axis 406 of the lens 405. This positioning allows the scattering optics 430 to redirect light from the light source 420 into diffuse light 440 that is directed toward the eye 205. At least some of the diffuse light 440 is directed from a location that is less than the radius of the lens aperture (half the diameter 407) from the lens optical axis 406 and is directed toward the retina of the eye 205.

[0047] Image sensor 401 captures captured light that includes reflections of diffuse light 440 from the retina of eye 205. Such images of the retina and / or other eye parts may be used to determine and / or track position, orientation, autofocus, retinal properties, and / or other eye properties.

[0048] Figure 5A , Figure 5B , Figure 5C Additional exemplary eye tracking devices 500a through 500c are shown. Figure 5A , eye tracking device 500a includes housing 402 that at least partially encloses image sensor 401, camera lens 405 within aperture, polarizer 410, light source 520, and scattering optics 530. In this example, light source 420 is a divergent light source focused by focusing element 510 onto scattering optics 530, which can enable the use of relatively smaller scattering optics 530 (e.g., with respect to FIG. 4A to FIG. 4B The scattering optics 530 direct the diffuse light 440 toward the eye.

[0049] exist Figure 5B, eye tracking device 500b includes housing 402 that at least partially encloses image sensor 401, camera lens 405 within an aperture, polarizer 410, light source 420, and scattering optics having components 531a to 531b. In this example, diffuser component 531a of the scattering optics produces diffused light that is redirected by reflective component 531b as diffused light 440 directed toward the eye.

[0050] exist Figure 5C , eye tracking device 500c includes housing 402 that at least partially encloses image sensor 401, camera lens 405 within an aperture, polarizer 410, light source 521, and scattering optics 532. In this example, scattering optics 532 is a curved reflector having a shape / curvature that dictates the propagation of diffuse light, e.g., within a camera field of view.

[0051] Figure 6 An exemplary eye tracking device 600 is shown. The eye tracking device 600 includes a housing 402 that at least partially surrounds an image sensor 401, a camera lens 405 within an aperture, a polarizer 410, a waveguide 610, a light source 620, and a scattering optical device 630. Figure 7 As shown, light generated by a light source 620 (e.g., a collimated polarized LED) can be injected into the waveguide 610 via a coupling prism 725 or other diffractive optical device, and travel along at least a portion of the waveguide 610 within the waveguide 610, for example based on internal reflection, which can be total internal reflection. The scattering optical device 630 is one or more multi-directional output couplers partially on the aperture, which guides the internally reflected light out of the waveguide 610 as diffuse light 440, which is guided toward the eye 205, for example, via a diffractive optical element. The scattering optical device 630 may include several small output couplers with different properties. The scattering optical device 630 may include a transparent element that does not block the image sensor 401 from capturing image data. The scattering optical device 630 can spread the light out and maintain polarization, but also allow the light to reflect and travel to the image sensor 401.

[0052] Figure 8An exemplary eye tracking device 800 is shown. The eye tracking device 800 includes a housing 402 that at least partially encloses an image sensor 401, a camera lens 405 within an aperture, a polarizer 410, a waveguide 810, a light source 820, and a scattering optical device 830 along the front surface of the waveguide 810. Light generated by the light source 820 (e.g., a collimated polarized LED) can be injected into the waveguide and travel within the waveguide 810, for example based on internal reflection. The scattering optical device 830 is a multi-directional output coupler across the lens aperture that directs the internally reflected light out of the waveguide 810 as diffuse light 840, which is directed toward the eye, for example via a diffractive optical element. The scattering optical device 830 can include a transparent element that does not block the image sensor 401 from capturing image data. The scattering optical device 830 can spread the light out and maintain polarization, but also allow the light to reflect and travel to the image sensor 401. From the perspective of image sensor 401 , waveguide 810 may be at least partially transparent.

[0053] Fig. 9 An exemplary eye tracking device 900 is shown. The eye tracking device 900 includes a housing 402 that at least partially encloses an image sensor 401, a camera lens 405 within an aperture, a polarizer 410, a waveguide 910, a light source 920, and a scattering optical device 930 along a back surface of the waveguide 910. Light generated by the light source 920 (e.g., a collimated polarized LED) can be injected into the waveguide 910 and travel within the waveguide 910, for example, based on internal reflection. The scattering optical device 930 can propagate the light out and maintain polarization, but also allow the light to reflect and travel to the image sensor 401. From the perspective of the image sensor 401, the waveguide 910 can be transparent.

[0054] The scattering optical device 930 may include a dense (or sparse) array of very small reflectors on the waveguide 910 that direct light toward the eye at a wide span of angles. The scattering optical device 930 may include a plurality of relatively small but densely positioned scattering elements such that whenever light strikes one of the scattering elements, the light is scattered toward the eye. The waveguide 910 may include such scattering elements and thus have a reflection that is less than total internal reflection. For example, by etching a small depression (defect) in the glass or other material forming the waveguide 910, the scattering elements may be embedded in the surface of the waveguide. The scattering elements may be embedded in the waveguide 910 by injecting small particles in the waveguide 910 (e.g., near the surface of the waveguide). The number and / or positioning of such scattering elements may depend on the retinal imaging application and may be selected to provide a desired or sufficient amount of illumination for a particular application. A sparse set of scattering elements may produce sufficient retinal illumination for some applications. Similarly, for some applications, the scattering elements do not need to cover the entire surface of the waveguide 910.

[0055] Fig.10 An exemplary eye tracking device 1000 is shown. The eye tracking device 1000 includes a housing 402 that at least partially encloses an image sensor 401, a camera lens 405 within an aperture, a polarizer 410, a waveguide 1010, a light source 1020, and scattering optics that include a scattering element 1031 along a front surface of the waveguide 1010 and a mirror coating 1032 along a back surface of the waveguide. Light generated by the light source 1020 (e.g., a collimated polarized LED) can be injected into the waveguide 1010 and travel within the waveguide 1010, for example, based on internal reflection. The scattering optics can propagate the light out and maintain polarization, but also allow the light to reflect and travel to the image sensor 401. From the perspective of the image sensor 401, the waveguide 1010 can be transparent.

[0056] The scattering optics may include scattering elements 1031, which are dense (or sparse) arrays of very small reflectors on the waveguide 1010 that direct light toward the eye or toward a partial rearview mirror at a wide span of angles. The scattering elements 1031 may include a plurality of relatively small but densely positioned scattering elements such that each time light strikes one of the scattering elements, the light is scattered toward the eye. The waveguide 1010 may include such scattering elements and, therefore, have a reflection that is less than total internal reflection. The number and / or positioning of such scattering elements 1031 may depend on the retinal imaging application and may be selected to provide a desired or sufficient amount of illumination for a particular application. A sparse set of scattering elements may produce sufficient retinal illumination for some applications. Similarly, for some applications, the scattering elements need not cover the entire surface of the waveguide 1010.

[0057] The mirror coating 1032 on the waveguide 1010 can also direct light out of the waveguide 1010 and toward the eye. In some implementations, the scattering element 1031 scatters light back toward the mirror coating 1032, which reflects the scattered light as diffuse light, at least some of which is directed toward the eye. The mirror coating 1032 can be positioned near the optical axis of the lens 405 so that the diffuse light directed toward the eye is closely aligned with the camera element. The mirror element 1032 can be polarization dependent and can be included or not included.

[0058] Fig.11 An exemplary eye tracking device 1100 is shown. The eye tracking device 1100 includes a housing 402, an image sensor 401, a camera lens 405 within an aperture, a polarizer 410, a scattering optic 1135, and a light source 1120. Light generated by the light source 1120 (e.g., a collimated polarized LED) is directed toward the scattering optic 1130, which in one example is a mirror-coated Fresnel lens. The scattering optic 1130 reflects the light as diffuse light 440 and maintains polarization, but also allows the light reflection to travel to the image sensor 401. The scattering optic 1130 can be implemented by coating a portion (e.g., a central region) of a lens (e.g., lens 405 or polarizer 410) with a mirror coating and / or etching the surface of such a lens. The light source 1120 can provide light from inside the housing 402 or from outside the housing 402 of the eye tracking device 1100.

[0059] FIG. 12A to FIG. 12B Shows Fig.11 An exemplary configuration of a scattering optical device 1130. Fig. 12A A configuration is shown in which a SiO2 layer 1205 is adjacent to a polarizer 1215, wherein the SiO2 layer 1205 has an anti-reflective coating 1225 for the side portions and a mirror coating 1210 for the center portion. Similarly, Fig. 12B A configuration is shown in which a SiO2 layer 1205 is adjacent to a polarizer 1215, wherein the SiO2 layer 1205 has an anti-reflective coating 1225 for the side portions and a mirror coating 1210 for the center portion. The geometry of the center portion with the mirror coating 1210 can have various irregular / non-planar configurations that produce diffuse light reflection of light from the light source 1120 toward the eye 205.

[0060] Fig.13An exemplary eye tracking device 1300 using outward-facing illumination is shown. Eye tracking device 1300 includes a housing 402 that at least partially surrounds an image sensor 401, a camera lens 405 within an aperture, a polarizer 410, and a light source 1320. The light source may include one or more LEDs, VCSEL arrays, etc., may be configured to produce polarized light, and / or may be attached in a manner that minimizes blocking of return light reflections. The light produced by light source 1320 is diffuse light that is directed toward eye 205.

[0061] In some implementations, a transparent attachment member (eg, a fixing wire) is used to secure the light source 1320 (eg, on the lens 405 or the polarizer 410). Fig.14 Shows Fig.13 Attachment of light source 1320 in exemplary eye tracking device 1300. In this example, light source 1320 is secured in place using transparent wires 1420a to 1420c. Mechanical retaining structures (e.g., transparent wires 1420a to 1420c) may additionally be used to carry controls and current supply for light source 1320. The light source may additionally or alternatively be attached to an optical surface using an adhesive.

[0062] The size of the light source 1320 can be set to minimize the amount of blockage, for example, blocking less than 40%, 30%, 20%, 10%, 5% of the aperture of the camera lens 405. In some implementations, the light source 1320 has a circular cross-section (e.g., Fig.14 In other implementations, the light source 1320 has a linear, rectangular, or other shape, such as a strip including a plurality of LEDs arranged in a linear manner.

[0063] In some implementations, an illumination source (eg, a sparse illumination panel such as a micro-LED array) is co-aligned and in front of an image sensor so that the image sensor can sense by illumination. Both the illumination source and the image sensor can use the same lens.

[0064] Fig.15An exemplary eye tracking device 1500 is shown. The eye tracking device 1500 includes a housing 402 that at least partially encloses an image sensor 401, a camera lens 405 within an aperture, optics 1530, and a light source 1520. Light generated by the light source 1520 (e.g., a collimated polarized LED) is directed toward the optics 1530, which reflects the light toward the eye 205 as diffuse light and maintains polarization. The optics 1530 can be a miniaturized (e.g., smaller than the housing 402, smaller than the lens, etc.) polarizing beam splitter (PBS) plate that is in front of the image sensor 401 but behind the lens 405, i.e., packaged within the camera module. In alternative implementations, the optics 1530 (such as a PBS plate) is positioned in front of the lens 405 and / or is not packaged within the camera module.

[0065] Fig.16 16 is a flow chart illustrating an exemplary method 1600 for tracking eye characteristics. In some implementations, a device (e.g., Figure 1 The method 1600 is performed on a device 120 of the present invention. In some implementations, the method 1600 is performed on a mobile device, a desktop computer, a laptop computer, an HMD, or a server device. In some implementations, the method 1600 is performed by a processing logic component (including hardware, firmware, software, or a combination thereof). In some implementations, the method 1600 is performed on a processor that executes code stored in a non-transitory computer-readable medium (e.g., a memory).

[0066] At block 1602, method 1600 generates diffuse light directed toward a retina of an eye, and at block 1604, method 1600 generates an image of the retina using a camera including a lens having a lens optical axis and a lens aperture radius. The image is generated by capturing reflections of the diffuse light off the retina of the eye, wherein at least some of the diffuse light is directed from a location that is less than a distance from the lens optical axis than the lens aperture radius. At block 1606, method 1600 tracks the eye based on the image.

[0067] In some implementations, diffuse light is directed by a diffuse optical device or light source, wherein the diffuse optical device or light source is entirely within the lens aperture radius distance from the lens optical axis. Such optional positioning is shown in FIG. FIG. 5A to FIG. 5C , Figure 6 , Figure 7 , Figure 8 , Fig.11 , Fig.13 , Fig.14 and Fig.15 An exemplary device is shown in FIG.

[0068] In some implementations, method 1600 is performed at a device having a camera, the camera having a viewing angle, and the diffuse light is scattered across the entire viewing angle of the camera. In some implementations, method 1600 directs diffuse light from a sufficiently diffuse location relative to the eye and the camera such that at least some of the diffuse light will be directed toward and illuminate the retina regardless of the rotational orientation of the eye, e.g., in the full range of potential eye rotational orientations, and reflections of such light will be captured by the camera.

[0069] In some implementations, method 1600 is performed at a device that includes a waveguide that guides diffuse light, wherein the light source directs the light source into the waveguide. Examples of such configurations are described in Figure 6 , Figure 7 , Figure 8 , Fig. 9 and Fig.10 In some embodiments, the waveguide includes a scattering optical device, and the scattering optical device includes a diffuser plate, the diffuser plate includes a plurality of scattering elements, such as Fig. 9 Such multiple scattering elements may be etched into the surface of the waveguide, or may be particles injected into the waveguide. In some implementations, the waveguide includes an embedded diffuser and a partial backside coating, such as Fig.10 shown.

[0070] In some implementations, the waveguide includes a multi-directional output coupler, such as Figures 8 to 10 The multi-directional output coupler can be positioned over the entire aperture of the lens (e.g. Fig.10 ), or positioned on a partial aperture (as shown Figures 8 to 9 shown).

[0071] In some implementations, light is directed toward a scattering optic by a collimated light emitting diode (LED), such as FIG. 4A to FIG. 4B , FIG. 5A to FIG. 5C , Fig.11 and Fig.15 shown.

[0072] In method 1600, the scattering optics may be a reflective diffuser, a plurality of scattering elements, or a mirror coating.

[0073] In some implementations, method 1600 uses a relatively small beam splitter within a camera module. For example, a light source and a scattering optic are within a chamber of a camera, wherein the light source includes a light emitting diode (LED), the scattering optic includes a diffuser and a polarizing beam splitter, and wherein the LED directs directed light through the diffuser, the diffuser scatters the directed light, and the polarizing beam splitter reflects the scattered light in a diffuse direction. Fig.15 Such a configuration is shown in .

[0074] In some implementations, the diffuse light directed toward the retina and the light captured by the camera have perpendicular polarizations. For example, the diffuse light can have a first polarization that is perpendicular to a second polarization of the captured light.

[0075] In some implementations, method 1600 is performed by a head mounted device (HMD). The camera and illumination components of an eye tracking system on such an HMD may be located at fixed locations on the HMD and may therefore be used to track the position and / or orientation of the eye relative to the HMD over time. The camera, light source, and scattering optics of the eye tracking module may be housed within a housing attached to a frame portion of the HMD. In some cases, the eye tracking system provides real-time, live eye tracking when the user uses the HMD to view the surrounding physical environment and / or content displayed on the HMD (e.g., as an extended reality (XR) environment).

[0076] In some implementations, the light is IR light. In some implementations, the light source is an LED. Alternatively, another type of light source may be used that is sufficient to provide a retinal-based image when light from the light source is projected onto the eye.

[0077] Method 1600 can generate an image of a portion of the retina from an image sensor that corresponds to multiple reflections of light reflected and / or scattered from the retina of the eye. For example, the sensor can be an IR image sensor / detector. Method 1600 can obtain a representation of the eye (e.g., a registration image / map). The representation can represent at least some of the portions of the retina. For example, the representation can be a map of the retina generated by having the user adjust to a particular depth (e.g., infinity, 30 cm, 1 m, etc.) and scan through a gaze angle space representing the entire desired field of view (e.g., registration of a registration process). The captured images from such a registration phase can then be stitched together to form a map of the retina.

[0078] In some implementations, acquiring a representation of the eye is based on generating an enrollment image of the retina of the eye to be used with the eye tracking system (e.g., to register a new user before using the eye tracking system). In an exemplary implementation, the representation of the eye includes a map of at least some of the portions of the retina. In some implementations, generating the map of at least some of the portions of the retina includes acquiring an enrollment image of the user's eye and generating the map of at least some of the portions of the retina based on combining (stitching) at least a portion of two or more of the enrollment images of the eye. In some implementations, acquiring the enrollment image is performed while the user (i) adjusts the eye to a particular enrollment depth (e.g., infinity, 30 cm, 1 m, etc.) and (ii) scans through a gaze angle space representing a defined field of view. For example, before a user can access / use a particular program on a device, the system performs a user enrollment process that includes capturing an enrollment image of the retina that can be used during use of the program for eye tracking (e.g., when a new user first uses the HMD). Some implementations do not require building a map of the retina. For example, such an implementation may utilize enrollment images that are used as a database for a process (e.g., an algorithm, a machine learning model, etc.) that compares each new image to the database and determines the gaze angle accordingly.

[0079] Method 1600 may track eye characteristics based on a comparison of an image of a portion of the retina to a representation of the eye. In some implementations, tracking eye characteristics determines a position or orientation of the eye within a 3D coordinate system, for example relative to a device and / or a physical environment. In some implementations, tracking eye characteristics is based on a user adjustment distance determined via zoom and blur. Several methods and / or combinations of methods may be utilized to track eye characteristics based on a comparison of an image of a portion of the retina to a representation of the eye. In an exemplary implementation, tracking eye characteristics based on a comparison of an image of a portion of the retina to a representation of the eye includes estimating a degree of defocus of a feature. In some implementations, estimating the degree of defocus of a feature is based on focused pixels (e.g., an imaging technique used to determine focus / blur).

[0080] Fig.171 is a block diagram of an example device 1700. Device 1700 illustrates an exemplary device configuration for device 120. Although some specific features are shown, those skilled in the art will appreciate from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the specific implementations disclosed herein. To this end, as a non-limiting example, in some embodiments, the device 10 includes one or more processing units 1702 (e.g., microprocessors, ASICs, FPGAs, GPUs, CPUs, processing cores, etc.), one or more input / output (I / O) devices and sensors 1706, one or more communication interfaces 1708 (e.g., USB, FIREWIRE, THUNDERBOLT, IEEE 802.3x, IEEE 802.11x, IEEE 802.16x, GSM, CDMA, TDMA, GPS, IR, Bluetooth, ZIGBEE, SPI, I2C, and / or similar types of interfaces), one or more programming (e.g., I / O) interfaces 1710, one or more displays 1712, one or more internal-facing and / or external-facing image sensor systems 1714, memory 1720, and one or more communication buses 1704 for interconnecting these and various other components.

[0081] In some implementations, the one or more communication buses 1704 include circuits that interconnect system components and control communications between system components. In some implementations, the one or more I / O devices and sensors 1706 include at least one of the following: an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., a blood pressure monitor, a heart rate monitor, a blood oxygen sensor, a blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptic engine, or one or more depth sensors (e.g., structured light, time of flight, etc.), etc.

[0082] In some implementations, one or more displays 1712 are configured to present a view of a physical or graphical environment to a user. In some implementations, one or more displays 1712 correspond to holographic, digital light processing (DLP), liquid crystal display (LCD), liquid crystal on silicon (LCoS), organic light emitting field effect transistor (OLET), organic light emitting diode (OLED), surface conduction electron emission display (SED), field emission display (FED), quantum dot light emitting diode (QD-LED), microelectromechanical system (MEMS), and / or similar display types. In some implementations, one or more displays 1712 correspond to diffraction, reflection, polarization, holographic, etc. waveguide displays. In one example, device 10 includes a single display. In another example, device 1700 includes a display for each eye of the user.

[0083] In some implementations, the one or more image sensor systems 1714 are configured to acquire image data corresponding to at least a portion of the physical environment. For example, the one or more image sensor systems 1714 include one or more RGB cameras (e.g., with complementary metal oxide semiconductor (CMOS) image sensors or charge coupled device (CCD) image sensors), monochrome cameras, IR cameras, depth cameras, event-based cameras, etc. In various implementations, the one or more image sensor systems 1714 also include an illumination source that emits light. In various implementations, the one or more image sensor systems 1714 also include an on-camera image signal processor (ISP) that is configured to perform multiple processing operations on the image data.

[0084] Memory 1720 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid-state memory devices. In some specific implementations, memory 1720 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices or other non-volatile solid-state storage devices. Memory 1720 optionally includes one or more storage devices remotely located from one or more processing units 1702. Memory 1720 includes non-transitory computer-readable storage media.

[0085] In some implementations, the memory 1720 or a non-transitory computer-readable storage medium of the memory 1720 stores an optional operating system 1730 and one or more instruction sets 1740. The operating system 1730 includes processes for handling various basic system services and for performing hardware-related tasks. In some implementations, the instruction set 1740 includes executable software defined by binary information stored in the form of electrical charge. In some implementations, the instruction set 1740 is software that can be executed by one or more processing units 1702 to implement one or more of the techniques described herein.

[0086] The instruction set 1740 includes a tracking instruction set 1742, which can be embodied as a single executable software or multiple executable software. In some specific implementations, the tracking instruction set 1742 can be executed by the processing unit 702 to track eye characteristics as described herein. This can use one or more of the techniques discussed herein to determine eye position, orientation, autofocus, etc. based on comparison of one or more captured images of the retina with a representation of the eye, or use other possible appropriate methods. For these purposes, in various specific implementations, the instructions include instructions and / or logic for the instructions as well as heuristics and metadata for the heuristics.

[0087] Although the instruction set 1740 is shown as residing on a single device, it should be understood that in other implementations, any combination of elements may be located in separate computing devices. Fig.17 It is intended more as a functional description of various features present in a particular implementation that are different from the structural schematics of the implementation described herein. As one of ordinary skill in the art will recognize, items shown separately can be combined, and some items can be separated. The actual number of instruction sets and how the features are distributed among them will vary depending on the specific implementation and may depend in part on the specific combination of hardware, software, and / or firmware selected for a particular implementation.

[0088] It should be understood that the specific implementations described above are cited by way of example, and the present disclosure is not limited to what has been particularly shown and described above. On the contrary, the scope includes both combinations and sub-combinations of the various features described above, as well as variations and modifications of the various features that would occur to a person skilled in the art upon reading the foregoing description and that are not disclosed in the prior art.

[0089] As described above, one aspect of the present technology is to collect and use physiological data to improve the user's electronic device experience in interacting with electronic content. The present disclosure contemplates that, in some cases, the collected data may include personal information data that uniquely identifies a particular person or can be used to identify the interests, characteristics, or tendencies of a particular person. Such personal information data may include physiological data, demographic data, location-based data, phone numbers, email addresses, home addresses, device characteristics of personal devices, or any other personal information.

[0090] The present disclosure recognizes that the use of such personal information data in the present invention technology can be used to benefit users. For example, personal information data can be used to improve the interaction and control capabilities of electronic devices. Therefore, the use of such personal information data enables planned control of electronic devices. In addition, the present disclosure also anticipates other uses of personal information data that benefit users.

[0091] The disclosure also contemplates that entities responsible for the collection, analysis, disclosure, transmission, storage or other purposes of such personal information and / or physiological data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to the use of privacy policies and practices that are recognized as meeting or exceeding the industry or government requirements for maintaining the privacy and security of personal information data. For example, personal information from users should be collected for the legal and reasonable purposes of entities, and not shared or sold outside these legal purposes. In addition, such collection should only be carried out after the user's informed consent. In addition, such entities should take any necessary steps to safeguard and protect access to such personal information data, and ensure that other people who can access personal information data comply with their privacy policies and procedures. In addition, such entities can subject themselves to third-party assessments to prove that they comply with widely accepted privacy policies and practices.

[0092] Regardless of the foregoing, the present disclosure also contemplates implementations in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates that hardware elements or software elements may be provided to prevent or block access to such personal information data. For example, with respect to a content delivery service customized for a user, the technology of the present invention may be configured to allow a user to choose to "opt in" or "opt out" of participating in the collection of personal information data during registration for the service. In another example, a user may choose not to provide personal information data for a target content delivery service. In yet another example, a user may choose not to provide personal information but allow anonymous information to be transmitted for use in improving the functionality of the device.

[0093] Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, various embodiments of the present technology will not fail to function properly due to the lack of all or a portion of such personal information data. For example, content may be selected and delivered to a user by inferring preferences or settings based on non-personal information data or an absolute minimum amount of personal information, such as content requested by a device associated with the user, other non-personal information available to a content delivery service, or publicly available information.

[0094] In some embodiments, data is stored using a public / private key system that only allows the owner of the data to decrypt the stored data. In some other implementations, data can be stored anonymously (e.g., without identification and / or personal information about the user, such as legal name, username, time and location data, etc.). In this way, other users, hackers, or third parties cannot determine the identity of the user associated with the stored data. In some implementations, a user can access his or her stored data from a user device that is different from the user device used to upload the stored data. In these cases, the user may need to provide login credentials to access the data they store.

[0095] Numerous specific details are set forth herein to provide a comprehensive understanding of the claimed subject matter. However, those skilled in the art will appreciate that the claimed subject matter may be practiced without these specific details. In other instances, methods, devices, or systems known to those of ordinary skill are not described in detail so as not to obscure the claimed subject matter.

[0096] Unless otherwise specifically noted, it should be understood that throughout the specification, discussions utilizing terms such as "process," "compute," "calculate," "determine," and "identify" refer to the actions or processes of a computing device, such as one or more computers or similar electronic computing devices, that manipulate or transform data represented as physical electronic or magnetic quantities within a memory, register, or other information storage device, transmission device, or display device of a computing platform.

[0097] The one or more systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device may include any suitable arrangement of components that provide results conditioned on one or more inputs. Suitable computing devices include multi-purpose microprocessor-based computer systems that access stored software that programs or configures the computing system from a general-purpose computing device to a dedicated computing device that implements one or more specific implementations of the subject matter of the present invention. Any suitable programming, scripting, or other type of language or combination of languages ​​may be used to implement the teachings contained herein in software for programming or configuring a computing device.

[0098] The specific implementation of the method disclosed herein can be performed in the operation of such a computing device. The order of the boxes presented in the above examples can be changed, for example, the boxes can be reordered, combined or divided into sub-boxes. Some boxes or processes can be executed in parallel.

[0099] The use of "suitable for" or "configured to" herein is meant to be open and inclusive language that does not exclude devices that are suitable for or configured to perform additional tasks or steps. In addition, the use of "based on" is meant to be open and inclusive, as a process, step, calculation, or other action "based on" one or more of the stated conditions or values ​​may in practice be based on additional conditions or values ​​beyond those stated. The headings, lists, and numbers included herein are for ease of explanation only and are not intended to be limiting.

[0100] It will also be understood that, although the terms "first", "second", etc. may be used to describe various objects in this article, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first node may be referred to as a second node, and similarly, a second node may be referred to as a first node, which changes the meaning of the description, as long as all occurrences of the "first node" are consistently renamed and all occurrences of the "second node" are consistently renamed. Both the first node and the second node are nodes, but they are not the same node.

[0101] The terms used herein are only for describing specific implementations and are not intended to limit the claims. As used in the description of this specific implementation and the appended claims, the singular forms of "one", "a kind of" and "the" are intended to also cover the plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "or" used herein refers to and covers any and all possible combinations of one or more items in the associated listed items. It will also be understood that the terms "including" or "comprising" when used in this specification specify the presence of stated features, integers, steps, operations, objects or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, objects, parts or their groupings.

[0102] As used herein, the term “if” may be interpreted to mean “when the antecedent is true” or “when the antecedent is true” or “in response to determining” or “upon determining” or “in response to detecting” that the antecedent is true, depending on the context. Similarly, the phrase “if it is determined that [the antecedent is true]” or “if [the antecedent is true]” or “when [the antecedent is true]” is interpreted to mean “upon determining that the antecedent is true” or “in response to determining” or “upon determining” that the antecedent is true or “when detecting that the antecedent is true” or “in response to detecting” that the antecedent is true, depending on the context.

[0103] The foregoing description and summary of the present invention should be understood to be illustrative and exemplary in every aspect, rather than restrictive, and the scope of the present invention disclosed herein is determined not only by the detailed description of the exemplary specific implementations, but by the full breadth allowed by the patent law. It should be understood that the specific implementations shown and described herein are only illustrative of the principles of the present invention, and that various modifications can be implemented by those skilled in the art without departing from the scope and essence of the present invention.

Claims

1. A device comprising: a camera comprising a chamber having an aperture, the aperture being equipped with a lens through which captured light is received to form an image, the image being projected onto a surface for recording or conversion into electrical impulses, the camera lens having a lens optical axis and a lens aperture radius distance; a light source configured to generate light directed toward the scattering optics; the scattering optics being positioned to produce diffuse light by scattering the light produced by the light source, wherein at least some of the diffuse light is directed from a location less than a radius of the lens aperture from the lens optical axis and directed toward a retina of an eye, and wherein the captured light includes reflections of the diffuse light from the retina; and One or more processors configured to track the eye based on the image.

2. The apparatus of claim 1, wherein the entirety of the scattering optical device is within the lens aperture radius distance from the lens optical axis.

3. The apparatus according to claim 1, wherein: The camera has a viewing angle; and The scattering optics scatter light across the viewing angle of the camera.

4. The apparatus of claim 1 , wherein the scattering optical device is configured to direct the diffuse light from a fixed position relative to the eye toward a retina of the eye, wherein the scattering optical device is configured to illuminate at least a portion of the retina regardless of a rotational orientation of the eye.

5. The apparatus of claim 1, further comprising a waveguide, wherein the light source directs the light source into the waveguide.

6. The apparatus of claim 5, wherein the waveguide comprises the scattering optics, and the scattering optics comprises a diffuser plate comprising a plurality of scattering elements.

7. The apparatus of claim 6, wherein the plurality of scattering elements are etched into the surface of the waveguide.

8. The apparatus of claim 6, wherein the plurality of scattering elements are particles injected into the waveguide.

9. The apparatus of claim 5, wherein the waveguide comprises the scattering optics, and the scattering optics comprises an embedded diffuser and a partial backside coating.

10. The apparatus of claim 5, wherein the waveguide comprises the scattering optics, and the scattering optics comprises a multi-directional output coupler.

11. The apparatus of claim 10, wherein the multi-directional output coupler is positioned over the entire aperture.

12. The apparatus of claim 10, wherein the multi-directional output coupler is positioned over a portion of the aperture.

13. The apparatus of claim 1, wherein the light source comprises a collimated light emitting diode (LED).

14. The apparatus of claim 13, wherein the scattering optics comprises a reflective diffuser.

15. The apparatus of claim 13, wherein the scattering optics comprises a mirror coating on the lens.

16. The apparatus of claim 1, wherein the light source and the scattering optics are within the chamber of the camera, wherein: The light source comprises a light emitting diode (LED); The scattering optics include a diffuser and a polarizing beam splitter, Wherein the LED directs the directed light through the diffuser, the diffuser scatters the directed light, and the polarizing beam splitter reflects the scattered light in a diffuse direction.

17. The device of claim 1, wherein the scattered light has a first polarization that is perpendicular to a second polarization of the trapped light.

18. The device of claim 1, wherein the camera, the light source, and the scattering optics are housed within a housing that is attached to a frame portion of a head mounted device (HMD).

19. A device comprising: a camera comprising a chamber having an aperture, the aperture being equipped with a lens through which captured light is received to form an image, the image being projected onto a surface for recording or conversion into electrical impulses, the camera lens having a lens optical axis and a lens aperture radius distance; a light source configured to generate diffuse light, wherein at least some of the diffuse light is generated from a location less than a radius of the lens aperture from an optical axis of the lens and is directed toward a retina of an eye, and wherein the captured light includes reflections of the diffuse light from the retina; and One or more processors configured to track the eye based on the image.

20. The apparatus of claim 19, wherein the light source is secured at a location in the center of the lens.

21. The apparatus of claim 19, wherein the light source comprises a transparent wire.

22. The apparatus of claim 19, wherein the light source blocks less than 30% of the aperture.

23. The apparatus of claim 19, wherein the light source comprises a light emitting diode (LED) or a light emitting diode (VCSEL) array.

24. The apparatus of claim 21, wherein the light source is polarized.

25. A method comprising: At an electronic device having a processor: generating diffuse light that is directed toward a retina of the eye; generating an image of the retina using a camera including a lens, the lens having a lens optical axis and a lens aperture radius distance, the image being generated by capturing reflections of the diffuse light from the retina of the eye, wherein at least some of the diffuse light is directed from a location less than the lens aperture radius distance from the lens optical axis; as well as The eye is tracked based on the image.

Images