Efficient eye imaging system for near-eye displays with integrated photodiodes

Abstract

Various implementations disclosed herein capture sensor data regarding portions of an eye (e.g., the cornea, iris, or retina, etc.) with a near-eye device (e.g., an HMD) using an array of photodiodes embedded within the device's display. The device includes optics configured to focus light from the display on the retina. The device includes a light source configured to illuminate a portion of the eye to be imaged with light having a wavelength (e.g., IR light) that does not interfere with the visible light from the display (e.g., non-IR light). Some implementations, provide outer-eye (e.g., corneal) imaging by accounting for the device's optics having a retinal focus. This may involve, as examples, using wavelength-specific components (e.g., a wavelength specific aperture stop / pinhole, an array of aperture stops / pinholes; a second lens, a meta-surface (for back-and-forth reflections / refractions), or other features.

Description

Attorney Docket No. 097425-01465(P63841WOl)EFFICIENT EYE IMAGING SYSTEM FOR NEAR-EYE DISPLAYS WITH INTEGRATED PHOTODIODESTECHNICAL FIELD

[0001] The present disclosure generally relates to electronic devices, and in particular, to systems, methods, and devices for determining eye characteristics of users of near-eye electronic devices such as head-mounted devices (HMDs).BACKGROUND

[0002] Various existing techniques are used on near-eye devices (e.g., HMDs) to determine eye characteristics. For example, some devices use eye-imaging hardware, including light sources to produce patterns of light (e.g., glints) and / or sensors that capture light reflected off of portions of the eye, to determine eye characteristics. The positioning of displays and see-through portions in many such near-eye devices has generally limited the positioning of the eye-imaging hardware (e.g., the light sources and / or sensors are generally positioned on the peripheral side regions around the display(s) of an HMD, on the sides of the lenses of augmented reality (AR) glasses, etc. Such positioning may produce inefficiencies and inaccuracies, require more power, or have other negative consequences.SUMMARY

[0003] Various implementations disclosed herein include devices, systems, and methods that capture sensor data regarding portions of a user’s eye (e.g., the cornea, iris, or retina, etc.) during use of a near-eye device (e.g., an HMD), using an array of photodiodes embedded within the device’s display. The device includes optics configured to focus light from the display on the retina of the user. The device includes a light source configured to illuminate a portion of the eye to be imaged with light having a wavelength (e.g., IR light) that does not interfere with the visible light from the display (e.g., non-IR light). Some implementations provide outer-eye (e.g., corneal, iris, pupil, etc.) imaging by accounting for the device’s optics having a retinal focus. This mayAttomey Docket No. 097425-01465(P63841WOl) involve, as examples, using wavelength-specific components (e.g., a wavelength specific aperture stop / pinhole, an array of such aperture stops / pinholes; a second lens, a metasurface (e.g., for back-and-forth reflections / refractions), or other features. Some implementations provide outer-eye imaging by accounting for the optic’s retinal focus computationally (e.g., using a procedural algorithm or machine learning model).

[0004] In some implementations, an electronic device has a processor (e.g., one or more processors) that executes instructions stored in a non-transitory computer-readable medium to perform a method. The method performs one or more steps or processes. In some implementations, the method is performed at an HMD having a processor, a lens, a display, an eye-illumination light source, and a display with embedded photodiodes. The method involves displaying content by producing light via the display. The light may be light within a first wavelength range (e.g., wavelengths in the visible range). The method involves producing eye-illumination light via the eye-illumination light source to illuminate a portion of the eye (e.g., one or more of the retina, cornea, iris, etc.). The eye-illumination light may be light within a second wavelength range (e.g., wavelengths in the IR range) substantially outside of the first wavelength range. The second wavelength may be selected such that the eye-illumination light does not interfere with the nominal use of the device, e.g., it does not interfere with the user’s viewing content displayed by the display. The method involves captunng / detecting reflections of the eyeillumination light via the embedded photodiodes of the display. The eye-illumination light and the light produced via the display may pass through an element that affects the eye-illumination light more than the light produced via the display. The method further involves determining an eye characteristic of the eye based on the sensor data, (e.g., position, size, shape of the eye, gaze direction, etc., for eye- tracking, user authentication, eye health monitoring, etc.)

[0005] In accordance with some implementations, a non-transitory computer readable storage medium has stored therein instructions that are computer-executable to perform or cause performance of any of the methods described herein. In accordance with some implementations, a device includes one or more processors, a non-transitory memory, and one or more programs; the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors and the one orAttorney Docket No. 097425-01465(P63841WOl) more programs include instructions for performing or causing performance of any of the methods described herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] So that the present disclosure can be understood by those of ordinary skill in the art, a more detailed description may be had by reference to aspects of some illustrative implementations, some of which are shown in the accompanying drawings.

[0007] Figure 1 illustrates an exemplary head-mounted device (HMD) with photodiodes embedded within its display in accordance with some implementations.

[0008] Figure 2 illustrates exemplary HMD components including photodiodes embedded within a display for outer-eye imaging, in accordance with some implementations.

[0009] Figures 3A-B illustrate cross-sectional views of exemplary aperture configurations in accordance with some implementations.

[0010] Figure 4 illustrates exemplary HMD components including photodiodes embedded within a display for outer-eye imaging, in accordance with some implementations.

[0011] Figure 5 illustrates exemplary HMD components including photodiodes embedded within a display for outer-eye imaging, in accordance with some implementations.

[0012] Figure 6 illustrates exemplary HMD components including photodiodes embedded within a display for outer-eye imaging, in accordance with some implementations.

[0013] Figure 7 illustrates exemplary HMD components including photodiodes embedded within a display for outer-eye imaging, in accordance with some implementations.

[0014] Figure 8 illustrates exemplary HMD components including photodiodes embedded within a display and photometric stereo processing components, in accordance with some implementations.Attorney Docket No. 097425-01465(P63841WOl)

[0015] Figure 9 is a flowchart representation of a method for capture sensor data regarding portions of the eye (e.g., the cornea, iris, or retina, etc.) in a near-eye device (e.g., an HMD) using an array of photodiodes embedded within the device’s display, in accordance with some implementations.

[0016] Figure 10 illustrates device components of an exemplary device in accordance with some implementations.

[0017] Figure 11 illustrates a front view of a portion of a display in which both photodiodes and eye-illumination light sources are embedded amongst the pixels of the display at regularly-spaced locations in accordance with some implementations.

[0018] In accordance with common practice the various features illustrated in the drawings may not be drawn to scale. Accordingly, the dimensions of the various features may be arbitrarily expanded or reduced for clarity. In addition, some of the drawings may not depict all of the components of a given system, method or device. Finally, like reference numerals may be used to denote like features throughout the specification and figures.DESCRIPTION

[0019] Numerous details are described in order to provide a thorough understanding of the example implementations shown in the drawings. However, the drawings merely show some example aspects of the present disclosure and are therefore not to be considered limiting. Those of ordinary skill in the art will appreciate that other effective aspects or variants do not include all of the specific details described herein. Moreover, well-known systems, methods, components, devices and circuits have not been described in exhaustive detail so as not to obscure more pertinent aspects of the example implementations described herein.

[0020] Figure 1 illustrates an exemplary HMD 105 with photodiodes 125 embedded within its display 110. The HMD 105 includes a housing 150 (or enclosure) that houses various components of the HMD 105. The housing 150 includes (or is coupled to) an eye pad (not shown) disposed at a proximal (to the user 102) end of the housing 150. In various implementations, the eye pad is a plastic or rubber piece that comfortably andAttorney Docket No. 097425-01465(P63841WOl) snugly keeps the HMD 105 in the proper position on the face of the user 102 (e.g., surrounding the eye of the user 102).

[0021] The housing 150 houses a display 110 that displays an image, emitting light towards or onto the eye of the user 102. In various implementations, the display 110 emits the light through an eyepiece having one or more optical elements 115 (e.g., lenses) that refracts the light emitted by the display 110, making the display appear to the user 102 to be at a virtual distance farther than the actual distance from the eye to the display 110. For the user 102 to be able to focus on the display 110, in various implementations, the virtual distance is at least greater than a minimum focal distance of the eye (e.g., 7 cm). Further, in order to provide a better user experience, in various implementations, the virtual distance is greater than 1 meter. The optical elements 115 is generally configured to focus light produced by the display (e.g., displayed content) on the retina of the user’s eye.

[0022] The housing 150 also houses an eye characteristic determining system including one or more light sources 120, photodiodes 125, and a controller 130. The one or more light sources 120 emit light towards or onto the eye of the user 102 that reflects (e.g., as a circle other pattern of glints) reflections of which can be detected by the photodiodes 125. Based on the reflections (e.g., the light pattern), the controller 130 determines an eye characteristic of the user 102. For example, the controller 130 may determine a gaze direction and / or a blinking state (eyes open or eyes closed) of the user 102. As another example, the controller 130 may determine a pupil center, a pupil size, or a point of regard with respect to the pupil of the eye of the user 102. Thus, in various implementations, the light is emitted by the one or more light sources 120, reflects off the eye of the user 102, and is detected by the photodiodes 125.

[0023] In the example of Figure 1, both the photodiodes 125 and the one or more light sources 120 are embedded within the display 110, e.g., positioned in between pixel elements of the display 110 for example to form an array or matrix of equally or unequally spaced photodiodes.

[0024] Figure 11 illustrates an example in which both photodiodes and eyeillumination light sources are embedded amongst the pixels 1110a, 1110b, ... 111 On ofAttomey Docket No. 097425-01465(P63841WOl) a display 1100 at locations 1120a-i. These locations 1120a-i in this example forms a pattern of regularly-spaced locations throughout the pattern formed by the pixels 1110a, 1110b, ... l l lOn. The number of both photodiodes and eye-illumination light sources may (as illustrated in Figure 11) be significantly less than the number of display pixels. In various implementations, the number of photodiodes and eye-illumination light sources may be Vi, !4, 1 / 8, 1 / 16, 1 / 64, etc. the number of display pixels. In some implementations, the limited number of photodiodes provides a relatively low resolution image of a portion of the eye. In some implementations, the eye illumination light and / or photodiodes are configured to focus on a relatively small portion of the eye such that a limited number of photodiodes (e.g., less than the number of display pixels) provides adequate resolution for the various uses to which the eye image may be obtained and analyzed. In some implementations, photodiode data captured over time is combined to synthesize a higher-resolution image of a portion of the eye. Additionally, or alternatively, resolution enhancements processes (e.g., based on machine learning, using algorithms that are not based on learning, etc.) may be used to enhance the resolution of the eye images. In some implementations, light is focused on a relatively small portion of the eye such that a limited number of photodiodes (e.g., less than the number of display pixels) provides adequate resolution for the various uses to which the eye image may be obtained and analyzed.

[0025] Returning to Figure 1, in alternative implementations, only the photodiodes 125 are embedded with the display 110 and the light source 120 is positioned separate from the display 110 in the HMD 105. In alternative implementations, some or all of the one or more light sources are not embedded with the display 110, e.g., some or all of the one or more light sources 120 are positioned separate from (e.g., beyond the edges of) the display 110 in the HMD 105.

[0026] In some implementations, the display 110 (e.g., pixels used to display content to the user) emits light in a first wavelength range and the one or more light sources 120 emit light in a second wavelength range. Similarly, the photodiodes 125 may detect light in the second wavelength range. In various implementations, the first wavelength range is a visible wavelength range (e.g., a wavelength range within the visible spectrum of approximately 400-700 nm) and the second wavelength range is a near-infraredAttomey Docket No. 097425-01465(P63841WOl) wavelength range (e.g., a wavelength range within the near-infrared spectrum of approximately 700-1400 nm).

[0027] In various implementations, eye tracking (or, in particular, a determined gaze direction) is used to enable user interaction (e.g., the user 102 selects an option on the display 110 by looking at it), provide foveated rendering (e.g., present a higher resolution in an area of the display 110 the user 102 is looking at and a lower resolution elsewhere on the display 110), or correct distortions (e.g., for images to be provided on the display 110).

[0028] In various implementations, the one or more light sources 122 emit light towards the eye of the user 102 which reflects in the form of a plurality of glints.

[0029] In various implementations, the photodiodes 125 capture / detect information, at a particular point in time or multiple points in time at a frame rate, that is used to generate an image of the eye of the user 102. Each image may include a matrix of values corresponding to pixels of the image which correspond to locations of a matrix / array of photodiodes 125 positioned within the display 120, e.g., in between content display pixels of the display 110.

[0030] In various implementations, HMD 105 includes externally-facing sensors (e.g., cameras, not shown) for capturing information from outside of the head-mounted device 105. For example, to capture image data of the physical environment that the user 102 is viewing. The image data can include light intensity image data and / or depth data. For example, a first camera may be a video camera for capturing RGB data and a second camera may be a depth sensor (e.g., a structured light, a time-of-flight, or the like) for capturing depth data. These cameras may capture images or other information about a surrounding physical environment that is presented as a view on the display 110 of the HMD 105, e.g., as part of an extended reality (XR) environment.

[0031] A physical environment refers to a physical world that people can sense and / or interact with without aid of electronic devices. The physical environment may include physical features such as a physical surface or a physical object. For example, the physical environment may correspond to a physical park that includes physical trees, physical buildings, and physical people. People can directly sense and / or interact withAttorney Docket No. 097425-01465(P63841WOl) the physical environment such as through sight, touch, hearing, taste, and smell. In contrast, an XR environment refers to a wholly or partially simulated environment that people sense and / or interact with via an electronic device. For example, the XR environment may include augmented reality (AR) content, mixed reality (MR) content, virtual reality (VR) content, and / or the like. With an XR system, a subset of a person’s physical motions, or representations thereof, are tracked, and, in response, one or more characteristics of one or more virtual objects simulated in the XR environment are adjusted in a manner that comports with at least one law of physics. As one example, the XR system may detect head movement and, in response, adjust graphical content and an acoustic field presented to the person in a manner similar to how such views and sounds would change in a physical environment. As another example, the XR system may detect movement of the electronic device presenting the XR environment (e.g., a mobile phone, a tablet, a laptop, or the like) and, in response, adjust graphical content and an acoustic field presented to the person in a manner similar to how such views and sounds would change in a physical environment. In some situations (e.g., for accessibility reasons), the XR system may adjust characteristic(s) of graphical content in the XR environment in response to representations of physical motions (e.g., vocal commands).

[0032] In some implementations, the HMD 105 performs eye characteristic determinations by detecting eye shape, eye positions and / or eye movements. For example, an eye characteristic determining or eye tracking system may include one or more infrared (IR) light-emitting diodes (LEDs), an illumination source (e.g., an IR light source) that emits light (e.g., IR light) towards the eyes of the user, and / or IR photodiodes. NIR light may be used in addition or alternatively to IR light. In some implementations, images captured by the eye tracking system may be analyzed to detect position and movements of the eyes of the user, or to detect other information about the eyes such as pupil dilation or pupil diameter. Moreover, the point of gaze estimated from the eye tracking images may enable gaze-based interaction with content shown by the device.

[0033] In some implementations, the HMD 105 includes one or more graphical user interfaces (GUIs), one or more processors, memory and one or more modules, programs or sets of instructions stored in the memory for performing multiple functions. In someAttomey Docket No. 097425-01465(P63841WOl) implementations, the users may interact with a GUI through voice commands, finger contacts on a touch-sensitive surface, hand / body gestures, remote control devices, or other user input mechanisms. In some implementations, the functions include viewing / listening to content, image editing, drawing, presenting, word processing, website creating, disk authoring, spreadsheet making, game playing, telephoning, video conferencing, e-mailing, instant messaging, workout support, digital photographing, digital videoing, web browsing, digital music playing, or digital video playing. Executable instructions for performing these functions may be included in a computer readable storage medium or other computer program product configured for execution by one or more processors.

[0034] Some implementations provide methods of imaging the cornea, iris, retina, or other portion of (or around) a user’s eye in a near-eye system (such as HMD 105) whose display is embedded with an array of photodiodes (e.g., photodiodes 125 embedded in display 110). The photodiodes may be configured to function similarly to an image sensor in that they receive light and convert it to electric signals / data. The photodiodes may be organized in a predetermine / optimized fashion on the display surface in providing efficient imaging, while also maintaining satisfactory front of screen performance, e.g., not interfering with the display pixels’ ability to provide a high quality / high resolution image for the user to see. The eye imaging may be performed to facilitate eye tracking, authentication, vision health, or for other purposes.

[0035] Various configurations disclosed herein are used to perform efficient eye imaging within the physical / optical constraints of a near-eye display (e.g., an HMD). Various benefits may be achieved by having the imaging system co-located / aligned with the display, e.g., based on the eye imaging / tracking system capturing eye data from a desirable / ideal point of view. Furthermore, co-location of display and eye imaging / tracking components may reduce or eliminate the need for calibration (e.g., between eye imaging / tracking and display components) in comparison to devices that utilize an eye tracking camera system that is separate from the display. Another advantage may result in implementations in which the integration with the display enables implementation of a relatively larger light collection area, which in turn may help to improve signal to noise ratio, resolution, and offer larger gaze tracking angles. SomeAttorney Docket No. 097425-01465(P63841WOl) of the implementations enable 3D imaging (e.g., of the cornea or iris) and / or improve eye-tracking accuracy and performance.

[0036] In some implementations, a near-eye device such as an HMD includes an optic lens to facilitate the user viewing content displayed by a display of the device. The optic lens, for example, may be positioned or otherwise configured such that an image of the display forms on the retina of the user’s eye. Some implementations include photodiodes and / or light sources (i.e., eye imaging components) embedded within such a display that are positioned such that light directed towards or reflections received from the user’s eye must pass through the optic lens. Some implementations perform retinal imaging without requiring that the eye imaging components account for the optic lens. Since the lens is already focused on the retina, there is no need to adjust for this focus in the case of retinal imaging. However, some implementations, image other portions of the user’s eye, such as the cornea or iris, and account for the retinal focus of the optic lens.

[0037] Figure 2 illustrates exemplary HMD components including photodiodes embedded within a display for outer-eye imaging. In this example, the HMD components include optical elements 115 (e.g., one or more lenses) and display 110. The HMD also includes additional components that are omitted from the illustration for simplicity. In this example, the display 110 is configured to present content (e.g., via pixels) that is viewed by the eye of the user 102 through the optical elements 115. The optical elements 115 may be configured to provide a forward path (e.g., towards the eye 102) of light having certain wavelength characteristics (e.g., light primarily in the visible spectrum providing visible, RGB content).

[0038] In addition, the display 110 includes photodiodes 125 that are configured to capture light reflected from one or more portions of the eye of the user 102 and these photodiodes 125 may be configured to capture light having a second wavelength characteristic (e.g., light primarily outside of the visible spectrum, IR light, etc.). The HMD may include one or more lights sources to illuminate portions of the eye (e.g., with glints) having the second wavelength characteristic. The HMD may include such illumination sources on or in the display 110 (e.g., embedded within), separately fromAttorney Docket No. 097425-01465(P63841WOl) the display 110 and directed towards the eye 102, or separately from the display 110 and directed via an optical path to emit light at the display 110 towards the eye 102.

[0039] To facilitate imaging (via the photodiodes 125) of non-retinal eye portions (e.g., cornea, iris, etc.) the light path of light reflecting off of the eye 102 is controlled. In this example, the light path in the reverse direction (i.e., from the eye 102 to the photodiodes 125) is controlled via elements 217a, 217b that form a “stop” or aperture in the optical elements 115 (e.g., one or more lenses). The photodiodes 125, in this example, are concentrated in a designated region of the display 110. This designated region may correspond to the stop or aperture in the optical elements 115. The optical elements 115 (e g., one or more lenses) may include an IR aperture and / or blocks (e.g., elements 217a, 217b) that block light having the second wavelength characteristic captured by the photodiodes 125 (e.g., light primarily outside of the visible spectrum, IR light, etc.) everywhere except within a certain region (e.g., the center of the optical elements 115).

[0040] In some implementations specific to retinal imaging, no optical change may be required.

[0041] In some implementations, an optical element 115 is primarily configured to provide optics with respect to the visible wavelength spectrum, but also provides a stop that is specific to non-visible wavelength light. In some implementations, there is no index change (e.g., no additional lens as in Figure 5), i.e., just a stop / aperture. In some circumstances, this may provide relatively reasonable modulated transfer function (MTF)-based eye determinations and acceptable glint-based eye characteristic determinations.

[0042] In some implementations, the optical elements 115 include or otherwise form a circular aperture such as aperture 305a shown in Figure 3A. In other implementations, the optical elements 115 include or otherwise form a non-circular and / or engineered aperture such as aperture 305b illustrated in Figure 3B. A non-circular or engineered aperture, in some circumstances, may provide improved centroid / spot finding, improved signal-to-noise ratio (SNR), and / or other benefits. The shape of one or more apertures may be configured to facilitate recovery of a sharp image.Attomey Docket No. 097425-01465(P63841WOl)

[0043] In some implementations, an image of an eye portion formed from light captured via one or more photodiodes is sharpened via an algorithm or machine learning model, e.g., via a computer-implemented process that computationally reduces blur, recovers sharpness, etc.

[0044] Figure 4 illustrates exemplary HMD components including photodiodes embedded within a display for outer-eye imaging. In this example, the HMD components include optical elements 115 (e.g., one or more lenses) and display 110. Additional components of the HMD are omitted from the illustration for simplicity. In this example, the display 110 is configured to present content (e.g., via pixels) that is viewed by the eye of the user 102 through the optical elements 115. The optical elements may be configured to provide a forward path (e.g., towards the eye 102) of light having certain wavelength characteristics (e.g., light primarily in the visible spectrum providing visible, RGB content). In addition, the display 110 includes a first set of photodiodes 125a and a second set of photodiodes 125b that are configured to capture light reflected from one or more portions of the eye of the user 102. These sets of photodiodes 125a, 125b may be configured capture light having a second wavelength characteristic (e.g., light primarily outside of the visible spectrum, IR light, etc.). The HMD may include one or more lights sources to illuminate portions of the eye (e.g., with glints) having the second wavelength characteristic. The HMD may include such illumination sources on or in the display 110 (e.g., embedded within), separately from the display 110 and directed towards the eye 102, and / or separately from the display 110 and directed via an optical path to emit light at the display 110 towards the eye 102.

[0045] To facilitate imaging (via the sets of photodiodes 125a, 125b) of non-retinal eye portions (e.g., cornea, iris, etc.) the light path of light reflecting off of the eye 102 is controlled. In this example, the light path in the reverse direction (i.e., from the eye 102 to the photodiodes 125) is controlled via elements that form a plurality of “stops” or apertures in the optical elements 115 (e.g., one or more lenses). The first set of photodiodes 125a and the second set of photodiodes 125b, in this example, are concentrated in two separate, designated regions of the display 110. The optical elements 115 (e.g., one or more lenses) include a plurality of IR apertures and / or iris blocks that block light having the second wavelength characteristic captured by the sets ofAttorney Docket No. 097425-01465(P63841WOl) photodiodes 125a, 125b (e.g., light primarily outside of the visible spectrum, IR light, etc.) everywhere except within the relevant regions (e.g., corresponding to the sets of photodiodes 125a, 125b). The optical elements 115 may additionally provide IR pass optics (e.g., in a center or other region) having certain characteristics.

[0046] In some implementations, optical element 115 can be transitioned over time, e.g., from a single aperture (shown in Figs. 2, 3 A, 3B) to a multi-aperture implementation (e.g., shown in Fig. 4).

[0047] Some implementations provide a tunable lens that enables focal length control of the optical elements 115 (or portions thereof) over time. This may enable adjustments that account for different users having different eye shapes / cornea-to-retina distances. Optical element 115 characteristics may account for differences in rotation, diameter, bulge, and other eye characteristics amongst a population of device users.

[0048] Figure 5 illustrates exemplary HMD components including photodiodes embedded within a display for outer-eye imaging. In this example, the HMD components include optical elements 115 (e.g., one or more lenses) and display 110. Additional components of the HMD are omitted from the illustration for simplicity. In this example, the display 110 is configured to present content (e.g., via pixels) that is viewed by the eye of the user 102 through the optical elements 115. The optical elements 115 may be configured to provide a forward path (e.g., towards the eye 102) of light having certain wavelength characteristics (e.g., light primarily in the visible spectrum providing visible, RGB content). In addition, the display 110 includes photodiodes 125 that are configured to capture light reflected from one or more portions of the eye of the user 102. The photodiodes 125 may be configured to capture light having a second wavelength characteristic (e.g., light primarily outside of the visible spectrum, IR light, etc.). The HMD may include one or more lights sources to illuminate portions of the eye (e.g., with glints) having the second wavelength characteristic. The HMD may include such illumination sources on the display 110 (e.g., embedded within), separately from the display 110 and directed towards the eye 102, and / or separately from the display 110 and directed via an optical path to emit light at the display 110 towards the eye 102.Attorney Docket No. 097425-01465(P63841WOl)

[0049] Optical power elements may be used to facilitate imaging (via the photodiodes) of non-retinal eye portions (e.g., cornea, iris, pupil, etc.). In some implementations, an optical element 115 introduces optical power in a particular region to facilitate gathering more light for detection via photodiodes 125 (e.g., relative to using a stop / aperature) and / or improving the quality of image / data captured by the photodiodes 125. In some implementations, the optical elements 115 provide IR lensing that is equivalent to lens imaging. In some circumstances, this may provide improved MTF- based eye characteristic determinations.

[0050] In Figure 5, optical power for certain light (e.g., light having the second wavelength characteristic) is introduced by sub-lens 511. In contrast to aperture-based implementations (e.g., Figure 2), using a sub-lens 511 may enable more light to be captured by the photodiodes 125, e.g., reducing energy loss.

[0051] In some implementations, an optical element 115 is non-blocking in the visible wavelength spectrum, but provides a plurality of stops that are specific to non-visible wavelength light. This may provide light field imaging and / or may enable 3D imaging of non-retinal / outer portions of the eye (e.g., the cornea, iris, etc.). Sub-apertures may be configured to allow forming sub images having slightly different points of view. An image processing algorithm or machine learning model may be used to interpret such data to determine 3D characteristics of the non-retinal / outer portions of the eye (e.g., the cornea, ins, etc.).

[0052] In some implementations, the optical elements 115 provide an IR-only meta surface. This may involve forming a meta-surface, e.g., surface features on one or more paths inside or outside of the optical elements 115 that influence particular wavelengths of light (e.g., only non-visible or IR wavelengths) and leave other wavelengths of light undisturbed. In some circumstances, this may provide improved MTF-based eye determinations. In some implementations, the optical elements 115 may provide IR only “sub” catadioptric folding to focus IR light.

[0053] Figure 6 illustrates exemplary HMD components including photodiodes embedded within a display for outer-eye imaging. In this example, the HMD components include optical elements 615 (e.g., one or more lenses) and display 610. AdditionalAttomey Docket No. 097425-01465(P63841WOl) components of the HMD are omitted from the illustration for simplicity. In this example, the display 610 is configured to present content (e.g., via display pixels 612) that is viewed by the eye of the user 102 through the optical elements 615. The optical elements 615 may be configured to provide a forward path (e.g., towards the eye 102) of light having certain wavelength characteristics (e.g., light primarily in the visible spectrum providing visible, RGB content). In addition, the display 610 includes photodiodes 614 that are configured to capture light reflected from one or more portions of the eye of the user 102. The photodiodes 614 may be configured to capture light having a second wavelength characteristic (e.g., light primarily outside of the visible spectrum, IR light, etc.). The HMD may include one or more lights sources to illuminate portions of the eye (e.g., with glints) having the second wavelength characteristic. The HMD may include such illumination sources on or in the display 610 (e.g., embedded within), separately from the display 610 and directed towards the eye 102, and / or separately from the display 610 and directed via an optical path to emit light at the display 610 towards the eye 102.

[0054] Optical power elements may be used to facilitate imaging (via the photodiodes) of non-retinal eye portions (e.g., cornea, iris, etc.). In some implementations, the optical elements 615 introduce optical power via reflections and / or refractions. In some implementations, the optical elements 615 provide reflective catadioptric properties, e.g., using multiple surfaces that bounce light back and forth, for example, in a “pancake” style lens. Instead of a single lens with an index variation, the optical elements 615 may include elements having mirror coatings that cause light rays to reflect internally, with the curvature of the optical elements performing lens functions.

[0055] The optical elements 615 may include components that affect particular wavelengths of light. In Figure 6, these include IR half mirror 619, IR quarter- wave plate (QWP) and APF (e.g., a reflective polarizer) 618, visible QWP and AFP 617, and visible half mirror 616. Light 640 from display pixel 612 reaches the user’s retina 632 and is affected only by the visible light affecting components of the optical elements 615 (e.g., visible QWP and AFP (e.g., a reflective polarizer) 617, and visible half mirror 616) and the user’s eye lens 634. In contrast, light 642 used for eye characteristic assessment reaches the user’s outer eye 630 and returns to the photodiodes 614, and is affected only by the IR light affecting components of the optical elements 615 (e.g., IR half mirror 619,Attorney Docket No. 097425-01465(P63841WOl)IR QWP and APF 618). This implementation may involve including one or more lens elements to a catadioptric display lens to fold IR light only and focus eye characteristic determination light (e.g., glints) on photodiodes 614 on the panel of the display 610. A polarized light source may be used for eye characteristic determination light (e.g., glints). The IR modulation components can also be placed between visible light modulation components instead of outside as shown in figure 6 as illustration.

[0056] In contrast to the implementation illustrated in Fig. 5, the implementation illustrated in Figure 6 utilizes a combination of back-and-forth reflections / refractions instead of an IR-specific additional lens.

[0057] It should be noted that in the implementations described herein, the light used for eye characteristic determinations (e.g., glints) need not be IR light. Light having any characteristics that do not interfere with the nominal function of the display (e.g., to display content that the user can see) may be used. In some implementations, the eye characteristic determination light is configured to avoid conflict with light generated by display pixels of the display. In some implementations, the display includes pixels that display content using pixels that provide dispersive light and a second light source is used to produce light that is appropriate for eye characteristic determinations, e.g., light that is not dispersive and / or has other desirable characteristics. In some implementations, a light source producing light for eye characteristic determinations produces a light pattern (e.g., a pattern of glints).

[0058] Figure 7 illustrates exemplary HMD components including photodiodes embedded within a display for outer-eye imaging. In this example, the HMD components include optical elements 115 (e.g., one or more lenses), display 110, and a light source 750. Additional components of the HMD are omitted from the illustration for simplicity. In this example, the display 110 is configured to present content (e.g., via pixels) that is viewed by the eye of the user 102 through the optical elements 115. The optical elements 115 may be configured to provide a forward path (e.g., towards the eye 102) of light having certain wavelength characteristics (e.g., light primarily in the visible spectrum providing visible, RGB content). In addition, the display 110 includes photodiodes 125 that are configured to capture light reflected from one or more portions of the eye of theAttorney Docket No. 097425-01465(P63841WOl) user 102. The photodiodes 125 may be configured to capture light having a second wavelength characteristic (e.g., light primarily outside of the visible spectrum, IR light, etc.).

[0059] The light source 750 directs light towards the outer surface 730 of the eye, that light reflects off the outer surface 730 of the eye 102, passes through optical element 115, and is captured via photodiodes 725. The light source 750 may provide directional light that is redirected over time. For example, the light source 750 may include a microelectro-mechanical systems (mems) mirror that scans a collimated beam, e.g., to complete a line-of-sight triangle. Using timing and control of the mems mirror, in combination with signals from the photodiodes 125, the system can reconstruct images of the outer surface 730 of the eye 102 (e.g., of the cornea, pupil, iris, glint locations, etc.) and / or derive other eye characteristics (e.g., gaze direction, pupil center, pupil dilation, etc ). Different photodiodes 125 providing signals regarding detected light at different times as the scanners makes a pass provide a set of information from which an image may be generated, e.g., to map glints, to make eye characteristic determinations, etc.

[0060] Figure 8 illustrates exemplary HMD components including photodiodes embedded within a display and photometric stereo processing components. In this example, the HMD components include optical elements 115 (e.g., one or more lenses), display 110, LED components LI, L2, ... LN, controller 850, and photometric stereo processing component 860. Additional components of the HMD are omitted from the illustration for simplicity. In this example, the display is configured to present content (e.g., via pixels) that is viewed by the eye of the user 102 through the optical elements 115. The optical elements may be configured to provide a forward path (e.g., towards the eye 102) of light having certain wavelength characteristics (e.g., light primarily in the visible spectrum providing visible, RGB content). In addition, the display 110 includes photodiodes 125 that are configured to capture light reflected from one or more portions of the eye of the user 102 and these may be configured to capture light having a second wavelength characteristic (e.g., light primarily outside of the visible spectrum, IR light, etc.).Attorney Docket No. 097425-01465(P63841WOl)

[0061] The LED components LI, L2, ... LN are configured to illuminate portions of the eye (e.g., with glints) having the second wavelength characteristic. The HMD may include such illumination sources on or in the display 110 (e.g., embedded within), separately from the display 110 and directed towards the eye 102, or separately from the display 110 and directed via an optical path to emit light at the display 110 towards the eye 102.

[0062] The LED components LI , L2, ... LN may provide an illumination scheme that allows photometric stereo. In this example, the LED components LI, L2, ... LN provide a set of LEDs used to illuminate the scene (e.g., eye, cornea, etc.). The LEDs may be individually controlled such that the controller 850 can control their on-sate and, optionally, intensity. By controlling illumination “pattern” (e.g., LI, L2 ON, L4, L6 OLE), images may be captured and used to perform photometric stereo computations (e g., at photometric stereo processing component 860) to provide 3D features. Photometric stereo can produce 3D features such as surface normal, surface roughness, and / or a 3D map. Such information may be used for various purposes including, but not limited to, improving eye-tracking, authentication, and anti-spoofing.

[0063] While LED components LI, L2, ... LN are described as LEDs in the example above, alternative illumination sources may be used in other implementations.

[0064] The techniques and systems illustrated in Ligures 2-8 may be combined with one another in various combinations to achieve various benefits, as will be understood by those of ordinary skill in the art.

[0065] Figure 9 is a flowchart representation of a method for capture sensor data regarding portions of the eye (e.g., the cornea, iris, or retina, etc.) in a near-eye device (e.g., an HMD) using an array of photodiodes embedded within the device’s display. In some implementations, the techniques of method 900 are performed on an HMD, AR glasses, or other device having a processor, a lens, an eye-illumination light source, and a display with embedded photodiodes. In some implementations, the method 900 is performed on processing logic, including hardware, firmware, software, or a combination thereof. In some implementations, the method 900 is performed on a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory). In someAttomey Docket No. 097425-01465(P63841WOl) implementations, the method 900 is performed in combination of one or more devices as described herein. For example, sensor data from a plurality of light sensors may be acquired at an HMD, but the processing of the data (e.g., assess an eye characteristic) may be performed at a separate device (e.g., a mobile device, server, etc.).

[0066] At block 902, the method 900 comprises displaying content by producing light via the display. The light produced via the display may be light within a first wavelength range (e.g., wavelengths in the visible range).

[0067] At block 904, the method comprises producing eye-illumination light via the eye-illumination light source to illuminate a portion of the eye (e.g., one or more of the retina, cornea, iris, pupil, etc.). The eye-illumination light may be light within a second wavelength range (e.g., wavelengths in the IR range) outside of the first wavelength range. The second wavelength range may be such that the eye-illumination light does not substantially interfere with the light produced by the display.

[0068] At block 906, the method 900 comprises capturing reflections of the eyeillumination light via the embedded photodiodes of the display, wherein the eye- illumination light and the light produced via the display pass through an element that affects the eye-illumination light more than the light produced via the display.

[0069] At block 908, the method 900 comprises determining an eye characteristic of the eye based on the sensor data. The eye characteristic may comprise a position, size, and / or shape of the eye or a portion of the eye, a gaze direction, a focus depth, an amount of dilation, an amount of pupil constriction, an amount of eye lens constriction, etc. The eye characteristic may correspond to characteristics used for eye-tracking, user authentication, eye health monitoring, etc.

[0070] Some implementations provide determination of eye characteristics of outer surfaces of the eye (e.g., cornea, iris, etc.). Some implementations provide outer eye imaging by accounting for the optical elements’ retinal focus by using wavelengthspecific components (e.g., a wavelength specific aperture stop / pinhole, sub-lenses, metasurfaces (e.g., for back-and-forth reflections / refractions), or other features.

[0071] In some implementations, the portion of the eye is an outer-eye portion of the eye. In some implementations, producing the eye-illumination light to illuminate the portion of the eye comprises adjusting a focus of the eye-illumination light using anAttorney Docket No. 097425-01465(P63841WOl) element that affects light having the second wavelength characteristic (e.g., corresponding to eye determinations) without affecting light having the first wavelength characteristic (e.g., corresponding to light used to display content to the user). In some implementations, the element is an aperture stop as illustrated in Figures 2, 3A, and 3B. In some implementations, the element is an array of aperture stops each focusing the eyeillumination on a different sub-portion of the portion of the eye, as illustrated in Figure 4. In some implementations, the element is a second lens (e.g., embedded within the lens) as illustrated in Figure 5. In some implementations, the element is configured to produce reflections and refractions to change the focus of the eye-illumination light, as illustrated in Figure 6.

[0072] Some implementations provide outer-eye (e.g., corneal, iris, etc.) imaging by accounting for the optics retinal focus computationally (e.g., using an algorithm or machine learning model). Some implementations produce the eye-illumination light to illuminate the portion of the eye by adjusting a focus of the eye-illumination light computationally. Some implementations produce the eye-illumination light to illuminate the portion of the eye comprises by adjusting a focus of the eye-illumination light via a machine learning model.

[0073] Some implementations use a directional light source to illuminate the eye with light for eye characteristic determinations. The light source may be a scanning light source (e.g., MEMS scanner, projection component) configured to project light in a plurality of different directions over time (e.g., one or more MEMS lasers).

[0074] In some implementations, the eye characteristic is a position, size, or shape of the eye portion. In some implementations, the method 900 further involves tracking a gaze direction of the eye based on the eye characteristic, authenticating the user based on the eye characteristic, and / or assessing a health characteristic of the eye based on the eye characteristic.

[0075] Figure 10 is a block diagram of an example device 1000. Device 1000 illustrates an exemplary device system configuration for a device (e.g., devices 105, 125, 165, etc.). While certain specific features are illustrated, those skilled in the art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity, and so as not to obscure more pertinent aspects of theAttorney Docket No. 097425-01465(P63841WOl) implementations disclosed herein. To that end, as a non-limiting example, in some implementations the device 1000 includes one or more processing units 1002 (e.g., microprocessors, ASICs, FPGAs, GPUs, CPUs, processing cores, and / or the like), one or more input / output (I / O) devices and sensors 1006, one or more communication interfaces 1008 (e.g., USB, FIREWIRE, THUNDERBOLT, IEEE 802.3x, IEEE 802.1 lx, IEEE 802.16x, GSM, CDMA, TDMA, GPS, IR, BLUETOOTH, ZIGBEE, SPI, I2C, and / or the like type interface), one or more programming (e.g., I / O) interfaces 1010, one or more displays 1012, one or more interior and / or exterior facing image sensor systems 1014, a memory 1020, and one or more communication buses 1004 for interconnecting these and various other components.

[0076] In some implementations, the one or more communication buses 1004 include circuitry that interconnects and controls communications between system components. In some implementations, the one or more I / O devices and sensors 1006 include at least one of an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., blood pressure monitor, heart rate monitor, blood oxygen sensor, blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptics engine, one or more depth sensors (e.g., a structured light, a time-of-flight, or the like), and / or the like.

[0077] In some implementations, the one or more displays 1012 are configured to present a view of a physical environment or a graphical environment to the user. In some implementations, the one or more displays 1012 correspond to holographic, digital light processing (DLP), liquid-crystal display (LCD), liquid-crystal on silicon (LCoS), organic light- emitting field-effect transitory (OLET), organic light-emitting diode (OLED), surface-conduction electron-emitter display (SED), field-emission display (FED), quantum-dot light- emitting diode (QD-LED), micro-electromechanical system (MEMS), and / or the like display types. In some implementations, the one or more displays 1012 correspond to diffractive, reflective, polarized, holographic, etc. waveguide displays. In one example, the device 105 includes a single display. In another example, the device 105 includes a display for each eye of the user.Attorney Docket No. 097425-01465(P63841WOl)

[0078] In some implementations, the one or more image sensor systems 1014 are configured to obtain image data that corresponds to at least a portion of the physical environment. For example, the one or more image sensor systems 1014 include one or more RGB cameras (e g., with a complimentary metal-oxide-semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor), monochrome cameras, IR cameras, depth cameras, event-based cameras, and / or the like. In various implementations, the one or more image sensor systems 1014 further include illumination sources that emit light, such as a flash. In various implementations, the one or more image sensor systems 1014 further include an on-camera image signal processor (ISP) configured to execute a plurality of processing operations on the image data.

[0079] The memory 1020 includes high-speed random-access memory, such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices. In some implementations, the memory 1020 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 1020 optionally includes one or more storage devices remotely located from the one or more processing units 1002. The memory 1020 includes a non-transitory computer readable storage medium.

[0080] In some implementations, the memory 1020 or the non-transitory computer readable storage medium of the memory 1020 stores an optional operating system 1030 and one or more instruction set(s) 1040. The operating system 1030 includes procedures for handling various basic system services and for performing hardware dependent tasks. In some implementations, the instruction set(s) 1040 include executable software defined by binary information stored in the form of electrical charge. In some implementations, the instruction set(s) 1040 are software that is executable by the one or more processing units 1002 to carry out one or more of the techniques described herein.

[0081] The instruction set(s) 1040 include an illumination analysis instruction set 1042 and an eye characteristic instruction set 1044. The instruction set(s) 1040 may be embodied a single software executable or multiple software executables.

[0082] In some implementations, the illumination analysis instruction set 1042 is executable by the processing unit(s) 1002 to produce a reflection by directing lightAttorney Docket No. 097425-01465(P63841WOl) towards an eye using a light source (e.g., a MEMS scanner), receive sensor data from a sensor (e.g., a set of one or more photodiodes) and generate data based on the sensor data. To these ends, in various implementations, the instruction includes instructions and / or logic therefor, and heuristics and metadata therefor.

[0083] In some implementations, the eye characteristic instruction set 1044 is executable by the processing unit(s) 1002 to determine a characteristic of the eye based on the sensor data such as identifying and tracking a position and / or orientation of an eye, a gaze direction, the cornea shape, and the like, using one or more of the techniques discussed herein or as otherwise may be appropriate. To these ends, in various implementations, the instruction includes instructions and / or logic therefor, and heuristics and metadata therefor.

[0084] Although the instruction set(s) 1040 are shown as residing on a single device, it should be understood that in other implementations, any combination of the elements may be located in separate computing devices. Moreover, Figure 10 is intended more as functional description of the various features which are present in a particular implementation as opposed to a structural schematic of the implementations described herein. As recognized by those of ordinary skill in the art, items shown separately could be combined and some items could be separated. The actual number of instructions sets and how features are allocated among them may vary from one implementation to another and may depend in part on the particular combination of hardware, software, and / or firmware chosen for a particular implementation.

[0085] Numerous specific details are set forth herein to provide a thorough understanding of the claimed subject matter. However, those skilled in the art will understand that the claimed subject matter may be practiced without these specific details. In other instances, methods, apparatuses, or systems that would be known by one of ordinary skill have not been described in detail so as not to obscure claimed subject matter.

[0086] Unless specifically stated otherwise, it is appreciated that throughout this specification discussions utilizing the terms such as “processing,” “computing,” “calculating,” “determining,” and “identifying” or the like refer to actions or processesAttorney Docket No. 097425-01465(P63841WOl) of a computing device, such as one or more computers or a similar electronic computing device or devices, that manipulate or transform data represented as physical electronic or magnetic quantities within memories, registers, or other information storage devices, transmission devices, or display devices of the computing platform.

[0087] The system or systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device can include any suitable arrangement of components that provides a result conditioned on one or more inputs. Suitable computing devices include multipurpose microprocessor-based computer systems accessing stored software that programs or configures the computing system from a general purpose computing apparatus to a specialized computing apparatus implementing one or more implementations of the present subject matter. Any suitable programming, scripting, or other type of language or combinations of languages may be used to implement the teachings contained herein in software to be used in programming or configuring a computing device.

[0088] Implementations of the methods disclosed herein may be performed in the operation of such computing devices. The order of the blocks presented in the examples above can be varied for example, blocks can be re-ordered, combined, or broken into sub-blocks. Certain blocks or processes can be performed in parallel.

[0089] The use of “adapted to” or “configured to” herein is meant as open and inclusive language that does not foreclose devices adapted to or configured to perform additional tasks or steps. Additionally, the use of “based on” is meant to be open and inclusive, in that a process, step, calculation, or other action “based on” one or more recited conditions or values may, in practice, be based on additional conditions or value beyond those recited. Headings, lists, and numbering included herein are for ease of explanation only and are not meant to be limiting.

[0090] It will also be understood that, although the terms “first,” “second,” etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another. For example, a first node could be termed a second node, and, similarly, a second node could be termed a first node, which changing the meaning of the description, so long as all occurrences ofAttorney Docket No. 097425-01465(P63841WOl) the “first node” are renamed consistently and all occurrences of the “second node” are renamed consistently. The first node and the second node are both nodes, but they are not the same node.

[0091] The terminology used herein is for the purpose of describing particular implementations only and is not intended to be limiting of the claims. As used in the description of the implementations and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, objects, or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, objects, components, or groups thereof.

[0092] As used herein, the term “if’ may be construed to mean “when” or “upon” or “in response to determining” or “in accordance with a determination” or “in response to detecting,” that a stated condition precedent is true, depending on the context. Similarly, the phrase “if it is determined [that a stated condition precedent is true]” or “if [a stated condition precedent is true]” or “when [a stated condition precedent is true]” may be construed to mean “upon determining” or “in response to determining” or “in accordance with a determination” or “upon detecting” or “in response to detecting” that the stated condition precedent is true, depending on the context.

[0093] The foregoing description and summary of the invention are to be understood as being in every respect illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is not to be determined only from the detailed description of illustrative implementations but according to the full breadth permitted by patent laws. It is to be understood that the implementations shown and described herein are only illustrative of the principles of the present invention and that various modification may be implemented by those skilled in the art without departing from the scope and spirit of the invention.

Claims

Attorney Docket No. 097425-01465(P63841WOl)What is claimed is:

1. A method comprising: at a processor of a head-mounted device (HMD) comprising a lens, an eye-illumination light source, and a display with embedded photodiodes: displaying content by producing light via the display; producing eye-illumination light via the eye-illumination light source to illuminate a portion of the eye; capturing reflections of the eye-illumination light via the embedded photodiodes of the display, wherein the eye-illumination light and the light produced via the display pass through an element that affects the eye-illumination light more than the light produced via the display; and determining an eye characteristic of the eye based on the sensor data.

2. The method of claim 1 , wherein the portion of the eye is a retina of the eye.

3. The method of claim 1, wherein the light produced via the display has a first focal distance produced by the lens.

4. The method of claim 1 , wherein the portion of the eye is an outer-eye portion of the eye, wherein producing the eye-illumination light to illuminate the portion of the eye comprises adjusting a focus of the eye-illumination light using the element to affect light having a second wavelength characteristic of the eyeillumination light without affecting light having a first wavelength characteristic of the light produced via the display.

5. The method of claim 4, wherein the element is an aperture stop.

6. The method of claim 4, wherein the element is an array of aperture stops each focusing the eye-illumination on a different sub-portion of the portion of the eye.Attorney Docket No. 097425-01465(P63841WOl)7. The method of claim 4, wherein the element is a second lens.

8. The method of claim 4, wherein the element is configured to produce reflections and refractions to change the focus of the eye-illumination light.

9. The method of any of claims 1-8, wherein the portion of the eye is an outer-eye portion of the eye, wherein producing the eye-illumination light to illuminate the portion of the eye comprises adjusting a focus of the eye-illumination light computationally.

10. The method of any of claims 1 -9, wherein the portion of the eye is an outer-eye portion of the eye, wherein producing the eye-illumination light to illuminate the portion of the eye comprises adjusting a focus of the eye-illumination light via a machine learning model.

11. The method of any of claims 1 -9, wherein the light source is a scanning light source configured to project light in a plurality of different directions over time.

12. The method of any of claims 1-11, wherein the eye characteristic is a position, size, or shape of the eye portion.

13. The method of any of claims 1-12 further comprising tracking a gaze direction of the eye based on the eye characteristic.

14. The method of any of claims 1-13 further comprising authenticating the user based on the eye characteristic.

15. The method of any of claims 1-14 further comprising assessing a health characteristic of the eye based on the eye characteristic.Attorney Docket No. 097425-01465(P63841WOl)16. The method of any of claims 1-15, wherein the portion of the eye is a cornea or iris of the eye.

17. A system comprising: a display with embedded photodiodes: a lens; an eye-illumination light source; a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising any one of the methods of claims 1-16.

18. A non-transitory computer-readable storage medium, storing program instructions executable via a processor to perform operations comprising any one of the methods of claims 1-16.

Images