Scleral and ocular feature detection with crossed polarizers for eye tracking

Abstract

Various implementations disclosed herein include devices, systems, and methods that perform camera eye tracking using cross polarized illumination. For example, a process may direct light having a polarization along an axis towards an eye of a user. In response, an image corresponding to reflections of the light off of the eye may be captured via a camera configured to capture light at an orthogonal polarization relative to the polarization of the light. The image may depict a region of a sclera of the eye. The process may further track the eye based on the sclera in the image.

Description

Attorney Docket No. 097425-01466(P65770W01)SCLERAL AND OCULAR FEATURE DETECTION WITH CROSSED POLARIZERS FOR EYE TRACKINGTECHNICAL FIELD

[0001] The present disclosure generally relates to systems, methods, and devices that track eye movement using cross polarized illumination directed at an eye of a user to detect non-vascular features on a sclera of the eye.BACKGROUND

[0002] Existing techniques for tracking user gaze may be improved with respect to visibility and simplicity to provide accurate gaze tracking results.SUMMARY

[0003] Various implementations disclosed herein include devices, systems, and methods that perform camera-based eye / gaze tracking (to track eye movement) using cross-polarization. Various implementations may involve directing polarized light towards a user’s eye (e.g., light having a first polarization aligned with a first axis) and a light detection device configured to capture cross-polarized reflections of the light (e.g., light have a second polarization aligned with a second axis). Various implementations detect non-vascular features on a sclera (of the eye) that may be otherwise invisible (e.g., undetectable in captured images) using standard illumination techniques or ambient light. For example, collagen present in the sclera comprises birefringent properties that have a refractive index that depends on a polarization and propagation direction of light. In some implementations, the collagen present in the sclera may become visible (e.g., in captured images) using crossed polarized detection thereby enabling an eye tracking process with based on identification of visible features of the detected collagen.

[0004] In some implementations, edge detection filters such a gaussian filter, a bandpass filter, etc. may be utilized to further enhance visible features of the collagen.

[0005] In some implementations, an eye / gaze tracking process may include using a coaxial illumination structure with a polarizing beam splitter to illuminate a sclera (of aAttorney Docket No. 097425-01466(P65770W01) user’s eye) with polarized light. In some implementations, an eye / gaze tracking process may include using a ring illumination structure with crossed polarizers to illuminate a sclera with polarized light.

[0006] In response to illuminating a sclera of an eye of a user with polarized light, a camera facing the eye of the user may be configured to capture resulting light at orthogonal polarization relative to the illumination to capture an image corresponding to reflections of light from a region of the sclera. In some implementations, specular components (e.g., mirror like reflections) of the resulting light may be removed. In some implementations, polarization of rotating material (e.g., collagen) in the sclera may be enhanced.

[0007] In some implementations, the lighting may be produced via a vertical-cavity surface-emitting laser (VCSEL), a light emitting diode (LED), etc.

[0008] In some implementations, a camera eye / gaze tracking process may be implemented via a head mounted device (HMD) configured to allow ambient light to reach an eye and the ambient light may be polarized using a polarizer on a lens of the HMD.

[0009] In some implementations, the tracking process may be used to additionally track user facial features around the eye to predict facial expressions.

[0010] In some implementations, an electronic device has a processor (e.g., one or more processors) that executes instructions stored in a non-transitory computer-readable medium to perform a method. The method performs one or more steps or processes. In some implementations, the electronic device directs light having a polarization along an axis towards an eye of a user. In some implementations, an image corresponding to reflections of the light off of the eye is captured via a camera configured to capture light at an orthogonal polarization relative to the polarization of the light. The image may depict a region of a sclera of the eye. In some implementations, the eye may be tracked based on the image.

[0011] In accordance with some implementations, a device includes one or more processors, a non-transitory memory, and one or more programs; the one or more programs are stored in the non-transitory memory and configured to be executed by theAttorney Docket No. 097425-01466(P65770W01) one or more processors and the one or more programs include instructions for performing or causing performance of any of the methods described herein. In accordance with some implementations, a non-transitory computer readable storage medium has stored therein instructions, which, when executed by one or more processors of a device, cause the device to perform or cause performance of any of the methods described herein. In accordance with some implementations, a device includes: one or more processors, a non-transitory memory, and means for performing or causing performance of any of the methods described herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] So that the present disclosure can be understood by those of ordinary skill in the art, a more detailed description may be had by reference to aspects of some illustrative implementations, some of which are shown in the accompanying drawings.

[0013] Figures 1A-B illustrate exemplary electronic devices operating in a physical environment in accordance with some implementations.

[0014] Figure 2 illustrates is a diagram depicting a front view and a side view of a representation of an eyeball, in accordance with some implementations.

[0015] Figures 3A-3C illustrate differing views of an eye of a user with polarized illumination directed towards the eye to improve eye-related functions such as sclera recognition, in accordance with some implementations.

[0016] Figure 4 illustrates a view of a structure with a polarizing beam splitter, a coaxial illumination structure, and a camera, in accordance with some implementations.

[0017] Figure 5 illustrates a view of a structure with a light emitting diode (LED) implemented ring illumination structure comprising polarizers in accordance with some implementations.

[0018] Figure 6 illustrates a view of a structure with a VCEL implemented ring illumination structure comprising a diffuser and a polarizer, in accordance with some implementations.Attorney Docket No. 097425-01466(P65770W01)

[0019] Figure 7 is a flowchart representation of an exemplary method that tracks eye movement using cross polarized illumination directed at an eye of a user to detect non- vascular features on a sclera of the eye, in accordance with some implementations.

[0020] Figure 8 is a block diagram of an electronic device of in accordance with some implementations.

[0021] In accordance with common practice the various features illustrated in the drawings may not be drawn to scale. Accordingly, the dimensions of the various features may be arbitrarily expanded or reduced for clarity. In addition, some of the drawings may not depict all of the components of a given system, method or device. Finally, like reference numerals may be used to denote like features throughout the specification and figures.DESCRIPTION

[0022] Numerous details are described in order to provide a thorough understanding of the example implementations shown in the drawings. However, the drawings merely show some example aspects of the present disclosure and are therefore not to be considered limiting. Those of ordinary skill in the art will appreciate that other effective aspects and / or variants do not include all of the specific details described herein. Moreover, well-known systems, methods, components, devices and circuits have not been described in exhaustive detail so as not to obscure more pertinent aspects of the example implementations described herein.

[0023] Figures 1 A-B illustrate exemplary electronic devices 105 and 110 operating in a physical environment 100. In the example of Figures 1 A-B, the physical environment 100 is a room that includes a desk 120. The electronic devices 105 and 110 may include one or more cameras, one or more lighting sources having at least one polarizer, one or more microphones, depth sensors, or other sensors that can be used to capture information about and evaluate the physical environment 100 and the objects within it, as well as information (e.g., eye tracking information) about the user 102 of electronic devices 105 and 110. The information about the physical environment 100 and / or user 102 may be usedAttorney Docket No. 097425-01466(P65770W01) to provide visual and audio content and / or to identify the current location of the physical environment 100 and / or the location of the user within the physical environment 100.

[0024] In some implementations, views of an extended reality (XR) environment may be provided to one or more participants (e.g., user 102 and / or other participants not shown) via electronic devices 105 (e.g., a wearable device such as an HMD) and / or 110 (e.g., a handheld device such as a mobile device, a tablet computing device, a laptop computer, etc ). Such an XR environment may include views of a 3D environment that is generated based on camera images and / or depth camera images of the physical environment 100 as well as a representation of user 102 based on camera images and / or depth camera images of the user 102. Such an XR environment may include virtual content that is positioned at 3D locations relative to a 3D coordinate system associated with the XR environment, which may correspond to a 3D coordinate system of the physical environment 100.

[0025] Various implementations disclosed herein include devices, systems, and methods that implement gaze tracking approaches that use image data. In some implementations, gaze may be tracked using imaging data to determine eye position or eye orientation using a pupil plus glint model, using a depth camera (e.g., stereo, structured light projection, time-of-flight (ToF), etc.) with 3D point cloud registration, or using an appearance-based model.

[0026] In some implementations, a camera-based eye tracking process may be used to track a position, orientation, and / or movement of an eye of a user. The camera-based eye tracking process may be implemented via usage of cross polarized illumination directed along an axis towards the user’s eye to detect non-vascular features of a sclera that may be otherwise invisible with standard illumination or ambient light. For example, collagen present in the sclera typically includes birefringent properties having a refractive index that is dependent on a polarization and propagation direction of light thereby enabling the collagen to become visible when using crossed polarized detection techniques. In some implementations, edge detection filters such a gaussian filter, a bandpass filter, etc. may be utilized to further enhance visible features of the collagen.

[0027] In some implementations, an eye tracking process may include using at least one polarized illuminator to illuminate a sclera with polarized light by directing light of a lighting source having a polarization in a direction towards an eye of a user. In someAttorney Docket No. 097425-01466(P65770W01) implementations, the light source may be a coaxial illumination structure with a polarizing beam splitter. In some implementations, the light source may be a ring illumination structure with crossed polarizers.

[0028] In response, a camera (e.g., configured to capture light at an orthogonal polarization relative to the polarization of the light) facing the user’s eye may be configured to capture an image (depicting a region of a sclera) corresponding to reflections of the light from the eye.

[0029] In some implementations, specular components of the resulting light in the image may be removed.

[0030] In some implementations, the eye may be tracked based on the image. For example, tracking the eye may include detecting non- vascular features of the sclera based on the image. In some implementations, the non-vascular features comprise birefringent properties of collogen present in the sclera.

[0031] In some implementations, polarization of rotating material (e.g., collagen) in the sclera may be enhanced.

[0032] In some implementations, the eye tracking process may be used to additionally track user facial features around the eye to predict facial expressions.

[0033] Figure 2 illustrates a diagram depicting a front view 200a and a side view 200b of a representation of an eyeball 200 to illustrate implementations for eye / gaze tracking, in accordance with some implementations. Eyeball 200 comprises a cornea 230 that includes a transparent front part of an eye that covers an iris 215 and a pupil 243 of the eyeball 200. Likewise, eyeball 200 comprises a sclera 210.

[0034] In some implementations, a position of the eyeball 200 may be determined based on identifying non-vascular features of the sclera 210 surface by using polarized illumination directed towards eyeball 200 to detect the non-vascular features on sclera 210. For example, collagen present in the sclera comprises birefringent properties allowing the collagen to become visible using crossed polarized detection. In response, a camera configured to capture light at an orthogonal polarization relative to the polarization of the light, captures an image corresponding to reflections of the light off of the sclera 210 and uses the image to track a position, orientation, and / or movement of the eyeball 200.Attomey Docket No. 097425-01466(P65770W01)

[0035] Figures 3A-3C illustrate views 302a, 302b, and 302c of an eye 300 of a user with polarized illumination directed towards the eye 300 to improve eye- related functions such as sclera 304 recognition, in accordance with some implementations. For example, cross polarized illumination directed towards the eye 300 increases a visibility of collagen 308 present in sclera 304 such that one or more cameras capture images of the user's eyes for processing to perform sclera recognition, eye / gaze tracking, and / or other eye-related functions.

[0036] Figure 3A illustrates view 302a representing a real size view of eye 300 being illuminated via a lighting structure having a polarization directed along an axis towards the eye 300. For example, a lighting structure may include, inter alia, a coaxial illumination structure with polarizing beam splitter as illustrated with respect to figure 4 infra, a ring illumination with crossed polarizers as illustrated with respect to figures 5 and 6 infra, etc. In response to illumination of eye 300, view 302a illustrates visible collagen 308 present in a sclera 304 of eye 300 as collagen 308 comprises birefringent properties having a refractive index that depends on a polarization and propagation and direction of light. Therefore, collagen 308 becomes visible via usage of crossed polarized detection thereby enabling an eye tracking process with respect to identification of visible features of the detected collagen. Likewise, view 302a illustrates a portion 304a of sclera 304 being magnified (e.g., using a zoom feature) as illustrated in figure 3B, infra.

[0037] Figure 3B illustrates view 302b representing a magnified view of portion 304a of sclera 304. The magnified view of portion 304a of sclera 304 (of figure 3B) illustrates a view of trackable features of collagen 308 (e.g., collagen fibers) that may be captured as an image and used for tracking a position, orientation, and / or movement of eye 300.

[0038] Figure 3C illustrates view 302c representing a magnified and filtered view of portion 304a of sclera 304. The magnified and filtered view of portion 304a of sclera 304 (of figure 3B) illustrates a filtered view of trackable features of collagen 308 (e.g., collagen fibers) that may be captured as an image and used for tracking a position, orientation, and / or movement of eye 300. In some implementations, edge detection filters such a gaussian filter, a bandpass filter, a special bandpass filter etc. may be utilized to further enhance visible features of the collagen for captured via an image to be used for tracking a position, orientation, and / or movement of eye 300.Attorney Docket No. 097425-01466(P65770W01)

[0039] Figure 4 illustrates a view 400 of a structure 402 with a polarizing beam splitter 404, a coaxial illumination structure 408, and a camera 410, in accordance with some implementations. Polarizing beam splitter 404 is configured to transmit light of a specified polarization while reflecting light of orthogonal polarization. For example, an incoming unpolarized or polarized light beam from a light source 409 (e.g., a diode) is split into two separate output beams: a first beam 41 la that retains an original polarization and second beam 411b that has a polarization perpendicular to the first beam 411a.

[0040] View 400 illustrates structure 402 directing polarized illumination along an axis 412 towards an eye 414 of a user to enable collagen within a sclera 416 of eye 414 to become visible. In response to the polarized illumination directed to the eye 414, an image may be captured via camera 410 configured to capture light at an orthogonal polarization relative to the polarization of the light via a path 419. The image depicts a region of sclera 416 thereby enabling visible features of collagen of sclera 416 to be used for tracking a position, orientation, and / or movement of eye 414.

[0041] Figure 5 illustrates a view 500 of a structure 502 with a light emitting diode (LED) 509 implemented ring illumination structure 503 comprising polarizers (e.g., a polarizing filter) 522 and 524, in accordance with some implementations. Likewise, structure 502 comprises a camera 510. Structure 502 implements a cross polarization process via placement of polarizer 522 (e.g., a linear polarizer) in front of LED 509 (i.e., light source) and polarizer 524 (e.g., a linear polarizer) in front of camera 510.

[0042] Ring illumination structure 503 is configured to transmit light (from LED 509) through polarizer 522 to illuminate (via beam 511 traveling along axis 512) a sclera 516 of eye 514. View 500 illustrates structure 503 directing polarized illumination along axis 512 towards an eye 514 to enable collagen within sclera 416 of eye 514 to become visible. In response to the polarized illumination directed to the eye 514, an image may be captured via camera 510 configured to capture light at an orthogonal polarization relative to the polarization of the light via a path 519. The image depicts a region of sclera 516 thereby enabling visible features of collagen of sclera 516 to be used for tracking a position, orientation, and / or movement of eye 514.Attorney Docket No. 097425-01466(P65770W01)

[0043] Figure 6 illustrates a view 600 of a structure 602 with a VCSEL 609 implemented ring illumination structure 603 comprising a diffuser 622 and a polarizer 624, in accordance with some implementations. Likewise, structure 602 comprises a camera 610. Structure 602 implements a cross polarization process via placement of diffuser 622 within an illumination beam 611 produced by VCSEL 609 (i.e., light source) and polarizer 624 (e.g., a linear polarizer) in front of camera 610.

[0044] Ring illumination structure 603 is configured to transmit light (from VCSEL 609) through diffuser 622 to illuminate (via beam 611 traveling along axis 612) a sclera 616 of eye 614. View 600 illustrates structure 503 directing polarized illumination along axis 512 towards an eye 514 to enable collagen within sclera 416 of eye 614 to become visible. In response to the polarized illumination directed to the eye 614, an image may be captured via camera 610 configured to capture light at an orthogonal polarization relative to the polarization of the light via a path 619. The image depicts a region of sclera 616 thereby enabling visible features of collagen of sclera 616 to be used for tracking a position, orientation, and / or movement of eye 614.

[0045] Figure 7 is a flowchart representation of an exemplary method 600 that tracks eye movement using cross polarized illumination directed at an eye of a user to detect non-vascular features on a sclera of the eye, in accordance with some implementations. In some implementations, the method 700 is performed by a device, such as a mobile device, desktop, laptop, HMD, or server device. In some implementations, the device has a screen for displaying images and / or a screen for viewing stereoscopic images such as a head-mounted display (HMD such as e.g., device 105 or 110 of Figure 1). In some implementations, the method 700 is performed by processing logic, including hardware, firmware, software, or a combination thereof. In some implementations, the method 700 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory). Each of the blocks in the method 700 may be enabled and executed in any order.

[0046] At block 702, the method 700 directs light having a polarization along an axis towards an eye of a user. For example, a polarized light beam from a light source 409 may be directed along an axis 412 towards an eye 414 as described with respect to figure 4.Attorney Docket No. 097425-01466(P65770W01)

[0047] In some implementations, the light may be directed via a coaxial illumination structure with a polarizing beam splitter such as coaxial illumination structure 408 as described with respect to figure 4.

[0048] In some implementations, the light may be directed via a ring illumination structure with crossed polarizers such as ring illumination structure 503 as described with respect to figure 5.

[0049] In some implementations, the light may be produced via a VCSEL (e.g., VCSEL 609 of figure 6).

[0050] In some implementations, the light may be produced via an LED (e.g., LED 509 of figure 5).

[0051] At block 704, the method 700 an image may be captured via a camera configured to capture light at an orthogonal polarization relative to the polarization of the light. For example, an image may be captured via camera 610 configured to capture light at an orthogonal polarization relative to the polarization of the light via a path 619 as described with respect to figure 6. In some implementations, the image corresponds to reflections of the light off of the eye. In some implementations, the image depicts a region of a sclera of the eye such as portion 304a of sclera 304 as described with respect to figure 3.

[0052] In some implementations, polarization of rotating material (e.g., collagen) in the sclera may be enhanced.

[0053] In some implementations, the device is an HMD configured to allow ambient light to reach the eye and the ambient light may be polarized using a polarizer on a lens of the HMD.

[0054] At block 706, the method 700 tracks the eye (e.g., a position, orientation, movement, etc.) based on the image. In some implementations, the tracking process includes detecting features of the sclera of the eye based on the image. In some implementations, the tracking process includes detecting non-vascular features of the sclera based on the image. In some implementations, the tracking process includes detecting birefringent features of the sclera based on the image. In some implementations,Attorney Docket No. 097425-01466(P65770W01) the birefringent features comprise collagen present in the sclera. In some implementations, birefringent properties of collagen fibers in the sclera may be used to create trackable features using polarized light as follows:

[0055] In some implementations, an infrared light (e.g., 900nm) may be directed at an eye of the user with a single horizontal polarization and resulting light may be collected with a camera polarized perpendicularly (e.g., vertical polarization) thereby blocking direct reflection. For example, collagen fibers in the sclera include an optical property (birefringent property) that causes them to change the polarization of light passing through the collagen fibers. Subsequently, when the collagen fibers rotate the polarization (e g., with eye movement or rotation), some of the light may match the camera’s polarization (vertical) and may become visible thereby creating trackable features as follows:

[0056] Light that passes through or interacts with the collagen fibers may create detectable features in the sclera and since these fibers move with the eye's motion / rotation, they become trackable features that may be used for eye tracking.

[0057] Accordingly, block 706 is configured to take advantage of polarization optics to enhance, contrast and detect otherwise hidden structures in an eye (i.e., collagen fibers) thereby enabling eye feature tracking based on birefringent properties of the collagen fibers to enable an eye-tracking process.

[0058] For example, trackable features of collagen 308 (e.g., collagen fibers) captured as an image may be used for tracking a position, orientation, and / or movement of an eye 300 as described with respect to figure 3.

[0059] In some implementations, specular components may be removed from the image.

[0060] In some implementations, an edge detection filter may be applied to the image such that the tracking process includes detecting features of the sclera based on the filtered image.

[0061] Figure 8 is a block diagram of an example device 800. Device 800 illustrates an exemplary device configuration for electronic devices 105 and 110 of Figure 1 . WhileAttorney Docket No. 097425-01466(P65770W01) certain specific features are illustrated, those skilled in the art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity, and so as not to obscure more pertinent aspects of the implementations disclosed herein. To that end, as a non-limiting example, in some implementations the device 800 includes one or more processing units 802 (e.g., microprocessors, ASICs, FPGAs, GPUs, CPUs, processing cores, and / or the like), one or more input / output (I / O) devices and sensors 806, one or more communication interfaces 808 (e.g., USB, FIREWIRE, THUNDERBOLT, IEEE 802.3x, IEEE 802.1 lx, IEEE 802.14x, GSM, CDMA, TDMA, GPS, IR, BLUETOOTH, ZIGBEE, SPI, I2C, and / or the like type interface), one or more programming (e.g., I / O) interfaces 810, output devices (e.g., one or more displays) 812, one or more interior and / or exterior facing image sensor systems 814, a memory 820, and one or more communication buses 804 for interconnecting these and various other components.

[0062] In some implementations, the one or more communication buses 804 include circuitry that interconnects and controls communications between system components. In some implementations, the one or more I / O devices and sensors 806 include at least one of an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., blood pressure monitor, heart rate monitor, blood oxygen sensor, blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptics engine, one or more depth sensors (e.g., a structured light, a time-of-flight, or the like), one or more cameras (e.g., inward facing cameras and outward facing cameras of an HMD), one or more infrared sensors, one or more heat map sensors, and / or the like.

[0063] In some implementations, the one or more displays 812 are configured to present a view of a physical environment, a graphical environment, an extended reality environment, etc. to the user. In some implementations, the one or more displays 812 are configured to present content (determined based on a determined user / object location of the user within the physical environment) to the user. In some implementations, the one or more displays 812 correspond to holographic, digital light processing (DLP), liquidcrystal display (LCD), liquid-crystal on silicon (LCoS), organic light-emitting fieldeffect transitory (OLET), organic light-emitting diode (OLED), surface-conductionAttorney Docket No. 097425-01466(P65770W01) electron-emitter display (SED), field-emission display (FED), quantum-dot lightemitting diode (QD-LED), micro-electromechanical system (MEMS), and / or the like display types. In some implementations, the one or more displays 812 correspond to diffractive, reflective, polarized, holographic, etc. waveguide displays. In one example, the device 800 includes a single display. In another example, the device 800 includes a display for each eye of the user.

[0064] In some implementations, the one or more image sensor systems 814 are configured to obtain image data that corresponds to at least a portion of the physical environment 100. For example, the one or more image sensor systems 814 include one or more RGB cameras (e.g., with a complimentary metal-oxide-semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor), monochrome cameras, IR cameras, depth cameras, event-based cameras, and / or the like. In various implementations, the one or more image sensor systems 814 further include illumination sources that emit light, such as a flash. In various implementations, the one or more image sensor systems 814 further include an on-camera image signal processor (ISP) configured to execute a plurality of processing operations on the image data.

[0065] In some implementations, sensor data may be obtained by device(s) (e.g., devices 105 and 110 of Figure 1) during a scan of a room of a physical environment. The sensor data may include a 3D point cloud and a sequence of 2D images corresponding to captured views of the room during the scan of the room. In some implementations, the sensor data includes image data (e.g., from an RGB camera), depth data (e.g., a depth image from a depth camera), ambient light sensor data (e.g., from an ambient light sensor), and / or motion data from one or more motion sensors (e.g., accelerometers, gyroscopes, IMU, etc ). In some implementations, the sensor data includes visual inertial odometry (VIO) data determined based on image data. The 3D point cloud may provide semantic information about one or more elements of the room. The 3D point cloud may provide information about the positions and appearance of surface portions within the physical environment. In some implementations, the 3D point cloud is obtained over time, e.g., during a scan of the room, and the 3D point cloud may be updated, and updated versions of the 3D point cloud obtained over time. For example, a 3D representation mayAttorney Docket No. 097425-01466(P65770W01) be obtained (and analyzed / processed) as it is updated / adjusted over time (e.g., as the user scans a room).

[0066] In some implementations, sensor data may be positioning information, some implementations include a VIO to determine equivalent odometry information using sequential camera images (e.g., light intensity image data) and motion data (e.g., acquired from the IMU / motion sensor) to estimate the distance traveled. Alternatively, some implementations of the present disclosure may include a simultaneous localization and mapping (SLAM) system (e.g., position sensors). The SLAM system may include a multidimensional (e.g., 3D) laser scanning and range-measuring system that is GPS independent and that provides real-time simultaneous location and mapping. The SLAM system may generate and manage data for a very accurate point cloud that results from reflections of laser scanning from objects in an environment. Movements of any of the points in the point cloud are accurately tracked over time, so that the SLAM system can maintain precise understanding of its location and orientation as it travels through an environment, using the points in the point cloud as reference points for the location.

[0067] In some implementations, the device 800 includes an eye tracking system for detecting eye position and eye movements (e.g., eye gaze detection). For example, an eye tracking system may include one or more infrared (IR) light-emitting diodes (LEDs), an eye tracking camera (e.g., near-IR (NIR) camera), and an illumination source (e.g., an NIR light source) that emits light (e.g., NIR light) towards the eyes of the user. Moreover, the illumination source of the device 800 may emit NIR light to illuminate the eyes of the user and the NIR camera may capture images of the eyes of the user. In some implementations, images captured by the eye tracking system may be analyzed to detect position and movements of the eyes of the user, or to detect other information about the eyes such as pupil dilation or pupil diameter. Moreover, the point of gaze estimated from the eye tracking images may enable gaze-based interaction with content shown on the near-eye display of the device 800.

[0068] The memory 820 includes high-speed random-access memory, such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices. In some implementations, the memory 820 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices,Attorney Docket No. 097425-01466(P65770W01) or other non-volatile solid-state storage devices. The memory 820 optionally includes one or more storage devices remotely located from the one or more processing units 802. The memory 820 includes a non-transitory computer readable storage medium.

[0069] In some implementations, the memory 820 or the non-transitory computer readable storage medium of the memory 820 stores an optional operating system 830 and one or more instruction set(s) 840. The operating system 830 includes procedures for handling various basic system services and for performing hardware dependent tasks. In some implementations, the instruction set(s) 840 include executable software defined by binary information stored in the form of electrical charge. In some implementations, the instruction set(s) 840 are software that is executable by the one or more processing units 802 to carry out one or more of the techniques described herein.

[0070] The instruction set(s) 840 includes a light directing instruction set 842 and an eye tracking instruction set 844. The instruction set(s) 840 may be embodied as a single software executable or multiple software executables.

[0071] The light directing instruction set 842 is configured with instructions executable by a processor to direct light having a polarization along an axis towards an eye of a user.

[0072] The eye tracking instruction set 844 is configured with instructions executable by a processor to tracking an eye (e.g., position, orientation, movement) based on an image depicting a region of a sclera of the eye.

[0073] Although the instruction set(s) 840 are shown as residing on a single device, it should be understood that in other implementations, any combination of the elements may be located in separate computing devices. Moreover, Figure 8 is intended more as functional description of the various features which are present in a particular implementation as opposed to a structural schematic of the implementations described herein. As recognized by those of ordinary skill in the art, items shown separately could be combined and some items could be separated. The actual number of instructions sets and how features are allocated among them may vary from one implementation to another and may depend in part on the particular combination of hardware, software, and / or firmware chosen for a particular implementation.Attorney Docket No. 097425-01466(P65770W01)

[0074] Those of ordinary skill in the art will appreciate that well-known systems, methods, components, devices, and circuits have not been described in exhaustive detail so as not to obscure more pertinent aspects of the example implementations described herein. Moreover, other effective aspects and / or variants do not include all of the specific details described herein. Thus, several details are described in order to provide a thorough understanding of the example aspects as shown in the drawings. Moreover, the drawings merely show some example embodiments of the present disclosure and are therefore not to be considered limiting.

[0075] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombmation.

[0076] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0077] Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirableAttorney Docket No. 097425-01466(P65770W01) results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

[0078] Embodiments of the subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, e.g., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus. Alternatively, or additionally, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. A computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0079] The term “data processing apparatus” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, or combinations, of the foregoing. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a crossplatform runtime environment, a virtual machine, or a combination of one or more ofAttorney Docket No. 097425-01466(P65770W01) them. The apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing and grid computing infrastructures. Unless specifically stated otherwise, it is appreciated that throughout this specification discussions utilizing the terms such as “processing,” “computing,” “calculating,” “determining,” and “identifying” or the like refer to actions or processes of a computing device, such as one or more computers or a similar electronic computing device or devices, that manipulate or transform data represented as physical electronic or magnetic quantities within memories, registers, or other information storage devices, transmission devices, or display devices of the computing platform.

[0080] The system or systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device can include any suitable arrangement of components that provides a result conditioned on one or more inputs. Suitable computing devices include multipurpose microprocessor-based computer systems accessing stored software that programs or configures the computing system from a general purpose computing apparatus to a specialized computing apparatus implementing one or more implementations of the present subject matter. Any suitable programming, scripting, or other type of language or combinations of languages may be used to implement the teachings contained herein in software to be used in programming or configuring a computing device.

[0081] Implementations of the methods disclosed herein may be performed in the operation of such computing devices. The order of the blocks presented in the examples above can be varied for example, blocks can be re-ordered, combined, and / or broken into sub-blocks. Certain blocks or processes can be performed in parallel. The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0082] The use of “adapted to” or “configured to” herein is meant as open and inclusive language that does not foreclose devices adapted to or configured to perform additional tasks or steps. Additionally, the use of “based on” is meant to be open and inclusive, in that a process, step, calculation, or other action “based on” one or more recited conditions or values may, in practice, be based on additional conditions or valueAttorney Docket No. 097425-01466(P65770W01) beyond those recited. Headings, lists, and numbering included herein are for ease of explanation only and are not meant to be limiting.

[0083] It will also be understood that, although the terms “first,” “second,” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first node could be termed a second node, and, similarly, a second node could be termed a first node, which changing the meaning of the description, so long as all occurrences of the “first node” are renamed consistently and all occurrences of the “second node” are renamed consistently. The first node and the second node are both nodes, but they are not the same node.

[0084] The terminology used herein is for the purpose of describing particular implementations only and is not intended to be limiting of the claims. As used in the description of the implementations and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0085] As used herein, the term “if’ may be construed to mean “when” or “upon” or “in response to determining” or “in accordance with a determination” or “in response to detecting,” that a stated condition precedent is true, depending on the context. Similarly, the phrase “if it is determined [that a stated condition precedent is true]” or “if [a stated condition precedent is true]” or “when [a stated condition precedent is true]” may be construed to mean “upon determining” or “in response to determining” or “in accordance with a determination” or “upon detecting” or “in response to detecting” that the stated condition precedent is true, depending on the context.

Claims

Attorney Docket No. 097425-01466(P65770W01)What is claimed is:

1. A method comprising: at an electronic device having a processor: directing light having a polarization along an axis towards an eye of a user; capturing, via a camera configured to capture light at an orthogonal polarization relative to the polarization of the light, an image corresponding to reflections of the light off of the eye, the image depicting a region of a sclera of the eye; and tracking the eye based on the image.

2. The method of claim 1, wherein the tracking comprises detecting features of the sclera of the eye based on the image.

3. The method of any of claims 1-2, wherein the tracking comprises detecting non-vascular features of the sclera based on the image.

4. The method of any of claims 1-3, wherein the tracking comprises detecting birefringent features of the sclera based on the image.

5. The method of claim 4, wherein the birefringent features comprise collagen present in the sclera.

6. The method of any of claims 1-5 further comprising removing specular components of the image.

7. The method of any of claims 1-6 further comprising enhancing polarization of rotating material in the sclera.

8. The method of any of claims 1-7, wherein the light is directed by a coaxial illumination structure with a polarizing beam splitter.Attorney Docket No. 097425-01466(P65770W01)9. The method of any of claims 1-7, wherein the light is directed by a ring illumination structure with crossed polarizers.

10. The method of any of claims 1-9, wherein the light is produced via a vertical-cavity7surface-emitting laser (VCSEL).

11. The method of any of claims 1-9, wherein the light is produced via a light emitting diode (LED).

12. The method of any of claims 1-11, wherein the device is a head mounted device configured to allow ambient light to reach the eye, and the method further comprises polarizing the ambient light using a polarizer.

13. The method of claim 12, wherein the polarizer is on a lens of the head mounted device.

14. The method of any of claims 1-11, further comprising applying an edge detection filter to the image, wherein the tracking comprises detecting features of the sclera of the eye based on the filtered image.

15. A non-transitory computer-readable medium comprising instructions that when executed by a processor cause the processor to perform operations comprising any one of the methods of claims 1-14.Attorney Docket No. 097425-01466(P65770W01)16. An electronic device comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the electronic device to perform operations comprising: directing light having a polarization along an axis towards an eye of a user; capturing, via a camera configured to capture light at an orthogonal polarization relative to the polarization of the light, an image corresponding to reflections of the light off of the eye, the image depicting a region of a sclera of the eye; and tracking the eye (e.g., position, orientation, movement) based on the image.

17. The electronic device of claim 16, wherein the tracking comprises detecting features of the sclera of the eye based on the image.

18. The electronic device of any of claims 16-17, wherein the tracking comprises detecting non-vascular features of the sclera based on the image.

19. The electronic device of any of claims 16-18, wherein the tracking comprises detecting birefringent features of the sclera based on the image.

20. The electronic device of claim 19, wherein the birefringent features comprise collagen present in the sclera.

Images