Method and system for eye sensing

Abstract

The present invention relates to a method for sensing an eye. The method comprises the step of projecting a sparse light pattern (4) onto said eye (1), by means of a projector (3). The method further comprises the step of imaging light (5) reflected by the eye and received, by means of optics, on sensing units (7) of an optical event-based sensor (6), wherein the light (5) received on said optical sensor (6) includes a reflected sparse light pattern. The method further comprises the step of detecting a discontinuity in the reflected sparse light pattern compared to the projected sparse light pattern. The method further comprises the step of determining a position of the eye relative to the optical sensor at least partly based on said detected discontinuity.

Description

[0001] METHOD AND SYSTEM FOR EYE SENSING

[0002] TECHNICAL FIELD

[0003] The present invention relates to a method and system for eye sensing.

[0004] BACKGROUND

[0005] Eye tracking is important for many applications. As an example, eye tracking is applied in various AR or VR applications, in particular in AR or VR wearable devices. Eye tracking can also provide valuable information about a person's state, for example in detecting attention, fatigue, anxiety or other states. However, actual eye tracking methods cannot derive an absolute eye position and lack accuracy. Moreover, it is common to use heavy image processing for tracking the eye. This makes such solutions complex, slow, and power inefficient.

[0006] There is a need therefore for power efficient, fast, and simple solutions for eye tracking with improved accuracy. The present invention aims to resolve at least in part the problems mentioned above.

[0007] SUMMARY OF THE INVENTION

[0008] It is an object of embodiments of the present invention to overcome the limitations discussed above and obtain a power efficient, simple, and fast solution for eye tracking or sensing.

[0009] In a first aspect, the present invention relates to a method for sensing an eye, the method comprising the steps of:

[0010] projecting a sparse light pattern onto said eye, by means of a projector,

[0011] imaging light by the eye and received, by means of optics, on sensing units of an optical event-based sensor, wherein the light received on said optical sensor includes a reflected sparse light pattern, detecting a discontinuity in the reflected sparse light pattern compared to the projected sparse light pattern,

[0012] determining a position of the eye relative to the optical sensor at least partly based on said detected discontinuity.

[0013] A sparse light pattern can be defined in analogy with a sparse matrix. A sparse light pattern can for example include a static line or curve, a plurality of parallel or non-parallel lines or curves, or other relatively simple patterns in a field which is further empty. A static dot cannot be considered as a pattern. An event-based sensor can be defined as optical sensor only outputting data when an event is detected. Output data can then comprise coordinates of the detected event, in particular of the reflected light pattern on said optical sensor. The optical sensor is preferably a highframe rate sensor. Each sensing unit of the optical sensor, preferably an event-based sensor which preferably comprises a photo detector, more preferably a single photon detector, for example a single photon avalanche detector SPAD. The method can allow to determine a relative position of the eye independently of a general eye model and without the need for a further RGB camera.

[0014] It is an advantage of embodiments of the present invention that the gaze of the eye can be obtained based on the reflected detection of the light pattern, hence no need for heavy image processing. This is advantageous in having a power efficient method. It is an advantage of embodiment of the present invention that a fast tracking of the eye is obtained, due to requiring only one or a few time-windows. The method is also reliable and accurate, since it relies on light reflections from the eye.

[0015] Preferred embodiments of the first aspect of the invention comprise one or a suitable combination of more than one of the following features.

[0016] The discontinuity can be a change in direction of reflected light in the reflected light pattern or a transition between a detection and a non-detection of reflected light in the reflected light pattern. In the first case, the change in direction can allow to determine a position of the limbus where there is a sudden change in depth difference causing the change in direction of reflected light in the reflected light pattern. In the second case, a transition between a detection and a non-detection of reflected light in the reflected light pattern can allow to determine an edge of the pupil of the eye since a pupil will not reflect any light causing an interruption in the reflection of a projected light pattern. Other discontinuities in the reflected light pattern can be associated to other portions of the eye.

[0017] It is preferred that the reflected sparse light pattern covers less than 10% of a field of view of the optical sensor. By using a light pattern which can be detected on a relatively small fraction of the field of view of the optical sensor, processing can be done relatively fast with relatively limited power.

[0018] The light pattern is preferably projected for at least one time window, wherein the light pattern covers less than 10% of the eye in said time window. It is an advantage of embodiments of the present invention that a power efficient system is obtained, as well as a low latency system, since the amount of data to be handled is small at each time step as well as over time, since the detections corresponding to the reflection of the light pattern is used for sensing the eye, unlike flood illumination in the prior art which requires getting data from the full eye at each time step.

[0019] The method may further comprise a step of scanning said light pattern on said eye over consecutive time steps. When projecting a dot over consecutive time steps while scanning, a dot becomes a line or a curve, so a dot can qualify as a sparse light pattern when it is scanned.The method preferably further comprises the step of scanning said light pattern on said eye along reconfigurable positions over consecutive time steps, e.g. by means of a scanner, wherein said light pattern is different on said eye for any two consecutive time steps. It is an advantage of embodiments of the present invention that the eye is sparsely illuminated with light, and therefore the amount of data to be handled is low, as explained above.

[0020] The projector can be a plurality of projectors and the step of projecting a sparse light pattern may be performed by different projectors of the plurality of projectors over consecutive time steps. Said plurality of projectors may for example be an array of projectors such that every projector of said plurality of projectors has a slightly different location. Switching or alternating between projectors to project a sparse light pattern on the eye can thus be an alternative to scanning a sparse light pattern.

[0021] The method preferably further comprises the step of determining properties of the eye, wherein said properties of the eye comprise the pose and / or orientation and / or gaze of the eye.

[0022] The first area preferably corresponds to at least one of the pupil and limbus of the eye. It is an advantage of embodiment of the present invention that the pupil of the eye can be distinguished from other parts of the eye, due to the little to no reflections by said pupil. It is an advantage of embodiment of the present invention that a fast monitoring of the pupil is obtained, given the fast scanning.

[0023] The properties of the eye preferably comprise at least one of: an instantaneous position of the pupil, an instantaneous orientation of the pupil, and an instantaneous dimension of the pupil. It is an advantage of embodiment of the present invention that the eye can be continuously tracked.

[0024] In particular, the method may comprise the step of determining a diameter of the pupil based on a plurality of discontinuities, in particular of transitions between a detection and a non-detection of reflected light in the reflected light pattern. This can for example be done based on a single projection of a plurality of lines, or on a scanning projection of a single line. A diameter of the pupil may then be used to derive a state of anxiety of the person.

[0025] The method preferably further comprises the step of constructing photon statistics for photons received by said optical sensor, said statistics comprising the number of photons received by each sensing units of said optical sensor and determining a reflectivity profile of at least portions of said eye based on said photon statistics. The reflectivity profile can then be used to determine an iris of the person. Differences in reflectivity profile within the eye can help in distinguishing parts of the eye, such as distinguishing between the sclera, which is highly reflective, and the iris, which is less reflective.It is an advantage of embodiment of the present invention that information about positions of different parts of the eye is obtained.

[0026] The method preferably further comprises the step of identifying a person based on the determined reflectivity of at least a portion of the eye. The reflectivity profile of the eye can be combined with a statistical eye model. This can allow identification via a person's iris. It is advantageous identification can be done without complex image processing.

[0027] The projector and optical sensor are preferably oriented with respect to each other, such that a first Purkinje reflection and / or a fourth Purkinje reflection is received by said optical sensor. It is advantageous that such reflections allow detecting the pupil in a more accurate way. The method can then further comprise the step of deriving a change in direction of eye gaze over consecutive time steps. As an example, a dot can be projected into the pupil and can be scanned. When an event-based sensor which is capable of detecting events at a higher frequency than eye movement, detects a change in distance between a first Purkinje reflection and a fourth Purkinje reflection, it can be concluded that the eye is moving. The gaze direction and the direction of eye movement can then be derived. The scanner can then be configured to follow said eye movement.

[0028] The light pattern is preferably a pulsed light pattern. It is an advantage of embodiments of the present invention that power is concentrated in one pulse, which allows easier detection and filtering.

[0029] The method may further comprise the step of determining a relative distance between the optical sensor and a plurality of points of the eye reflecting said light pattern, preferably based on triangulation, and determining a gaze direction of the eye. An angle of projection of the sparse light pattern is known at every consecutive time steps, in other words, at each projection instance. This can allow to build a depth map of at least some points of the eye. Alternatively, a depth map based on time of flight could be used. When a plurality of dots or line positions of the sclera and cornea are known, an eye model can be made. At least some of said plurality of points of the eye can advantageously correspond to said discontinuity in the reflected sparse light pattern.

[0030] The method preferably comprises the step of triangulating locations of detections of said light pattern, with the corresponding instantaneous projection vector direction of the projector. It is an advantage of embodiments of the present invention that 3D information of the eye is obtained, and therefore the eye gaze or pose is obtained.

[0031] At least some of the triangulated locations of detections preferably correspond to the points on said transition. It is an advantage of embodiments of the present invention that the 3D position of the pupil can be obtained, possibly for eye gaze estimation.The method may further comprise the step of determining an amplitude of a saccade. A saccade can have a positional amplitude, i.e. indicating an amplitude of a change in position, and / or can have a temporal amplitude, i.e. a time during which a gaze is moved away. Both types of amplitudes of saccades can be determined. Average saccade rates can happen up to 10 or 20 times per second, however a minimal duration of a saccade could be as low as 2 millisecond. For the detection of high frequency saccades, event-based sensors with a relatively high sampling rate may be used, in particular having a refresh rate of at least be 10 times higher than saccade frequency, for example having a refresh rate of at least 1000 / sec, or more preferably of 10 000 / sec. Amplitudes of saccades can provide information on attention or focus of the person.

[0032] The method can comprise the step of determining an absolute position (e.g. 3D position) of the eye, a direction of gaze and a gaze position when two detectors are used and when a distance between said two detectors is known and fixed.

[0033] The method preferably comprises the step of filtering the detection determined by one sensing unit of said optical sensor, based on the presence of a detection over at least one neighboring sensing unit to said one sensing unit at one time step, and / or based on the presence of said detection over at least two consecutive time steps. It is an advantage of embodiments of the present invention that accurate optical sensing is obtained.

[0034] The method preferably further comprises the step of partially reading said optical sensor, wherein said optical sensor only outputs data when an event is detected, wherein said data corresponds to the reflected light pattern. The optical sensor is preferably an event-based sensor. It is an advantage of embodiments of the present invention that a power efficient optical sensor is obtained, since it is only required to read the pixels which detected an event.

[0035] The method preferably comprises the step of determining a first set of areas on the optical sensor in which the number of detections is below a threshold number of detections within predetermined time steps, and determining different a second set of areas on the optical sensor in which the number of detections is above said threshold number of detections within said predetermined time steps, and based thereon, the method further comprises the step of determining a corresponding area on the eye to each of said first and second set of parts. This is advantageous in differentiating between at least two areas of the eye.

[0036] Preferably, the eye is being defined at least by two orthogonal dimensions X and Y, where in the method further comprises the steps of:

[0037] defining at least two parts of the eye wherein the parts have at least one dimension smaller than a dimension X, Y of the eye;adapting the optical sensor for imaging each of the parts of the eye, wherein said optical sensor having sensing units corresponding to at least two different points or areas of said eye, wherein each point or area of said at least two different points or areas corresponds to a different part of said at least two different parts;

[0038] determining the positions of the light pattern projected on each part of the eye at any time of scanning on basis of:

[0039] o a determined position of the projection vector direction of the light pattern on the eye at the time of scanning and / or;

[0040] o a measurement of light pattern positions by a sensing unit of the optical sensor at the time of scanning.

[0041] The method preferably further comprises the step of:

[0042] - configuring the optics to image the eye from at least 2 different viewpoints simultaneously onto said optical sensor, such that each sensing unit of said optical sensor corresponds to at least two different points on said eye, and such that each light pattern on said eye creates at least two light patterns onto said optical sensor,

[0043] wherein said method further comprises the step of determining the distance between the optical sensor to the light pattern on the eye by determining the distance in sensing units between the at least two light patterns projected onto the optical sensor.

[0044] In a second aspect, the present invention relates to a system for eye sensing, comprising:

[0045] an optical sensor comprising a plurality of sensing units,

[0046] a projector adapted to project a light pattern onto an eye,

[0047] optics adapted to image light reflected by the eye and received on sensing units of said optical sensor, wherein the light received on said optical sensor includes a reflected light pattern,

[0048] wherein the system comprises a controller adapted to perform the method as claimed in the first aspect..

[0049] Preferably, the optical sensor has a refresh rate at least 10 times higher than a saccade amplitude, preferably a refresh rate higher than 1000 per second, more preferably higher than 10000 per second.

[0050] In a third aspect, the present invention relates to a wearable device, for example on the face, comprising the system according to the second aspect, and / or operating according to the method of the first aspect.

[0051] The above and other characteristics, features and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the invention. This description is given for the sake of example only, without limiting the scope of the invention.DESCRIPTION OF THE FIGURES

[0052] The disclosure will be further illustrated by means of the following description and the appended figures.

[0053] Figure 1 shows schematically an eye sensing system (100), comprising an optical sensor (6) and a projector (3), according to embodiments of the present invention.

[0054] Figure 2 shows a simplified photon counting for different sensing units (7), according to embodiments of the present invention.

[0055] Figure 3 shows a light pattern scanning the eye (1), according to embodiments of the present invention.

[0056] Figure 4 shows a laser line pattern (10) scanning the eye (1), according to embodiments of the present invention. Figure 5 shows an example of the discontinuity, according to embodiment of the present invention.

[0057] Any reference signs in the claims shall not be construed as limiting the scope. In the different drawings, the same reference signs refer to the same or analogous elements.

[0058] DETAILED DESCRIPTION

[0059] The present invention relates to a method and system for eye sensing.

[0060] In a first aspect, the present invention relates to a method for sensing an eye.

[0061] As used herein, "sensing" an eye may refer to detecting, measuring, and / or determining one or more properties or features of the eye, including but not limited to: pupil position, shape, or size, eye movement (e.g., saccades or fixations), gaze direction or line of sight, eye orientation or pose in three-dimensional space, or biometric or physiological characteristics of the eye (e.g., iris patterns, Purkinje reflections, or other optical reflections). The term also encompasses sensing processes used for gaze tracking, user identification, or interaction with eye-responsive systems.

[0062] The method comprises the step of projecting a light pattern onto said eye, by means of a projector (e.g. a light source, or a laser source or others). Said light pattern may be a light beam projecting a dot-like light pattern on the eye, or may be linear light pattern (e.g. a laser line), which is creating a line pattern (e.g. a vertical or horizontal line, or otherwise). Other structure light patterns are envisaged, as long as different parts of the eye are illuminated on the eye, and the reflection from said different parts of the eye is obtained.The method comprises the step of imaging the reflected light from the eye, by means of optics, on sensing units of an optical sensor. The reflected light has a pattern on said optical sensor. The optical sensor comprises a plurality of sensing units, preferably arranged in a matrix configuration (e.g. rows and columns). In some embodiments, the invention can also work if the plurality of sensing units is arranged in a line, as is understood from the below description.

[0063] Imaging is done, for example, by means of the appropriate optics, imaging the reflected light from the eye. For example, the light is reflected, after hitting the eye, imaged by said optics, and finally received and detected by the optical sensor. In preferred embodiments intended for wearable applications, such as augmented reality (AR) or virtual reality (VR) glasses, the eye sensing system is integrated with a refractive or diffractive optical element, such as a prescription lens or a waveguide display. The projector and optical event-based sensor may be arranged such that the projected sparse light pattern passes through this optical element to the eye, and the reflected light travels back through the optical element before being imaged on the sensing units. The components, including the projector and the optical sensor, may be positioned on a surface of the optical element, coupled to a frame supporting the optical element, or embedded directly within a stacked lens structure. This ensures a compact and robust integration suitable for a wearable device.

[0064] The method further comprises the step of receiving the reflected light by said optical sensor. For example, the method further comprises the step of determining locations of detection of said reflected light on said optical sensor. For example, if the eye is broken down into a plurality of points, then each point or group of points corresponds to a sensing unit or group of sensing units. For example, each reflection by a point in the eye corresponds to a sensing unit. In other words, the optics ensure the reflected light reaches the corresponding sensing unit. That being said, it is not necessary to project the light pattern on all points or parts of the eye, but it is sufficient to sparsely project the light pattern on the eye, to get reflections from different parts of the eye, which still allows to differentiate the different parts of the eye, for example the pupil, which reflects very low or no light. The sparse projection is advantageous in allowing fast and power efficient operation. In other words, at every time step, the number of pixels which detect a reflection event (a detection of reflected light from the eye) is much lower than the total number of pixels in the optical sensor i.e. there is a relatively small number of events detected at each time step i.e. instead of flood illumination as done in the prior art, sparse illumination is done. In this invention, it is not necessary to get information of the whole frame at each time step, but to only get the coordinate relating to where the detection is.

[0065] The method further comprises the step of defining an area of the eye, wherein said area is delineated by a discontinuity in the reflected light pattern, or edge points of the reflected light pattern. A transition occurs at said discontinuity. This discontinuity is either from a positive detection status to a negative detection status, or vice versa. For example, the reflected light pattern is interrupted or absent after being reflected from a part of the eye, forexample due to the fact that different parts of the eye reflect light differently or not reflect at all. Having knowledge of illuminated light could help in understanding the discontinuity of the reflected light pattern, but this can also be deduced based on the difference between the reflected light pattern out and in said area.

[0066] For example, by looking at the start and stop of traces of the reflected optical signals, it is possible to deduce where the pupil (or other eye parts) is, since it reflects substantially no light back to the optical sensor. For example, knowing the starting and stopping time of receiving reflected optical signal indicates the pupil location, as shown in Fig. 3 (b). For example, by knowing the starting and stopping time of both the illuminated and reflected optical signals, it is possible to indicate the pupil location.

[0067] In one specific implementation, the projected light pattern has a relationship with the reflected light pattern outside of said area. This relationship does not imply a 1-to-l correspondence. For example, inside said area, the reflection may not be as strong or there may not be a reflection from the eye. For example, due to the fact that different parts of the eye reflect light differently. In another specific implementation, the projected light pattern does not result in a reflected light pattern on the optical sensor inside said area. In another specific implementation, the projected light pattern results in a reflected light pattern having different reflection properties inside said area compared to outside of said area. In another specific implementation, the projection light pattern has the same reflection properties inside and outside of the area, wherein only at the discontinuity, the reflection properties are different.

[0068] Alternatively, another mode of operation may be envisaged. This alternative mode of operation is based on the number of detections or number of photons received by the sensing units of the optical sensor. For example, in one embodiment, the invention relates to measuring reflections from the eye and determining based thereon parts of the eye. For example, classifying into a first area of the eye, points on the eye corresponding to locations of detections with a number of detections below a threshold number of detections within a predetermined detection area of the optical sensor, and classifying into a second area of the eye, points on the eye corresponding locations of detections with a number of detections above said threshold number of detections within said predetermined detection area of the optical sensor.

[0069] Using either mode of operations, it is possible to determine parts of the eye, more specifically the pupil, and its position and orientation, as discussed further below. Not only the pupil can be distinguished, but also other parts, since different parts reflect light differently. For example, based on the number of detections or photons received, it is possible to distinguish the pupil, sclera, iris, eye lid, from each other. By possibly combining this with depth detection of the eye e.g. using triangulation as described below, it is possible to determine the gaze direction of the eye, given the elliptical structure of the eye.It is noteworthy that the invention can be carried out by different types of light sources, patterns, and beams, for example converging as well as diverging beams.

[0070] The invention provides a simple system with low processing requirements, that is power efficient. This is because there is no need for complex computer vision or heavy image processing, since parts of the eye, more specifically the pupil of the eye, can be tracked by simple light illumination and reception, and further by analyzing the detections (e.g. number of detections) obtained at each sensing unit on the optical sensor. For example, a location on the optical sensor having no (or very few) detections (e.g. very few photons) received, likely corresponds to the pupil, since the pupil reflects very little to no light. This is with the exception to Purkinje reflections, however the optical sensor and projector can be oriented with respect to each other in order to allow or disallow detection of Purkinje reflections, as explained below. Even with the presence of Purkinje reflection, it is possible to determine the pupil location, as Purkinje reflections are strong reflections resulting from a small area in the pupil.

[0071] Preferably, the light pattern is projected for at least one time window. The light pattern covers less than 10% of the eye in said time window. In other words, this projection is a sparse projection, unlike flood projection or illumination in the prior art. In other words, the eye is only sparsely illuminated by light pattern which covers only a small part of the eye, less than 10%, preferably less than 5%, more preferably less than 2% or 1%. Due to this, the amount of data to be received, collected, and processed, i.e. the reflection data, is minimal, and therefore a power efficient method is obtained. For example, this is advantageous if the sensing unit only outputs a signal in case of a positive detection status, or in other words, sensing units which do not detect light do not output any signal, and therefore, it is only needed to read a small number of signals.

[0072] Preferably, the method further comprises the step of scanning said light pattern on said eye along reconfigurable positions over consecutive time step. In other words, the scanning may be done along a predetermined trajectory on said eye, but said trajectory may be reconfigurable / programmable. The scanning may be done by means of a scanner or scanning means. For example, scanning said line or dot on the eye sequentially. Other dynamic structure light patterns are envisaged, as long as different parts of the eye are illuminated and scanned on the eye, and the reflection from said different parts of the eye is obtained.

[0073] In a more specific example, the reconfiguration positions may be a reconfigurable trajectory of the scanning along the eye.

[0074] The light pattern is different on said eye for any two consecutive time steps. This means that the pattern is moving in time, such that it is at a different position at each time step, such that reflections from the eye can be obtained for different parts of the eye. It is desirable to scan the eye sparsely, for example to cover less than 10% of the eyewithin a predetermined number of time windows, such that the number of outputs from the optical sensor remain small, as explained above.

[0075] The scanning has two purposes. Firstly, the scanning is used for finding the position of the eye. For example, the field of view of the sensor covers a much bigger area than the eye, and therefore it is needed to find the eye box first. This may not need to be repeated once the eye box is detected. However, in case the relative position between the eye and the optical sensor has changed e.g. due to slippage of the wearable device which has the optical sensor, then it may need to be done again. Secondly, after the eye box has been detected, the scanning is used for differentiating the different parts of the eye.

[0076] For example, if the eye location changes, e.g. due to slippage of the wearable device, then the trajectory may be reconfigured based on this information.

[0077] Scanning may be ID or preferably 2D scanning. For example, scanning in a Lissajous fashion or raster scan, or the like, for example a repetitive scanning pattern, or any other suitable light pattern that allows scanning big areas of the eye and to get the reflection. The scanning means may for example be a MEMS scanner or a mirror (e.g. rotatable), or an optical phase array, or meta surface approach beam scanning, or the like. If scanning is in ID, then the optical sensor may also comprise a plurality of linearly arranged sensing units (i.e. along ID).

[0078] Instead of having scanning means such as MEMS or the like, it is possible to have a different kind of scanning. For example, having a plurality of light sources, and switching them ON and OFF on the eye, to produce a similar effect to scanning using MEMS. Another example is mechanically moving the light source to scan the light pattern. It is to be understood that scanning does not necessarily require a scanner.

[0079] Preferably, the method further comprises the step of determining properties of the eye, wherein said properties comprise the pose or gaze of the eye. By knowing the location of different parts of the eye, it is possible to deduce the pose or gaze of the eye i.e. where the eye is looking at, for example using triangulation as explained below to understand the 3D position of points in the eye, and therefore the pose of the eye.

[0080] Preferably, the first area corresponds to at least one of the pupil and the limbus of the eye (or both). For example, since the pupil of the eye produces no (or little) reflections (except the Purkinje reflections as discussed below), it is possible to distinguish between the pupil of the eye and other parts of the eye e.g. the iris. However, the invention is not limited to pupil detection, as for example, it is possible to use the teaching to determine the eye limbus. For example, since the limbus does not reflect light in a similar way to the iris and the sclera, for example does not reflect any light, it is possible to distinguish it from the iris and sclera. For example, if the pattern is a line or group of lines,then the reflected light will be interrupted for the sensing units corresponding to the limbus and pupil. The invention is therefore generalized to differentiating different eyes parts based on the differences in the reflections thereof.

[0081] Preferably, the method further comprises the step of detecting contours or delineation of the pupil, which is needed for eye gaze estimation, as discussed herein.

[0082] Preferably, the method further comprises the step of determining properties of the eye, wherein said properties of the eye comprise at least one of: 1) an instantaneous position of the pupil, 2) an instantaneous orientation of the pupil, and 3) an instantaneous dimension of the pupil. Other properties of the eye can also be determined, such as the gaze direction or line of sight, and the eye orientation or pose. The position of the pupil may be 2D or 3D, preferably 3D as explained below. For example, the absolute position with respect to a coordinate system.

[0083] Regarding 1 and 3) by finding the delineation of the pupil as described above, it is possible to always find the instantaneous position and dimension of the pupil. For example, the pupil may move, but may also dilate, such that the delineation of the pupil changes. Regarding 2) the instantaneous orientation of the pupil can be found by different methods. For example, by finding the depth of the eye with respect to the optical sensor at different points (as described below, for example by triangulation, but can also be done by time-of-flight measurement), the orientation of the eye and therefore the eye gaze can be found.

[0084] Preferably, the method further comprises the step of constructing photon statistics for photons received by said optical sensor, said statistics comprising the number of photons received by each sensing units of said optical sensor. This is advantageous in obtaining information about positions of different parts of the eye. For example, a "grey scale" image is obtained, as explained below, by monitoring statistics of detection while laser pattern is scanning the eye. For example, the method comprises the step of counting the photons received on each sensing unit, and further comprises the step of establishing photon statistics on said optical sensor, and determining, based on said photon statistics, a grey scale image or representation of the eye, which differentiates the different parts of the eye. For example, the statistics allows one to understand which parts of the optical sensor had the highest photon count, and which have the lowest photon count, for example, in a given time interval. For example, the method comprises the step of differentiating between the pupil and at least one of the following eye parts, based on said photon statistics: Sclera, Iris, Limbus, and eye lid. Since different parts of the eye reflect light differently, for example the pupil does not reflect too much light, and the iris reflects more light than the pupil, but less light than the sclera, it is possible to distinguish between these parts. One advantage of this is identification, as explained below. Photon counting is also advantageous in allowing operation in case the reflected signal is weak, for example due to the presence of many particles which e.g. scatters, reflects, absorbs the light.For example, the method comprises the step of constructing a hit map, aggregated over multiple windows of detection, whereby the number of detections per pixel are aggregated, more specifically summed. The resulting per pixel hit statistics gives a kind of intensity map indicating how strong the reflection is in that location.

[0085] Preferably, the method comprises the step of filtering noisy detections on said sensing units, based on said photon statistics. This provides accurate and reliable sensing of the eye.

[0086] Preferably, the method further comprises the step of identifying a person based on said classification of said eye. Identification of a person can be done by using photon statistics from different parts of the eye and pupil, and more specifically properties of the iris of the eye, given that the resolution of the optical sensor is suitable. Since the iris pattern is different for different people, and therefore photon statistics for light reflected from the iris is also distinct for each person. Identification is therefore done without needing complex image processing or the need for colored imaging. In this case, a "grey scale" image is sufficient for identification i.e. an image wherein more photons are received at certain parts of the eye, and less photons are received at other parts of the eye, and no or very few photons are received at other parts of the eye. Although referred to as an image, there is no need to get photons from all parts of the eye; getting information from only parts of the eye, by a scan of a limited time, is sufficient. Furthermore, the identification of a person based on the determined reflectivity profile of the iris can be implemented as a robust biometric security feature. The unique pattern of the iris serves as a biometric template that can be used for user authentication on a device. This may be used as a primary authentication mode or as part of a multi-factor authentication process. Such a system is also capable of providing anti-spoofing protection by verifying that the detected iris pattern matches a pre-enrolled template of the authentic user in real-time, thus ensuring the user is genuine and physically present. Such a configuration can be put for example on a wearable device.

[0087] Preferably, the projector and optical sensor are oriented (and / or reoriented, at later time instances) with respect to each other, such that a first Purkinje reflection and / or a fourth Purkinje reflection is received by said optical sensor. Detecting Purkinje reflections allows detecting the pupil in a more accurate way. For example, detecting the Purkinje reflections may allow estimating the pupil position and orientation in a rough way, or to allow verifying the accuracy of the detected pupil delineation. The invention in its broadest sense does not require detecting such reflections, and depending on the way of orienting the optical sensor and the optical sensor with respect to each other, it is possible to allow or disallow detection of said Purkinje reflections.

[0088] Preferably, the method comprises the step of distinguishing said first Purkinje reflection from other reflections from the eye. This can be done based on the location of the first Purkinje reflection i.e. originating from the pupil. However, an alternative preferred embodiment is that it is done based on photon statistics. This is because the first Purkinje reflection is much stronger than other reflections from the eye, therefore the number of photons is muchhigher. Alternatively or additionally, the method may also distinguish the first Purkinje reflection from the fourth Purkinje reflection, given that the two do not coincide. For example, the projector and optical sensor are oriented such that the first and fourth Purkinje reflections are received on different sensing units of the optical sensor, thereby being distinguishable since the first Purkinje reflection is much stronger than the fourth Purkinje reflection, and therefore the number of photons reflected is much higher.

[0089] In case of mounting this system on a wearable device, and in order to improve the signal-to-noise ratio and the accuracy of detecting the discontinuity, the system may further comprise a polarization control unit. This unit is configured to modify the polarization state of the sparse light pattern projected by the projector. By actively controlling the polarization, the system can be adapted to account for the optical properties of any intervening lenses or display elements in a wearable device. This allows for the maximization of desired reflections from the eye surface and the minimization of internal reflections and other sources of noise, thereby resulting in a more distinct and reliably detectable discontinuity in the reflected light pattern.

[0090] Preferably, the light pattern is a pulsed light pattern. For example, a laser pulse. This is advantageous in distinguishing the noise from the active light, since the power is concentrated in one pulse i.e. each pulse has a higher power while maintaining the same average power, thereby allowing easier detection and filtering. In other words, the reflected light can be distinguished from the noise, since the power of the pulses are higher than the noise, and since the pulse can be expected within a certain time window.

[0091] Also, using a pulsed light pattern is advantageous when using persistence or neighborhood conditions (explained below) for better filtering, as it allows to check said conditions based on the pulse length, instead of based on observation windows as in the case of a continuous pulse, which is typically longer than the pulse length. For example, the pulse width is at least 1 nanosecond, preferably at least 10 nanoseconds.

[0092] Preferably, the light pattern is at least a point, curve, or line. For example, at one point in time, a point, or a line is being scanned on the eye.

[0093] Preferably, the method further comprises the step of triangulating locations of detections of said light pattern, with the corresponding instantaneous projection vector direction of the projector. This allows to obtain 3D information i.e. the depth value corresponding to said light pattern. This is useful since the eye has an elliptical shape, hence having 3D information of the eye allows to obtain the eye gaze or pose or orientation. Although triangulation is preferred, other methods can be used to determine the depth information at each point in the light pattern. For example, by time-of-flight measurement based on said light pattern emitted from said projector and received by said optical sensor. Triangulation is also advantageous in obtaining a robust solution against slippage. For example, if the optical sensor, being mounted on a wearable device, slips away from the eye, then it is possible, using triangulation, to account for this slippage. This is also useful in calibration, since different users may have different eye bone structure, and therefore it would not be needed to calibrate the system for each user independently, sincetriangulation is able to find the depth and take this into account. It is only needed to perform a rough triangulation e.g. no need to find triangulations for all points on the eye, but only on a handful points, to be able to account for the abovementioned issues. In other words, triangulation allows to find the relative position between the eye and the optical sensor.

[0094] The triangulation can also be obtained between two optical sensors, in case a second optical sensor is available. For example, triangulation may be done by triangulating detection points of a first optical sensor with instantaneous projection vector direction of said a light beam or pattern, or with detection points of a second optical sensor (i.e. looking at the eye).

[0095] It is advantageous to perform triangulation between the optical sensor and the projector, as this eliminates the need for a second optical sensor, hence reducing power consumption and shrinking the required footprint.

[0096] Triangulation also allows obtaining information about saccades and vergence of the eye.

[0097] As a further alternative, depth information may be determined by modulating the phase of the projected light. In such an embodiment, the system may include a spatial light modulator positioned in the path of the projected sparse light pattern. The spatial light modulator modifies the phase of the light pattern before it illuminates the eye. By receiving the reflected light at the optical sensor and analyzing its resulting phase state, for instance using phaseshifting algorithms, the system can determine the 3D depth information of the reflection points on the eye. This 3D information can subsequently be used to determine the eye gaze.

[0098] Preferably, at least some of said triangulated locations of detections correspond to the points on said transition. For example, the first area is defined by a curve, wherein at least the points on said curve are triangulated. This allows to get a position of the pupil in 3D for estimating the eye gaze or pose. For example, the depth is found for at least some points corresponding to the curve or delineation defining the pupil.

[0099] Preferably, the method comprises the step of filtering the detection determined by one sensing unit of said optical sensor, based on the presence of a detection over at least one neighboring sensing unit to said one sensing unit at one time step, and / or based on the presence of said detection over at least two consecutive time steps. The optical sensor is arranged in a plurality of rows and columns of sensing units. This allows reliable detection and filtering. For example, a noise signal would neither, in most cases, appear over many neighboring pixels, nor appear on the same pixel over multiple consecutive time steps. Filtering is particularly advantageous for this invention as it makes it easier to distinguish the active light from the noise. For example, the method comprises the step of distinguishing (e.g. by filtering) the active signal (e.g. the light spot on the eye) from the background noise, on the optical sensor.Furthermore, since the light pattern has a known shape e.g. dot-shaped or a semi-dot or disk-shaped or a collection of dots or any other suitable shape, the reflection and detection by the sensing units, and the number of sensing units / shape of detection, can be anticipated, and therefore easier filtering is obtained.

[0100] For example, the filtering is based on temporal and / or spatial conditions, for example based on neighboring sensing units or based on persisting detections in one sensing unit over multiple (e.g. at least two) time steps, or a combination thereof. This is also advantageous in improving error tolerance. For example, the filter exhibits either a spatial filtering method (i.e. at least two neighbouring pixels have a positive detection simultaneously), a temporal filtering method (i.e. pixels having a repeated positive detection status when receiving repeated pulses e.g. over multiple time windows), or a combination thereof.

[0101] Arranging the optical sensor as a plurality of rows and columns is advantageous in allowing the detected signal to be filtered, for example using temporal and / or neighborhood information, for example as described in PCT IB2021 054688, PCT EP2021087594, PCT IB 2022000323, and PCT IB2022058609. For example, checking the detection of the neighboring sensing units, since the detections are likely to be detected over more than one pixel, as described in PCT EP2021 087594. Alternatively or additionally, checking the persistence of the detections over time as described in PCT IB2022058609. Therefore, the detections which are persistent over a long period of time are likely true detections, and the detections otherwise are likely false detections. Alternatively or additionally, each detector is connected with one column bus and one row bus, as described in PCT / IB2021 / 054688, to prevent reading the output pixel by pixel, which is advantageous in reducing the noise and also making the system faster. Other filtering mechanisms may be envisaged, as described in PCT IB2021054688, PCT EP2021 087594, PCT IB2022 000323, and PCT IB2022 058609. For example, the system is adapted to filter the location of detection, determined by said plurality of sensing units. For example, reflected light is segregated from ambient light and noise. This may be based on the presence of said detection over one or two or more neighboring or semi-neighboring sensing units, for example by considering only pixels in the true status having at least one neighboring pixel also in the true status. This may additionally or alternatively be based on the presence of said detection over at least two consecutive time steps. For example, considering only pixels in the true status having at least one neighboring pixel also in the true status in the plurality of pixels obtained in one observation or a combination of the at least two consecutive or semi-consecutive observation windows.

[0102] Preferably, the method comprises the step of activating sensing units in a first region of said optical sensor where a detection of the reflected light by said optical sensor is expected, and deactivating sensing units in at least a second region where a detection of the reflected light by said optical sensor is not expected, wherein said activating and deactivating may be based on the instantaneous projection vector direction of the light beam or pattern on the eye, and / or on the locations of detection on the optical sensor. In this case, the light is being scanned along a trajectory. For example, it is known where the beam is shining at each moment, and under which angle, therefore a detection in a spot or area in which the beam is not shining at that moment, is likely a false detection. This is advantageous inY1

[0103] obtaining a power efficient system, since not the full optical sensor is activated, for example not all pixels or sensing units are operating all the time, but only those for which a detection is expected. This is also advantageous in reducing the noise, since only the relevant regions of the optical sensor are operating e.g. only the relevant pixels are operating, so less pixels are operating at one time instance. Therefore, less filtering is needed, and therefore more noise immunity is obtained, as well as less power is consumed on said filtering.

[0104] Activating and deactivating may also be based on the locations of detection on the optical sensor. For example, the projector is scanned in a predictable trajectory, such that the optical sensor can follow the detection without synchronization with the projector.

[0105] Preferably, the method further comprises the step of partially reading said optical sensor. For example, reading less than 10%, or preferably less than 5% of the optical sensor, wherein only the sensing units corresponding to the active light (the light which originates from the light source) are read. Said optical sensor only output data only when an event (i.e. when light corresponding to an active signal, i.e. not noise) is detected. The data correspond to the reflected light pattern. For example, the optical sensor is an event-based sensor. For example, sensors which sensing units or pixels would only fire upon detection of light (e.g. a positive detection status), or upon a change in the detected light. Therefore, an event can be defined as detection of light (a change from '0' to '1'), or as changes in the detection of light (e.g. changes in the light intensity levels, or a change from '1' to '0'). In other words, the pixels of the optical sensor do not need to be read in full, but only the pixels which indicate an event, are read. For example, the coordinates in which the light is received are provided to the reading circuit. This allows obtaining a power efficient optical sensor, since it is only required to read the pixels which detected light, and not all pixels of the optical sensor. For example, there is no need to read all sensing units as it is only needed to get the coordinates where there is an optical signal or event. This is different than frame-based imaging, where it is needed to read all pixels in the optical sensor, in which computer vision and heavy signal processing is needed. Instead, only signal processing of event streams is needed.

[0106] In other words, the sensor will not detect a 2D image at each time step but will detect only a group of points which correspond to the light pattern at each time step. This allows fast and low latency operation, is more power efficient, and reduces the amount of data that needs to be processed.

[0107] Here, it is important to differentiate between the light coming from the active signal (from the light source), and the noise). Of course, there will be noisy detections, however these will be filtered at the sensor level and before sending it to the central processing unit.

[0108] In other words, the detections are first filtered at the sensor level, then they are read by the central processing unit, hence only the sensing units corresponding to a detection of active light (and not noise) are read. This way it is possible to obtain a low latency and power efficient system.

[0109] Another example which can be combined with the previous examples, is that a zone of interest is defined, based on knowledge of the light source, and only sensors in that zone of interest are read.Preferably, for consecutive time steps, the detections by the optical sensor correspond to the light pattern. For example, if the light pattern moves, so does the detections, and this happens for consecutive time steps.

[0110] Preferably, the light pattern is dynamic and not statically created. For example, the light pattern changes with time steps.

[0111] Preferably, each sensing unit comprises a single photon detector, for example one single photon detector. This allows photon counting as described above. The SPDs are preferably a single photon avalanche diode (SPAD). The invention can also be generalized to photo detectors that are not single photon detectors.

[0112] Preferably, the method comprises the step of determining a first set of areas on the optical sensor in which the number of detections is below a threshold number of detections within predetermined time steps, and determining different a second set of areas on the optical sensor in which the number of detections is above said threshold number of detections within said predetermined time steps. Based on this, the method is able to determine a corresponding area on the eye to each of said first and second set of parts, and therefore differentiate between at least two areas of the eye, for example the pupil and the rest of the eye.

[0113] Preferably, the wavelength of the projector light is chosen such that the reflection from different parts of the eye is different and therefore the eye parts can be distinguished from one another, most specifically that the pupil reflects light very differently than the iris and sclera, for example little to no light is reflected by the pupil, except if designed to reflect Purkinje reflections.

[0114] Preferably, the projector and optical sensor are synchronized. For example, this can be done with a direct link synchronization i.e. the projector and sensor are connected by a link which allows the sensor to know the instantaneous projection vector direction. Alternatively, this can be done indirectly, by knowing the instantaneous projection vector direction in advance. Synchronization is advantageous for triangulation with one sensor and one projector as explained above, since triangulation requires information of the location of detection on the optical sensor as well as instantaneous projection vector direction. It is noteworthy, however, that there are other known methods other than triangulation for obtaining the depth. For example, a neural network may be used, instead of triangulation, to obtain the depth based on the instantaneous projection vector direction and the location of detection on the optical sensor.

[0115] For example, the method comprises the step of synchronizing the projector and optical sensor, and the step of measuring the depth (by triangulation or other methods) at different parts of the eye, such that eye-related information e.g. gaze information, are obtained.In one embodiment, synchronization may mean that information from the projector on the instantaneous position of the instantaneous projection vector direction is continuously being sent to the sensor. Alternatively or additionally to knowing the instantaneous projection vector direction at each time step, a preferred embodiment is configuring the scanning of said light beam to start from a predetermined region on said scene, which provides a possible starting point of operation for the optical sensor and the projector i.e. allows indirect synchronization between the optical sensor and the projector i.e. synchronization without having a link between the optical sensor and the projector. For example, the projector starts operating at a point in the predetermined region on which the projector starts scanning, while the optical sensor starts operating at the corresponding pixel or pixels. For example, the projector starts operating at the zero-crossing of the vertical axis of the projector, and the optical sensor starts operating at the corresponding zero-crossing of the vertical axis of the optical sensor. The zero-crossing of the vertical axis is defined as the point on the axis at which the value is zero. In this case, the vertical zero-crossing is relevant because there no parallax in that direction, therefore it is possible to correspond the zero-crossing of the projector to the optical sensor, and vice versa, and therefore allow the projector and optical sensor to start operating at the predetermined region.

[0116] In other words, by knowing the time step at which the scanning starts, and its instantaneous position, and the overall trajectory, it is possible to indirectly synchronize the projector to the optical sensor, without having a direct synchronization link between the optical sensor and projector.

[0117] However, in some cases, a direct synchronization link may still be preferred, for example to compensate for frequency mismatch. Alternatively, the projector provides a signal which is fed to a controller controlling the optical sensor, which signal allows the optical sensor to know when the operation will start, after which the predefined trajectory is known to the optical sensor. The controller may be an external controller or a controller within the optical sensor (e.g. logic).

[0118] In one implementation, the invention may be carried out using one sensing unit ora few rows and columns of sensing units, instead of a plurality of sensing units. For example, based on knowledge of the instantaneous projection vector direction, it is possible to create a 2D map of the reflection of different points in the eye. For example, if the laser pattern is incident on a point or points in the eye which reflects light, then the detection will be positive. This is an advantageous implementation for having a small footprint system. Although a few sensing units are more advantageous in detecting said discontinuity, having one sensing unit would also allow to detect said discontinuity.

[0119] Preferably, the eye is being defined at least by two orthogonal dimensions X and Y, where in the method further comprises the steps of: defining at least two parts of the eye wherein the parts have at least one dimension smaller than a dimension X, Y of the eye. The method further comprises the step of adapting the optical sensor for imaging each of the parts of the eye, wherein said optical sensor having sensing units corresponding to at least two different points or areas of said eye, wherein each point or area of said at least two different points or areas corresponds toa different part of said at least two different parts. The method further comprises the step of determining the positions of the light pattern projected on each part of the eye at any time of scanning on basis of:

[0120] o a determined position of the projection vector direction of the light pattern on the eye at the time of scanning and / or;

[0121] o a measurement of light pattern positions by a sensing unit of the optical sensor at the time of scanning. An advantage of this is that a compact optical sensor and optical sensing system can be obtained, since each sensing unit is used to detect reflected light from different parts in the scene, thereby requiring fewer sensing units and smaller optical sensor. Advantageously, the size of the optical sensor and the optical sensing system can be reduced of at least 20% or at least 30% or at least 40% or at least 50% compared to prior art devices since different parts of the total field of view of the optical sensor are concurrently imaged on the same optical sensor. These different parts may be substantially different with little or no overlap, or may share substantially parts of their individual field of views. In other words, this allows using less pixels but still getting the same result. Also, since the optical sensor can be smaller and can have a smaller number of sensing units, in comparison to a system in which each sensing unit corresponds to one point in the scene, the number of stand-by sensing units is reduced which consume less power. Advantageously, the noise is significantly reduced, since the number of sensing units operating at one time instance are less due to shrinking the size of the optical sensor. Therefore, less filtering is needed, and therefore more noise immunity is obtained, as well as less power is consumed on said filtering. It is also advantageous that the latency is reduced since less processing is needed. An additional advantage is that per part of the eye, the angular resolution is improved. In other words, the field of view per pixel is reduced and thus the angular resolution per pixel is improved. This is done by overlaying and imaging the separate parts of the scene by the same optical sensing area e.g. imaging at least two different points on one sensing unit. Advantageously, the angular resolution of the optical sensor can be significantly increased, without causing a huge increase to the power consumption of the sensor or the increase in area of the sensor.

[0122] Preferably, the method further comprises the step of configuring the optics to image the eye from at least 2 different viewpoints simultaneously onto said optical sensor, such that each sensing unit of said optical sensor corresponds to at least two different points on said eye, and such that each light pattern on said eye creates at least two light patterns onto said optical sensor. The method further comprises the step of determining the distance between the optical sensor to the light pattern on the eye by determining the distance in sensing units between the at least two light patterns projected onto the optical sensor. For example, two imaging optics are organized such that there is a parallax between the viewpoints of said imaging optics, whereby said imaging optics project an image of the eye on the same optical sensor. In this way, the disparity or distance in the image plane between the projected light pattern onto the optical sensor is a measurement of the distance to the light pattern on the eye. In this way a compact stereoscopic system is created using a single optical sensor. As a result, all required information to compute a depth measurement is created by the same optical sensor and a depth reconstruction computational pipeline can beimplemented which does not necessarily rely on external information (which for example comes from a separate sensor in typical stereoscopic system).

[0123] In a second aspect, the present invention relates to an optical sensing system for eye sensing. The system comprises an optical sensor, comprising a plurality of sensing units, preferably wherein each sensing unit comprises a single photon detector. The system further comprises a projector adapted to project a light pattern onto an eye.

[0124] The system further comprises optics adapted to image light reflected by the eye and received on sensing units of said optical sensor, i.e. the optics is able to make an image of said eye on sensing units of said optical sensor, as described in the first aspect. The light received on said optical sensor includes a reflected light pattern. The system further comprises a controller adapted to perform the method as claimed in the first aspect.

[0125] For example, based on the location of the light pattern imaged on said optical sensor, the depth information of the eye can be calculated by triangulation, as described in different parts of the document, or by different methods for measuring depth known in the art. For example, the reflected light from the eye is received by at least one sensing unit, after which the optical pulse is translated into an electrical signal or logic pulse. Each pulse is timestamped with respect to a reference clock.

[0126] For example, the system is adapted to determine, based on said reflected light, locations of detection of points in the eye, wherein each point corresponds to a location of detection on said optical sensor. That being said, it is not necessary to illuminate the complete eye, as understood from different parts of the disclosure herein.

[0127] The system is adapted to define an area of the eye. Said area is delineated by a discontinuity in the reflected light pattern, wherein a transition occurs at said discontinuity. Said discontinuity is from a positive detection status to a negative detection status, or from a negative detection status to a positive detection status. The projected light pattern corresponds to the reflected light pattern out of said area, as described in the first aspect.

[0128] Preferably, the system is adapted to construct, based on said locations of detection, a depth map of said eye, as explained in the first aspect. The depth map is however preferably only a partial depth map, as the object is not to construct a 3D model of the eye, but to find the relative distance between the eye and e.g. a wearable device carrying the optical sensor, or to find the orientation of the pupil, or others.

[0129] Preferably, the system further comprises scanning means (e.g. a scanner) adapted to scan the light pattern on the eye along reconfigurable positions over consecutive time steps. The light pattern is different on said eye for any two consecutive time steps, as described in the first aspect.Preferably, the system is adapted to obtain said at least a partial depth map by triangulation between the optical sensor and the projector, and / or by time-of-flight measurement based on said light beam emitted from said projector and received by said optical sensor, as described in the first aspect.

[0130] Preferably, each of said photodetectors is a single photon detector, preferably a single photon avalanche detector SPADs. Alternatively, said photo detector is an avalanche photo detector. Single photon detectors are advantageous since they are fast by nature, such that the position of the light pattern on the eye is detected in a fast manner and accurate manner. Single photon detectors also minimize the energy needed to be able to detect said pattern. SPADs are also sensitive to a single photon, which means that the active projected structure will need a minimum amount of energy. Another advantage is sub-nanosecond response time of detectors such as SPADs, meaning the photon is detected and encoded into a digital signal in nanoseconds.

[0131] For example, the detector is adapted to output a logic signal e.g. an electrical detection signal upon detection of a photon. For example, a detection signal may be represented by a signal comprising logic '1' e.g. a detection, while no detection signal may be represented by a signal comprising logic '0' e.g. no detection. Alternatively, a detection signal may be represented by or result in a pulse signal, e.g. a transition from logic '0' to logic '1', then a transition back from logic '1' to logic 'O', while no detection may be represented by (or result in) an absence of such a pulse signal. Preferably, each photo detector is arranged in a reverse biased configuration.

[0132] Any feature of the second aspect (system) may be as correspondingly described in the first aspect (method). For example, the system of the second aspect is operating based on (or similarly to) the method of the first aspect.

[0133] In a third aspect, the present invention relates to use of the method of the first aspect and / or the system of the second aspect, for detecting or predicting diseases, such as Parkinson diseases. Another use can be related to identifying certain health conditions based on pupil dilation. Another use can be identifying focus levels of the user, for example by identifying saccades or vergence of the eye, which can help in many applications such as wearable devices. Another use is determining the gaze of the eye.

[0134] In a fourth aspect, the present invention relates to a wearable device comprising the optical sensor, system, method, and use according to the first, second, and third aspects.

[0135] In a broader embodiment of the first aspect, at least one (or more) of the following features may be optional: the sparse light pattern, the event-based sensor, and the discontinuity.Further characteristics and advantages of embodiments of the present invention will be described with reference to the figures. It should be noted that the invention is not restricted to the specific embodiments shown in these figures or described in the examples, but is only limited by the claims.

[0136] Fig. 1 schematically represents an eye sensing system (100) according to the invention. In Fig. 1, a projector (3) projects an output (4) on the eye (1). This may be scanned along a predetermined trajectory (not shown), but not necessarily, as explained above. The incident light pattern (4) is then reflected (5) to the optical sensor (6). The optical sensor (6) comprises a plurality of pixels (7) or sensing units arranged in a matrix. Based on the reflected light (5) received by the optical sensor (6), it is possible to determine the location of parts of the eye, for example the pupil of the eye, since the reflections caused by the pupil are much smaller (or none) compared to that caused by other parts of the eye. For example, it is possible to determine at least one of the dilation, position, and orientation of the pupil.

[0137] Fig. 2 shows a simplified photon counting on the optical sensor (6). The information provided by the optical sensor (6) suggests the location, orientation and dilation of the pupil. Photon counting and statistics can provide a better understanding regarding other parts of the eye, for example the iris. In a more specific embodiment, a person's identity can be determined based on the photon counting and statistics in the area where the iris is located.

[0138] In Fig. 2, it is shown that the pupil reflects much less light (if any) compared to other parts of the eye (1). Sensing units (7) having a low photon count (or photon count below a predetermined threshold) are marked by (L), while sensing units having a high photon count (or photon count above a predetermined threshold) are marked by (H). The difference between the two values allows to identify the pupil with great accuracy. Using photon counting, it is possible to overcome the issue of light scattering and also filter out any false positive detections. Other filtering mechanisms, as explained above, can also help in filtering out false positive detections.

[0139] It is assumed that Purkinje reflection(s) would only appear if the orientation of the optical sensor with respect to the projector is such that such reflections can be detected. Even in the presence of Purkinje reflection(s), it is possible to operate the method in the same way as described above.

[0140] Fig. 3 (a) shows a light pattern (4) being scanned along a trajectory on the eye (1), according to one embodiment of the invention. As mentioned above, the light pattern (4) could be created by a light beam generating a light spot e.g. dot-like pattern being scanned across the eye (1) e.g. a laser dot. For example, scanned in a Lissajous or raster scan, or others. However, the pattern can be generalized to any other pattern, for example a line pattern being scanned at different positions of the eye (1) as shown in Fig. 4 (b), or other patterns. The object is to find the reflections fromthe eye (1) to understand the different parts of the eye (1). In the case of scanning a line as in Fig. 4, then this may not be done along a trajectory, but may instead be done for predetermined locations.

[0141] Fig. 3 (b) shows the reflections from the eye (1). Since the reflection from the pupil is significantly different from other parts of the eye (1), it is possible to find the curve (9) or borders defining the pupil, or the so-called delineation, as shown also in Fig. 3 (c). In other words, the delineation is defined by the areas in which the scan pattern produces no reflection that is received on the optical sensor (6).

[0142] Fig. 4 (a, b) shows a laser line (10) being projected at different parts of the eye (1). This can be done in different time steps (to, ti, t2), or in the same time step. The line may be sweeping or scanning across the eye in a stepped fashion, sinusoidal, or others. Other light patterns (i.e. other than dot or line patterns) are envisaged, as long as said patterns produce reflections that can allow one to understand the delineation of the eye (1).

[0143] Fig. 5 shows in (a) a projected light pattern, wherein the pattern is a line. Although the expected reflected light pattern is also a line, due to the curvature of the eye as well as the cornea, the reflected light pattern is as shown in (b). By doing this, we can determine the position of the cornea, which are indicated by reference sign 11. Also, the orientation may be derived based on the reflected light pattern.

[0144] In a broader embodiment, the discontinuity may relate to a difference between expected reflected sparse light pattern and measured reflected sparse light pattern. The difference allows to determine the position of at least some parts of the eye for example the cornea. For example, in case the projected sparse light pattern is a straight line, then the expected may also be a straight line. However, in case the measured reflected light pattern has a curve, which is different than the projected light, then there is a discontinuity. For example, the projected light pattern is a straight line, and the measured sparse light pattern is a first straight line followed by a curvy line followed by a second straight line. In this case, the points where the curvy line starts and end define the position of a distinct part of the eye, for example the cornea. Furthermore, the reflected light pattern allows to determine the orientation of the eye, since the reflected light pattern would be different when the eye is looking at different directions. In this case, the invention may be defined as:

[0145] a method for sensing an eye (1), the method comprising the steps of:

[0146] projecting a sparse light pattern (4) onto said eye (1), by means of a projector (3),

[0147] imaging light (5) reflected by the eye and received, by means of optics, on sensing units (7) of an optical event-based sensor (6), wherein the light (5) received on said optical sensor (6) includes a reflected sparse light pattern,

[0148] detecting a difference between expected reflected sparse light pattern and measured reflected sparse light pattern; anddetermining a position of the eye relative to the optical sensor at least partly based on said detected difference.

[0149] Other arrangements for accomplishing the objectives of the methods and devices embodying the invention will be obvious for those skilled in the art. The proceeding description gives details of certain embodiments of the present invention. It will, however, be clear that no matter how detailed the above turns out to be in text, the invention may be applied in many ways. It should be noted that the use of certain terminology when describing certain characteristics or aspects of the invention should not be interpreted as implying that the terminology herein is defined again to be restricted to specific characteristics or aspects of the invention to which this terminology is coupled.LIST OF REFERENCE SIGNS

[0150] 100 System

[0151] I Eye

[0152] 2 First area

[0153] 3 Projector

[0154] 4 Light pattern

[0155] 5 Reflected light

[0156] 6 Optical sensor

[0157] 7 Sensing unit

[0158] 8 Second area

[0159] 9 Pupil delineation or curve defining the pupil or edge points of the scanning light pattern 10 Laser line

[0160] II Points defining the position of the cornea

Claims

CLAIMS1. A method for sensing an eye (1), the method comprising the steps of:projecting a sparse light pattern (4) onto said eye (1), by means of a projector (3),imaging light (5) reflected by the eye and received, by means of optics, on sensing units (7) of an optical sensor, preferably an event-based sensor (6), wherein the light (5) received on said optical sensor (6) includes a reflected sparse light pattern,detecting a discontinuity in the reflected sparse light pattern compared to the projected sparse light pattern;determining a position of the eye relative to the optical sensor at least partly based on said detected discontinuity.

2. The method according to claim 1, wherein the discontinuity is a change in direction of reflected light in the reflected light pattern, or a transition between a detection and a non-detection of reflected light in the reflected light pattern.

3. The method according to any of the previous claims, wherein the reflected sparse light pattern (4) covers less than 10% of a field of view of the optical sensor.

4. The method according to any of the previous claims, wherein the method further comprises the step of:scanning said light pattern (4) on said eye (1) over consecutive time steps.

5. The method according to any of the previous claims, wherein the projector (3) is a plurality of projectors and wherein the step of projecting a sparse light pattern is performed by different projectors of the plurality of projectors over consecutive time steps.

6. A method according to any of the previous claims, wherein the method further comprises the step of determining a diameter of the pupil based on a plurality of discontinuities, in particular of transitions between a detection and a non-detection of reflected light in the reflected light pattern.

7. A method according to any of the previous claims, wherein the method further comprises the step of performing photon statistics for photons received by said optical sensor (6), said statistics comprising the number of photons received by each sensing units (7) of said optical sensor (6), and determining a reflectivity profile of at least portions of said eye based on said photon statistics.

8. The method according to claim 7 , wherein the method further comprises the step of identifying a person based on the determined reflectivity of at least a portion of the eye (1).

9. A method according to any of the previous claims, wherein the projector (3) and optical sensor (6) are oriented with respect to each other, such that a first Purkinje reflection and a fourth Purkinje reflection is received by said optical sensor (6), wherein the method further comprises the step of deriving a change in direction of eye gaze over consecutive time steps.

10. A method according to any of the previous claims, the method further comprises the step of determining a relative distance between the optical sensor and a plurality of points of the eye reflecting said light pattern, preferably based on triangulation, and determining a gaze direction of the eye.

11. A method according to claim 10, wherein at least some of said plurality of points of the eye correspond to said discontinuity in the reflected sparse light pattern.

12. The method according to any of the previous claims 10 - 11, wherein the method further comprises the step of determining an amplitude of a saccade.

13. An optical sensing system (100) for eye sensing, comprising:an optical sensor (6) comprising a plurality of sensing units (7),a projector (3) adapted to project a light pattern (4) onto an eye (1),optics adapted to image light (5) reflected by the eye and received on sensing units (7) of said optical sensor (6), wherein the light (5) received on said optical sensor (6) includes a reflected light pattern, wherein the system (100) comprises a controller adapted to perform the method as claimed in any of the previous claims 1 - 12.

14. The system of claim 13, wherein the optical sensor has a refresh rate at least 10 times higher than a saccade amplitude, preferably a refresh rate higher than 1000 per second, more preferably higher than 10000 per second.

15. A wearable device comprising the system according to any of claims 13 or 14.

Images