Reliability of left and right eye gaze tracking data

Abstract

Circuitry of the gaze / eye tracking system obtains one or more images of the left eye and one or more images of the right eye, determines a gaze direction of the left eye based on the at least one obtained left eye image, determines a gaze direction of the right eye based on the at least one obtained right eye image, and determines a gaze direction of the left eye based on the at least one obtained right eye image. A first confidence value is determined based on the obtained one or more images of the left eye, a second confidence value is determined based on the obtained one or more images of the right eye, and a final gaze direction is determined based at least in part on the first confidence value and the second confidence value. The first confidence value and the second confidence value represent indications of the reliability of the determined gaze directions of the left eye and the right eye, respectively. A corresponding method and computer readable medium are also provided.

Description

[0001] This application is a divisional application of an invention patent application with an international application date of September 11, 2017, an international application number of PCT / US2017 / 050949, a national application number of 201780096602.3, and an invention name of “Reliability of left and right eye gaze tracking data”. Background Art

[0002] The present disclosure relates generally to the field of eye tracking. In particular, the present disclosure relates to systems and methods for generating and / or using gaze tracking data indicating the gaze direction of an eye.

[0003] Several different eye tracking systems are known in the art. For example, such a system may be employed to allow a user to indicate a location on a computer display by looking at that location. The eye tracking system may capture an image of the user's face and then employ image processing to extract key features from the user's face, such as the center of the pupil and the flash from an illuminator illuminating the user's face. The extracted features may then be employed to determine where the user is looking at on the display. Naturally, factors such as accuracy, speed, and reliability / robustness of eye tracking are desired to achieve a positive user experience. Accordingly, several schemes have been proposed to mitigate the negative effects of different types of errors or inaccuracies that may occur in eye tracking systems.

[0004] For example, US2010 / 0328444 A1 (which is incorporated herein by reference in its entirety) discloses an eye tracker comprising at least one illuminator for illuminating an eye, at least two cameras for imaging the eye, and a controller. The illuminator and the cameras are configured so that at least one camera is coaxial with the reference illuminator and at least one camera is not coaxial with the reference illuminator. The controller is adapted to select one of the cameras to be active to maximize the image quality metric and avoid occluding objects. The eye tracker can operate in a dual camera mode to improve accuracy. A method and computer program product are also disclosed for selecting an active reference illuminator from a plurality of reference illuminators and a combination of an active camera from a plurality of cameras.

[0005] As another example, US2014 / 0362446 A1 (which is incorporated herein by reference in its entirety) discloses a head-mounted display (HMD) system. The HMD system includes an eye position detector, a dominant eye detector, and an image generator, the eye position detector including one or more cameras configured to detect the position of each eye of the HMD user, the dominant eye detector configured to detect the dominant eye of the HMD user, and the image generator configured to generate an image for display by the HMD according to the eye position of the HMD user. The image generator is configured to apply a greater weight to the detected position of the dominant eye than to the detected position of the non-dominant eye. The dominant eye is detected by one or more of the following steps: (i) an eye with a wider range of movement, (ii) an eye that is closest to and / or fastest to a gaze direction suitable for a displayed stimulus point, and (iii) an eye that maintains a position associated with a displayed stimulus point.

[0006] As another example, US2016 / 0004303 A1 (incorporated herein by reference in its entirety) discloses an eye gaze tracking system, which includes a gaze data acquisition system, which includes multiple light sources and multiple image sensors. The light sources are arranged to emit light to the user's head, and the image sensors are configured to receive the light. In an embodiment, the system further includes a gaze tracking module, which includes a visual feature extraction module, a point of interest (POR) calculation module, and a POR averaging module. The visual feature extraction module is configured to process the gaze data and extract visual features associated with each image sensor, and is configured to determine a confidence value associated with the accuracy of these parameters of each image sensor. The confidence value may, for example, depend on the head posture angle (yaw, pitch and roll angle), user distance, and feature detection reliability. The POR calculation module is configured to determine the POR based on the visual features for each image sensor. The POR averaging module is configured to use the confidence values ​​of the PORs of the image sensors to determine the average POR.

[0007] However, it would be desirable to provide additional systems and methods that address at least one of the above-mentioned issues. Summary of the invention

[0008] The object of the present disclosure is to solve at least one of the above problems.

[0009] According to a first aspect, an eye tracking system including a circuit is provided. The eye tracking system may also be referred to as a gaze tracking system. The circuit is configured to obtain one or more images of a user's left eye and one or more images of the user's right eye, determine (or calculate) a gaze direction of the user's left eye based on at least one image of the left eye obtained, and determine (or calculate) a gaze direction of the user's right eye based on at least one image of the right eye obtained. The circuit is configured to determine (or calculate) a first confidence value based on one or more obtained images of the left eye. The first confidence value represents an indication of the reliability of the determined gaze direction of the left eye. The circuit is configured to determine (or calculate) a second confidence value based on one or more obtained images of the right eye. The second confidence value represents an indication of the reliability of the determined gaze direction of the right eye. The circuit is configured to determine (or calculate) a final gaze direction based at least in part on the first confidence value and the second confidence value. The final gaze direction may also be referred to as a combined gaze direction.

[0010] Since the left eye and the right eye are located at different positions, even if the same image is used for gaze tracking of the left eye and the right eye, the optical effects caused by glasses or occluding elements such as eyelashes or noses may have different effects on the gaze direction determined for the left eye and the right eye. When the user focuses his sight on a certain point on a computer screen, for example, the different positioning of the two eyes usually also causes the eyes to face slightly different directions. The different positions and / or orientations of the left eye and the right eye can, for example, cause the flash from the illuminator to be located in different parts of the two eyes. Since different parts of the eyes have different shapes, this may cause the gaze direction determined based on the flash to have different quality / accuracy for the left eye and the right eye. The optical properties of the left eye and the right eye themselves may also be different. For example, the shape of the cornea may be different for the two eyes. The offset between the optical center of the eye and the position of the fovea may also be different between the left eye and the right eye. Although these differences may be relatively small, such differences may make the gaze direction determined for one of the eyes less reliable than the gaze direction determined for the other eye.

[0011] As described above, several different factors may potentially make the determined gaze direction of one eye more reliable than the determined gaze direction of the other eye. If the final gaze direction (or gaze point) is calculated via a simple average of the determined gaze directions for the left eye and the right eye, a temporary error in the determined gaze direction of one eye may cause a large error in the final (or combined) gaze direction obtained, even if the determined gaze directions of the other eyes are very reliable during the time period. Determining confidence values ​​for the left and right eyes, and determining the final (or combined) gaze direction based on the confidence values ​​allows these factors to be taken into account to provide a more accurate final (or combined) gaze direction to the user. For example, if the gaze direction determined for one eye is determined to be unreliable, a lower weight can be provided to the gaze direction than the gaze direction determined for the other eye.

[0012] For example, the left eye and the right eye may be illuminated by one or more illuminators, which may be light sources such as light emitting diodes, for example.

[0013] The images of the left eye and the right eye may, for example, have been captured by one or more image sensors, such as one or more cameras.

[0014] The determined gaze direction of the left eye may, for example, define an estimated gaze point of the left eye. Similarly, the determined gaze direction of the right eye may, for example, define an estimated gaze point of the right eye.

[0015] It should be understood that the reliability of the determined gaze direction of the left eye can be indicated, for example, via a positive scale in which a high confidence value indicates high reliability / confidence, or via a negative (or opposite) scale in which a high confidence value indicates low reliability / confidence. In other words, a high confidence value can be used to indicate high reliability or can be used to indicate high uncertainty / unreliability. Similarly, it will be appreciated that the reliability of the determined gaze direction of the right eye can be indicated, for example, via a positive scale in which a high confidence value indicates high reliability / confidence, or via a negative (or opposite) scale in which a high confidence value indicates low reliability / confidence.

[0016] The first confidence value may be determined (or calculated), for example, based on one or more parameters representing respective factors indicating (or influencing) the reliability of the gaze direction determined for the left eye. The one or more parameters may be calculated (or determined), for example, based on the one or more obtained left eye images. The first confidence value may, for example, refer to a first combined reliability parameter or a combined reliability parameter for the left eye.

[0017] The second confidence value may be determined (or calculated), for example, based on one or more parameters representing respective factors indicating (or influencing) the reliability of the gaze direction determined for the right eye. The one or more parameters may be calculated (or determined), for example, based on the one or more obtained right eye images. The second confidence value may, for example, refer to a second combined reliability parameter or a combined reliability parameter for the right eye.

[0018] According to some embodiments, the circuitry may be configured to determine a final gaze direction based on the determined gaze direction of the left eye, the determined gaze direction of the right eye, the first confidence value, and the second confidence value.

[0019] According to some embodiments, the determination of the final gaze direction may be based at least in part on a weighted combination (or weighted sum) of the determined gaze direction of the left eye and the determined gaze direction of the right eye.

[0020] In the weighted combination, the determined gaze direction of the left eye may be weighted, for example, based on (or by) the first confidence value, and the determined gaze direction of the right eye may be weighted, for example, based on (or by) the second confidence value.

[0021] According to some embodiments, the circuit can be configured to determine (or estimate) one or more flash positions in a predetermined area of ​​the left eye based on one or more images of the left eye obtained, and to determine (or estimate) one or more flash positions in a predetermined area of ​​the right eye based on one or more images of the right eye obtained.

[0022] The left eye and the right eye may, for example, be illuminated by one or more illuminators, which may, for example, be light sources such as one or more light emitting diodes.

[0023] The glint at the left and right eyes may be caused, for example, by reflections of light from one or more illuminators illuminating the eyes.

[0024] According to some embodiments, the circuit may be configured to determine (or estimate) the position of the pupil of the left eye and one or more glint positions at the left eye based on one or more images of the left eye, and determine the gaze direction of the left eye based on the one or more glint positions and the position of the pupil. The circuit may be configured to determine or estimate the position of the pupil of the right eye and one or more glint positions at the right eye based on one or more images of the right eye, and determine the gaze direction of the right eye based on the one or more glint positions and the pupil position.

[0025] According to some embodiments, the predetermined area for the left eye (and similarly for the right eye) may include a first area extending from the center of the cornea to the edge of the spherical area of ​​the cornea, a second area extending from the edge of the spherical area of ​​the cornea to the edge of the cornea, and / or a third area located outside the edge of the cornea.

[0026] The cornea of ​​the eye is generally roughly spherical in the central region of the cornea located around the pupil, but deviates from the spherical shape as it moves away from the center of the cornea. Therefore, the central region of the cornea can be regarded as a spherical region, and the region of the cornea outside the spherical region can be regarded as a non-spherical region.

[0027] According to some embodiments, the third region may be located at the sclera.

[0028] According to some embodiments, the determination of the first confidence value may be based at least in part on the distance between the corneal center (or pupil center) of the left eye and one or more glints at the left eye. The determination of the second confidence value may be based at least in part on the distance between the corneal center (or pupil center) of the right eye and one or more glints at the right eye.

[0029] The cornea of ​​the eye is generally roughly spherical in the central region around the pupil, but deviates from the spherical shape farther away from the center of the cornea and therefore also from the center of the pupil. Therefore, the optical properties of the eye may be more difficult to model in these regions, making glints located far away on the cornea (or even as far as the sclera) less reliable for calculation of gaze direction.

[0030] Determining the first confidence value may, for example, include associating a first distance value (i.e., the distance between one or more flashes of the left eye and the center of the cornea) with a reliability that is lower than the reliability associated with a second distance value. The first distance value may be higher than the second distance value. In other words, a gaze direction obtained using a flash positioned away from the center of the cornea may be assigned (or assigned) a lower reliability than a reliability assigned (or assigned) to a gaze direction obtained using a flash positioned closer to the center of the cornea. In other words, the reliability may decrease as the flash moves further away from the center of the cornea. It should be understood that other factors may also affect the overall reliability of the determined gaze direction, and the overall reliability of the determined gaze direction may therefore increase even if the flash moves away from the center of the cornea.

[0031] According to some embodiments, determining a first confidence value may include associating a flash located in a first area of ​​the left eye with a higher reliability than a reliability associated with a flash located in a second area of ​​the left eye. The first area of ​​the left eye may extend from a center of the cornea to an edge of a spherical area of ​​the cornea. The second area of ​​the left eye may extend from an edge of the spherical area of ​​the cornea to an edge of the cornea. In other words, a gaze direction obtained using a flash located in the first area may be assigned (or allocated) a higher reliability than a reliability assigned (or allocated) to a gaze direction obtained using a flash located in the second area. In other words, the reliability of the determined gaze direction may decrease as the flash moves from the first area to the second area. It should be understood that other factors may also affect the overall reliability of the determined gaze direction, and thus the overall reliability may increase even if the flash moves from the first area to the second area.

[0032] According to some embodiments, determining a first confidence value may include associating a flash located in a second area of ​​the left eye with a higher reliability than a reliability associated with a flash located in a third area of ​​the left eye. The third area of ​​the left eye may be located outside the edge of the cornea. In other words, a gaze direction obtained using a flash located in the second area may be assigned (or allocated) a higher reliability than a reliability assigned (or assigned) to a gaze direction obtained using a flash located in the third area. In other words, the reliability of the determined gaze direction may decrease as the flash moves from the second area to the third area. It should be understood that other factors may also affect the overall reliability of the determined gaze direction, and thus the overall reliability may increase even if the flash moves from the second area to the third area.

[0033] According to some embodiments, determining a second confidence value may include associating a flash located in a first area of ​​the right eye with a higher reliability than a reliability associated with a flash located in a second area of ​​the right eye. The first area of ​​the right eye may extend from a center of the cornea to an edge of a spherical area of ​​the cornea. The second area of ​​the right eye may extend from an edge of the spherical area of ​​the cornea to an edge of the cornea. In other words, a gaze direction obtained using a flash located in the first area may be assigned (or allocated) a higher reliability than a reliability assigned (or assigned) to a gaze direction obtained using a flash located in the second area. In other words, the reliability of the determined gaze direction may decrease as the flash moves from the first area to the second area. It should be understood that other factors may also affect the overall reliability of the determined gaze direction, and thus the overall reliability may increase even if the flash moves from the first area to the second area.

[0034] According to some embodiments, determining a second confidence value may include associating a flash located in a second area of ​​the right eye with a higher reliability than a reliability associated with a flash located in a third area of ​​the right eye. The third area of ​​the right eye may be located outside the edge of the cornea. In other words, a gaze direction obtained using a flash located in the second area may be assigned (or allocated) a higher reliability than a reliability assigned (or allocated) to a gaze direction obtained using a flash located in the third area. In other words, the reliability of the determined gaze direction may decrease as the flash moves from the second area to the third area. It should be understood that other factors may also affect the overall reliability of the determined gaze direction, and thus the overall reliability may increase even if the flash moves from the second area to the third area.

[0035] According to some embodiments, the determination of the first confidence value may be based on the position of one or more glints at the left eye relative to the pupil edge of the left eye. The determination of the second confidence value may be based on the position of one or more glints at the right eye relative to the pupil edge of the right eye.

[0036] The edge of the pupil can be used to estimate the position and size of the pupil. If the flash is located at the edge of the pupil, it will affect the pupil edge detection. Fewer detected pupil edge points will make the pupil position and pupil size determination less reliable, which may affect the accuracy of the pupil center and pupil edge calculations, and thus may affect the reliability of the determined gaze direction. If the flash is located at the edge of the pupil in a bright pupil image (where the pupil is illuminated so that it is bright in the image), the brightness of the pupil may cause the position of the flash to be incorrectly estimated, making the gaze tracking data unreliable. In this case, the flash located on the edge of the pupil can affect the pupil edge detection, while the flash position determination is deteriorated, both of which can affect the reliability of the determined gaze direction.

[0037] The determination of the first confidence value (and similarly, the determination of the second confidence value) may, for example, include associating a gaze direction determined based on a flash that overlaps the pupil edge with a lower reliability than the reliability associated with a gaze direction determined based on a flash that does not overlap the pupil edge. In other words, a gaze direction obtained using a flash that overlaps the pupil edge may be assigned (or allocated) a lower reliability than the reliability assigned (or allocated) to a gaze direction obtained using a flash that does not overlap the pupil edge.

[0038] According to some embodiments, the circuitry may be configured to determine a pupil center of the left eye, a pupil center of the right eye, an eyeball center of the left eye, and an eyeball center of the right eye.

[0039] According to some embodiments, the corneal center may be a location at the spherical region of the cornea where an imaginary line extending from the center of the eyeball through the center of the pupil intersects the spherical region of the cornea.

[0040] According to some embodiments, the determination of the first confidence value may be based at least in part on the contrast between the pupil and the iris of the left eye. A high pupil-iris contrast (e.g., the pupil is much brighter than the iris, or the iris is much brighter than the pupil) facilitates the determination of the pupil center, pupil position, and pupil size. If the pupil-iris contrast is too low, it may be difficult to estimate the position and size of the pupil, which may affect the reliability of the determined gaze direction.

[0041] Determination of the first confidence value may, for example, include associating a first level of contrast with a lower reliability than a reliability associated with a second level of contrast. The first level of contrast may be lower than the second level of contrast. In other words, a gaze direction determined based on an image with high contrast between the pupil and the iris may be given (or assigned) a higher reliability than a reliability assigned (or assigned) to a gaze direction determined using an image with low contrast between the pupil and the iris. In other words, the reliability of the determined gaze direction may decrease as the contrast between the pupil and the iris decreases.

[0042] According to some embodiments, the determination of the second confidence value may be based at least in part on the contrast between the pupil and the iris of the right eye. The determination of the second confidence value may, for example, include associating a first level of contrast (i.e., the pupil-iris contrast of the right eye) with a lower reliability than the reliability associated with the second level of contrast. The first level of contrast may be lower than the second level of contrast.

[0043] According to some embodiments, the determination of the first confidence value may be based at least in part on a first degree of concentration of the left eye gaze position determined at the display device relative to a predetermined position (or reference position) at the display device. Similarly, the determination of the second confidence value may be based at least in part on a second degree of concentration of the right eye gaze position determined at the display device relative to a predetermined position at the display device.

[0044] The same display device and predetermined position can be used for both eyes, for example. Alternatively, different predetermined positions and / or display devices can be used for the left eye and the right eye, for example.

[0045] It should be understood that the degree of centralization is a measure of the degree to which the determined gaze positions are clustered at the display device. The degree of centralization can also be considered a measure of statistical variability, or the magnitude of random error. The degree of centralization can also be considered a measure of the accuracy of gaze tracking.

[0046] High statistical variability (or large random errors) may indicate that the determined gaze direction is less reliable. Therefore, the determination of the first confidence value and the second confidence value may, for example, include associating the first level statistical variability with a lower reliability than the reliability associated with the second level statistical variability. The first level statistical variability may be higher than the second level statistical variability.

[0047] The degree of concentration may be monitored, for example, during gaze tracking, and / or the level of concentration (or level of statistical variability) may be determined during calibration of the gaze tracking system.

[0048] According to some embodiments, the determination of the first confidence value may be based at least in part on a first distance between a predetermined position at the display device and an average value of the determined left eye gaze position (e.g., at the display device). The determination of the second confidence value may be based at least in part on a second distance between a predetermined position at the display device and an average value of the determined gaze position of the right eye (e.g., at the display device).

[0049] The same display device and predetermined position can be used for both eyes, for example. Alternatively, different predetermined positions and / or display devices can be used for the left eye and the right eye, for example.

[0050] The first distance and the second distance between the predetermined position and the determined mean value of gaze positions may, for example, be regarded as a measure of the statistical deviation of the accuracy of gaze tracking, or a measure of the size of a systematic error.

[0051] A large statistical deviation may indicate that the determined gaze direction is less reliable. Therefore, the determination of the first confidence value and the second confidence value may, for example, include associating the first horizontal statistical deviation with a lower reliability, the lower reliability being compared to the reliability associated with the second horizontal statistical deviation. The first horizontal statistical deviation may be higher than the second horizontal statistical deviation. In other words, the gaze direction obtained when the level of statistical deviation is low may be assigned (or allocated) a higher reliability, the higher reliability being compared to the reliability assigned (or allocated) to the gaze direction obtained when the horizontal statistical deviation is high. In other words, the reliability may decrease as the level of statistical deviation increases. It should be understood that other factors may also affect the reliability of the determined gaze direction, and therefore the overall reliability may decrease even if the statistical deviation decreases.

[0052] The first distance and the second distance between the predetermined position and an average of the determined gaze positions for the left eye and the right eye, respectively, may be monitored, for example, during gaze tracking, or may be determined during a calibration step.

[0053] According to some embodiments, the determination of the first confidence value may be based at least in part on the intensity of the flash at the left eye, the shape of the flash at the left eye, and / or the number of flashes at the left eye. Similarly, the determination of the second confidence value may be based at least in part on the intensity of the flash at the right eye, the shape of the flash at the right eye, and / or the number of flashes at the right eye.

[0054] The glint may be located, for example, at the cornea of ​​the left eye and the right eye, respectively.

[0055] If the intensity of the flash is too high, it may result in a large white spot in the image, making it difficult to determine the location of the center of the flash, which may affect the reliability of the determined gaze direction for the eye. A flash with a high intensity may, for example, saturate pixels in the image, making most of the flash appear as bright in the image as the center of the flash. If the intensity of the flash is too low, it may also be difficult to accurately determine the location of the flash. Therefore, a flash with an intensity that is too high or too low may be associated with a lower reliability than a flash with an intensity that is within a certain intermediate intensity range (a suitable range may, for example, depend on the image sensor or camera that captured the image).

[0056] According to some embodiments, determination of the first confidence value may be based at least in part on an estimation of a similarity between an expected (or predetermined) flash shape and a shape of the flash detected at the left eye.

[0057] Determination of the first confidence value may, for example, include associating a first level of flash shape similarity between the flash shapes with a lower reliability than a reliability associated with a second level of flash shape similarity between the flash shapes. The first level of flash shape similarity may be lower than the second level of flash shape similarity. In other words, a gaze direction obtained using a flash having a shape similar to an expected flash shape may be assigned (or allocated) a higher reliability than a reliability assigned (or allocated) to a gaze direction obtained using a flash having a shape less similar to the expected flash shape. In other words, the reliability may decrease as the shape of the flash deviates more from the expected flash shape. It should be understood that other factors may also affect the reliability of the determined gaze direction, and thus the overall reliability may increase even if the shape of the flash deviates more from the expected flash shape.

[0058] A flash is typically caused by a reflection of light from an illuminator. The shape of the flash depends on the shape of the illuminator and the geometry of the surface from which the light is reflected. Therefore, the shape of the flash can be predicted. Deviations from the expected shape can indicate that the flash is located in an area of ​​the eye with unexpected optical properties (which may compromise the reliability of the determined gaze direction), or that the flash originates from a different light source than expected. Therefore, an unexpected flash shape can indicate a lower reliability of the determined gaze direction.

[0059] According to some embodiments, determination of the second confidence value may be based at least in part on an estimation of a similarity between an expected flash shape and a shape of a flash detected at the right eye.

[0060] Determination of the second confidence value may, for example, include associating a first level of flash shape similarity between the flash shapes with a lower reliability than a reliability associated with a second level of flash shape similarity between the flash shapes. The first level of flash shape similarity may be lower than the second level of flash shape similarity. In other words, a gaze direction obtained using a flash having a shape similar to an expected flash shape may be assigned (or allocated) a higher reliability than a reliability assigned (or allocated) to a gaze direction obtained using a flash having a shape less similar to the expected flash shape. In other words, the reliability may decrease as the shape of the flash deviates more from the expected flash shape. It should be understood that other factors may also affect the reliability of the determined gaze direction, and thus the reliability may be increased even if the shape of the flash deviates more from the expected flash shape.

[0061] According to some embodiments, determination of the first confidence value may be based at least in part on a comparison between the number of flashes detected at the left eye and a predetermined (or expected) number of flashes.

[0062] Determination of the first confidence value may, for example, include associating a situation in which the number of detected flashes is consistent with a predetermined (or expected) number of flashes with a higher reliability than a reliability associated with a situation in which the number of detected flashes deviates from the predetermined (or expected) number of flashes. In other words, a gaze direction determined using flashes from an image having a predetermined number of flashes may be assigned (or allocated) a higher reliability than a reliability assigned (or allocated) to a gaze direction determined using flashes from an image having a different number of flashes. In other words, the reliability may be reduced when the number of flashes deviates from the predetermined number of flashes. It should be understood that other factors may also affect the reliability of the determined gaze direction, and therefore the overall reliability may increase even if the number of detected flashes deviates from the predetermined number of flashes.

[0063] The reliability of the determined gaze direction can be increased by using flashes from multiple illuminators. If some expected flashes are missing from the image, this may affect the reliability of gaze tracking. If an unexpectedly high number of flashes are present in the image, it may be more difficult to identify the correct flashes for gaze tracking, and incorrect flashes may obscure image data associated with gaze estimation. For example, an unexpectedly large number of flashes may be caused by other light sources in the environment and / or flashes about to break away from the cornea.

[0064] According to some embodiments, determination of the second confidence value may be based at least in part on a comparison between the number of flashes detected at the right eye and a predetermined (or expected) number of flashes.

[0065] Determination of this second confidence value may, for example, include associating a situation where the number of flashes detected at the right eye is consistent with a predetermined (or expected) number of flashes with a higher reliability than the reliability associated with a situation where the number of flashes detected at the right eye deviates from the predetermined (or expected) number of flashes. In other words, a gaze direction determined using flashes from an image having a predetermined number of flashes may be assigned (or allocated) a higher reliability than a gaze direction determined using flashes from an image having a different number of flashes. In other words, the reliability may be reduced when the number of flashes deviates from the predetermined number of flashes. It should be understood that other factors may also affect the reliability of the determined gaze direction, and therefore the overall reliability may increase even if the number of detected flashes deviates from the predetermined number of flashes.

[0066] According to some embodiments, the determination of the first confidence value may be based at least in part on a parameter indicating the presence of reflections from the glasses in one or more obtained left-eye images, and / or based on a parameter indicating whether a certain area of ​​the left eye is at least partially occluded in one or more obtained left-eye images. Similarly, the determination of the second confidence value may be based at least in part on a parameter indicating the presence of reflections from the glasses in one or more obtained right-eye images, and / or based on a parameter indicating whether a certain area of ​​the right eye is at least partially occluded in one or more obtained right-eye images.

[0067] The glint and / or pupil used in gaze tracking may, for example, be buried in the reflection from the glasses, thereby affecting the reliability of the determined gaze direction. Calculating a first confidence value may, for example, include associating a gaze direction determined during the presence of reflections from the glasses in the left eye with a lower reliability than the reliability associated with gaze tracking data obtained in the absence of such reflections in the left eye. Similarly, calculating a second confidence value may, for example, include associating a gaze direction determined during the presence of reflections from the glasses in the right eye with a lower reliability than the reliability associated with gaze tracking data obtained in the absence of such reflections in the right eye.

[0068] Objects that occlude certain areas of the eye (such as eyelids or eyelashes) may occlude features used in gaze tracking, such as glints and / or pupils, thereby reducing the reliability of the determined gaze direction. Therefore, the gaze direction determined when certain areas of the eye are occluded can be associated with (or assigned) a lower reliability than the gaze direction obtained when these areas of the eye are occluded.

[0069] According to some embodiments, the determination of the first confidence value and the second confidence value may be based at least in part on a parameter for the left eye and the right eye, respectively, indicating a number of pupil edge pixels in one or more acquired images of the eye. Pupil edge pixels may be those one or more partially positioned image pixels along (or located at) a pupil edge that are visible in one or more acquired eye images.

[0070] The edge of the pupil can be used to estimate the position of the pupil center, the edge of the pupil, and the pupil size. This data can be used to estimate the gaze direction and / or gaze point (or fixation point) of the eye. If the number of image pixels located along the edge of the pupil (or at the edge of the pupil) is low, the reliability of the pupil position (and the gaze direction determined based on the pupil position) may be reduced.

[0071] The determination of the first confidence value and the second confidence value may, for example, include associating a first number of pupil edge pixels with a lower reliability, the lower reliability being compared to the reliability associated with the second number of pupil edge pixels. The first number of pupil edge pixels may, for example, be lower than the second number of pupil edge pixels. In other words, a gaze direction determined based on an image with a smaller number of pupil edge pixels may be assigned (or allocated) a lower reliability, the lower reliability being compared to a gaze direction determined based on an image with a larger number of pupil edge pixels. In other words, if the number of pupil edge pixels increases, the reliability may increase. It should be understood that other factors may also affect the reliability of the determined gaze direction, so that even if the number of pupil edge pixels increases, the overall reliability may decrease.

[0072] According to some embodiments, the circuit can be configured to determine, for the left eye and the right eye respectively, a first flash position at the eye caused by reflection of light from a first illuminator based on one or more images of the eye, predict a second flash position at the eye caused by reflection of light from a second illuminator based on the first flash position and a spatial relationship between the first illuminator and the second illuminator, and determine a second flash position at the eye caused by reflection of light from the second illuminator based on one or more images of the eye.

[0073] When the eye is illuminated by the first illuminator, one or more images may be captured, for example, by a camera, from which a first flash position is determined. When the eye is illuminated by the second illuminator, one or more images may be captured, for example, from which a second flash position is determined.

[0074] The position of the second glint at the eye can be predicted, for example, based on the position of the first glint at the eye and based on knowledge of the position and / or orientation of the second illuminator relative to the first illuminator, and optionally also based on knowledge of the position and / or orientation of the second illuminator relative to a camera arranged to capture the second image, a geometric model of the eye and / or knowledge of the estimated position of the eye.

[0075] It should be understood that "predicting" the second flash position based on the first flash position does not necessarily mean that the second flash appears after the first flash. The second flash may, for example, appear in the same image as the first flash. The second flash may, for example, appear in an image before or after the image in which the first flash appears.

[0076] According to some embodiments, the determination of the first confidence value and the second confidence value may be based at least in part on a difference (or distance) between the predicted second flash position and the determined second flash position.

[0077] A large difference between the predicted and determined position of the second flash at an eye may indicate that the gaze direction determined for that eye may not be as reliable as the gaze direction determined for the other eye.

[0078] Thus, calculating a first confidence value (and similarly calculating a second confidence value, but for the right eye rather than the left eye) may, for example, include associating a first size difference between the predicted and determined positions of the second flash at the left eye with a lower reliability than a reliability associated with a second size difference between the predicted and determined positions of the second flash at the left eye. The first size of the distance may be greater than the second size of the difference. In other words, a gaze direction obtained using an image in which the determined and predicted positions of the second flash are close to each other may be assigned (or allocated) a higher reliability than a reliability assigned (or allocated) to a gaze direction obtained using an image in which the predicted and determined positions of the second flash are farther apart from each other. In other words, the reliability may decrease as the distance between the predicted and determined positions of the second flash increases. It should be understood that other factors may also affect the reliability of the determined gaze direction, and the overall reliability may increase even if the distance between the predicted and determined positions of the second flash increases.

[0079] According to some embodiments, the determination of the first confidence value and the second confidence value may be based at least in part on a parameter indicating whether the eye is dominant relative to the other eye.

[0080] For example, a parameter indicating whether one eye is dominant relative to the other eye may be calculated based on how the respective eye moves and / or based on the respective deviation between the focus point of the respective eye and the reference point when the user is prompted to look at the reference point.

[0081] According to some embodiments, the system may include at least one illuminator for illuminating the eye, and one or more cameras for capturing images of the eye.

[0082] The at least one illuminator may be arranged, for example, coaxially with the at least one camera for capturing a bright pupil image.The at least one illuminator may be arranged, for example, non-coaxially with the at least one camera for capturing a dark pupil image.

[0083] According to some embodiments, the determination of the first confidence value and the second confidence value may be based at least in part on a parameter indicating the size of a residual remaining after calibration of the eye models used to calculate the gaze directions of the left eye and the right eye, respectively.

[0084] According to some embodiments, the circuit may be configured to perform gaze tracking for a left eye for a sequence of images, estimate a distance from the sequence of images to the left eye (e.g., the distance from a camera that captured the sequence of images to the left eye), and estimate a noise level in the estimated distance to the left eye. The circuit may be configured to perform gaze tracking for a right eye for a sequence of images, estimate a distance from the sequence of images to the right eye (e.g., the distance from a camera that captured the sequence of images to the right eye), and estimate a noise level in the estimated distance to the right eye. The determination of the first confidence value and the second confidence value may be based on parameters indicating noise levels in the estimated distances to the left eye and the right eye, respectively.

[0085] A high noise level in the estimated distance to the eye can indicate that the determined gaze direction of the eye is unreliable. Therefore, the calculation of the confidence value of the eye (such as the left eye and / or the right eye) can, for example, include associating a first noise level with a reliability lower than the reliability associated with the second noise level. The first noise level can be higher than the second noise level. In other words, the gaze direction obtained when the noise level in the estimated distance is high can be assigned (or assigned) a lower reliability, which is compared to the reliability assigned (or assigned) to the gaze direction obtained when the noise level in the estimated distance is low. In other words, when the noise level in the estimated distance increases, the reliability may decrease. It should be understood that other factors can affect the reliability of the determined eye gaze direction, and therefore even if the noise level in the estimated distance increases, the overall reliability can also increase.

[0086] According to a second aspect, a method is provided. The method includes obtaining one or more images of a user's left eye and one or more images of a user's right eye. The method includes determining a gaze direction of the user's left eye based on at least one obtained image of the left eye, and determining a gaze direction of the user's right eye based on at least one obtained image of the right eye. The method includes determining a first confidence value based on the one or more obtained left eye images. The first confidence value represents an indication of the reliability of the determined gaze direction of the left eye. The method includes determining a second confidence value based on the one or more obtained right eye images. The second confidence value represents an indication of the reliability of the determined gaze direction of the right eye. The method includes determining a final gaze direction based at least in part on the first confidence value and the second confidence value.

[0087] The method of the second aspect may be performed, for example, by the system of any embodiment of the first aspect, or by a circuit included in such a system.

[0088] Embodiments of the method according to the second aspect may, for example, comprise features corresponding to those of any embodiment of the system according to the first aspect.

[0089] According to a third aspect, one or more computer-readable storage media storing computer-executable instructions are provided, which when executed by a computing system performing eye / gaze data processing causes the computing system to perform a method. The method may, for example, be the method according to the second aspect.

[0090] Embodiments of the one or more computer-readable storage media according to the third aspect may, for example, comprise features corresponding to those of any embodiment of the system according to the first aspect.

[0091] The one or more computer-readable media may be, for example, one or more non-transitory computer-readable media.

[0092] It is noted that embodiments of the invention relate to all possible combinations of features in the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Exemplary embodiments will be described below with reference to the accompanying drawings, in which:

[0094] Figure 1 An eye tracking system according to an embodiment is shown;

[0095] Figure 2 An exemplary image of an eye is shown;

[0096] Figure 3 It is a cross-sectional view of a portion of the eye;

[0097] Figure 4a-4g shows an example of where the flash from the illuminator may be located in the left and right eye depending on where the user is looking;

[0098] Figure 5 An example of a bright pupil image is shown;

[0099] Figure 6 An eye is shown with a glint located at the edge of the pupil;

[0100] Figure 7a and 7b The estimated gaze point before and after calibration is shown;

[0101] Figure 8 Examples of low and high statistical variability are shown;

[0102] Fig. 9 Examples of low and high statistical deviations are shown;

[0103] Figure 10-12 is a flow chart of a method according to an embodiment; and

[0104] Fig.13It is a block diagram of a special-purpose computer system that can be used in at least some parts of the apparatus or system of the present invention, or implement at least some parts of the method of the present invention.

[0105] All figures are schematic and generally only show parts which are necessary in order to elucidate the various embodiments, while other parts may be omitted or merely suggested. DETAILED DESCRIPTION

[0106] The following description only provides exemplary embodiments and is not intended to limit the scope, applicability or configuration of the present disclosure. On the contrary, the following description of the exemplary embodiments will provide a possible description for implementing one or more exemplary embodiments to those skilled in the art. It should be understood that various changes may be made to the functions and arrangements of the various elements without departing from the spirit and scope of the present invention as set forth herein.

[0107] For example, any detail discussed about an embodiment may or may not be present in all expected versions of that embodiment. Likewise, any detail discussed about an embodiment may or may not be present in all expected versions of other embodiments discussed herein. Finally, if any detail about an embodiment herein is not discussed, it means that such detail may or may not be present in any version of any embodiment discussed herein.

[0108] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be appreciated by those of ordinary skill in the art that the embodiments may be practiced without these specific details. For example, the circuits, systems, networks, processes, and other elements of the present invention may be illustrated as components in block diagram form to avoid obscuring the embodiments in unnecessary detail. In other examples, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiments.

[0109] In addition, it should be noted that each embodiment can be described as a process, which is depicted as a flow chart, a flow chart, a data flow chart, a structure diagram or a block diagram. Although a flow chart can describe an operation as a sequential process, many operations can be performed in parallel or simultaneously. In addition, the order of the operations can be rearranged. The process can terminate when its operation is completed, but may have additional steps that are not discussed or included in the figure. In addition, not all operations in any particularly described process can occur in all embodiments. A process can correspond to a method, a function, a process, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to the function returning to the calling function or the main function.

[0110] The term "machine-readable medium" and the like include, but are not limited to, temporary and non-temporary, portable or fixed storage devices, optical storage devices, wireless channels, and various other media capable of storing, containing, or carrying instructions and / or data. Code segments or machine-executable instructions may represent any combination of procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, categories, or instructions, data structures, or program statements. A code segment may be coupled to another code segment or hardware circuit by passing and / or receiving information, data, independent variables, parameters, or memory contents. Information, independent variables, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0111] In addition, embodiments of the present invention may be implemented at least in part manually or automatically. Manual or automatic implementation may be performed or at least assisted by the use of machines, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented with software, firmware, middleware, or microcode, program code or code segments for performing the necessary tasks may be stored in a machine-readable medium. A processor may perform the necessary tasks.

[0112] Eye Tracking

[0113] Figure 1 An eye tracking system 100 (which may also be referred to as a gaze tracking system) according to an embodiment is shown. System 100 includes illuminators 111 and 112 for illuminating the eyes of a user and a light sensor 113 for capturing an image of the eyes of the user. Illuminators 111 and 112 may be, for example, light emitting diodes that emit light in an infrared band or a near infrared band. Light sensor 113 may be, for example, a camera, such as a complementary metal oxide semiconductor (CMOS) camera or a charge coupled device (CCD) camera.

[0114] The first illuminator 111 is arranged coaxially (or close to) the light sensor 113 so that the light sensor 113 can capture a bright pupil image of the user's eye. Due to the coaxial arrangement of the first illuminator 111 and the light sensor 113, light reflected from the retina of the eye returns through the pupil toward the light sensor 113, so that the pupil appears brighter than the iris surrounding it in the image of the eye illuminated by the first illuminator 111. The second illuminator 112 is arranged non-coaxially (or further away) from the light sensor 113 that captures the dark pupil image. Due to the non-coaxial arrangement of the second illuminator 112 and the light sensor 113, the light reflected from the retina of the eye does not reach the light sensor 113, and the pupil appears darker than the iris surrounding it in the image of the eye illuminated by the second illuminator 112. The illuminators 111 and 112 can, for example, illuminate the eye in turn so that each second image is sometimes a bright pupil image and sometimes a dark pupil image.

[0115] Eye tracking system 100 also includes circuitry 120 (e.g., including one or more processors) for processing images captured by light sensor 113. Circuitry 120 may be connected to light sensor 113 and illuminators 111 and 112, e.g., via a wired or wireless connection. In another example, circuitry 120 in the form of one or more processors may be provided in one or more stacked layers below the light-sensitive surface of light sensor 113.

[0116] Figure 2 An example of an image of an eye 200 captured by the light sensor 113 is shown. The circuit 120 may, for example, employ image processing (e.g., digital image processing) to extract features in the image. The circuit 120 may, for example, employ pupil center corneal reflection (PCCR) eye tracking to determine where the eye 200 is gazing. In PCCR eye tracking, the processor 120 estimates the location of the center of the pupil 210 and the location of the center of the glint 220 at the eye 200. The glint 220 is caused by the reflection of light from one of the illuminators 111 and 112. The processor 120 uses the glint 220 to calculate the user's position in space and uses the pupil 210 to calculate the location at which the user's eye 200 is pointed. Since there is typically an offset between the optical center of the eye 200 and the fovea of ​​the retina, the processor 120 performs calibration of the fovea offset to be able to determine where the user is gazing. The gaze directions obtained from the left eye and the right eye can then be combined to form a combined estimated gaze direction (or viewing direction). As described below, many different factors may influence how the gaze directions of the left and right eyes should be weighted relative to each other when forming this combination.

[0117] In reference Figure 1 In the described embodiment, illuminators 111 and 112 are arranged in an eye tracking module 110, which is located below a display viewed by a user. This arrangement is used as an example only. It should be understood that almost any number of illuminators and light sensors can be used for eye tracking, and such illuminators and light sensors can be distributed in many different ways relative to the display viewed by the user. It should be understood that the eye tracking scheme described in the present disclosure can be used, for example, for remote eye tracking (e.g., in a personal computer, a smart phone, or integrated in a vehicle) or for wearable eye tracking (such as in virtual reality glasses or augmented reality glasses).

[0118] Location of the flash on the cornea

[0119] The industry trend is toward larger screens. While eye trackers exist that support screens up to 27" (16:9), it is desirable to support screens larger than this. The problem with supporting larger screens is that when the user looks at a point close to the edge of the screen, the glare tends to break off the cornea, making it difficult to determine the user's point of focus (also known as the gaze point).

[0120] However, a glint on the right eye will not typically detach from the cornea at the same time as a glint on the left eye. This means that even if the gaze tracking data from one of the eyes is not very useful (or not very reliable), the gaze tracking data from the other eye can still be of good quality. Today, the gaze point on the screen is typically formed as a simple average of the gaze point of the left eye and the gaze point of the right eye. As a result, the user experience is limited by the worst eye, not the best eye. If the position of the glint at each eye is taken into account during the weighting of the gaze tracking data from the left and right eye, a user experience that is more consistent with the gaze tracking data of the best eye can be provided. The gaze tracking data for an eye may, for example, include a gaze direction of the respective eye and / or a gaze point of the respective eye.

[0121] Figure 3 Cross-sections of different portions of the eye 300 are shown. The cornea 310 has a central region 311, which is generally close to a spherical shape, and an outer region 312, which is non-spherical and therefore more difficult to model. The central region 311 (also referred to herein as the spherical region of the cornea 310) extends from a corneal center 313 to an edge 314 of the spherical region 314. The corneal center 313 is the location at the spherical region 311 of the cornea 310 where a virtual line 340 extending from the eyeball center 330 through the pupil center 350 intersects the spherical region 311 of the cornea 310. The outer region 312 of the cornea 310 (also referred to herein as the non-spherical region of the cornea 310) extends from an edge 314 of the spherical region 311 of the cornea 310 to an edge 315 of the cornea 310. Figure 3 The sclera 320 of the eye 300 is also shown.

[0122] In order to determine the gaze point of the eye 300, it is necessary to find the user's position in space, which is done by using the position of the glint and the geometry of the eye. The glint is also used to determine the rotation angle of the eye. Even if the glint is not completely detached from the cornea 310, it may be located in the non-spherical region 312 of the cornea 310, where the shape varies greatly from person to person. The shape of the non-spherical region 312 of the cornea 310 may even be different between a person's left and right eye.

[0123] Figure 4a-4gAn example is shown of where the flash from the illuminator may be located for the left and right eyes, depending on which part of the screen 420 the user 410 is looking at.

[0124] In these figures, screen 420 is shown in the upper right corner, and the user's actual gaze point (or actual focus point) is shown by circle 421. Point 422 on screen 420 represents a reference point that the user can look at, for example, during calibration. Figure 4a-4g The position of the flash 411 at the left eye 412 and the position of the flash 413 at the right eye 414 are shown.

[0125] exist Figure 4a In FIG. 4 , user 410 is looking directly at a camera located below screen 420. Both glints 411 and 413 are on the spherical area 311 of the cornea. Therefore, glints of both eyes appear to be suitable for gaze tracking.

[0126] exist Figure 4b , the user's gaze point 421 has moved upward and to the right on the screen 420. Both the left eye glint 411 and the right eye glint 413 are still in the spherical area 311 of the cornea. Therefore, the glint at both eyes seems to be very suitable for gaze tracking.

[0127] exist Figure 4c , the user's gaze point 421 has moved further up and to the right on screen 420. The left eye glint 411 is now in the non-spherical region 312 of the cornea, while the right eye glint 413 is still in the spherical region 311 of the cornea. Therefore, the glint 413 at the right eye seems to be very suitable for gaze tracking, while the glint 411 at the left eye will make gaze tracking less reliable.

[0128] exist Figure 4d , the user's gaze point 421 moves further up and to the right on the screen 420. The left eye glint 411 is in the non-spherical region 312 of the cornea, while the right eye glint 413 is in the spherical region 311 of the cornea. Therefore, the glint 413 at the right eye seems to be very suitable for gaze tracking, while the glint 411 at the left eye will make gaze tracking less reliable.

[0129] exist Figure 4e , the user's gaze point 421 has moved further up on the screen 420. The left eye glint 411 is close to the edge of the cornea 310, while the right eye glint 413 is in the non-spherical region 312 of the cornea. Since the left eye glint 411 is about to break away from the cornea 310, the geometry of the eye causes multiple glints to appear on the left eye 412 instead of just one glint. This makes gaze tracking based on the left eye glint 411 unreliable, while the right eye glint 413 may be slightly better suited for gaze tracking.

[0130] exist Figure 4f, the user's gaze point 421 has moved to the upper right corner of the screen 420. The left eye glint 411 is now on the sclera 320 (this can be seen by the non-circular shape of the glint and the presence of additional glint further outward on the sclera), while the right eye glint 413 is in the non-spherical region 312 of the cornea.

[0131] exist Figure 4g , the user's gaze point has moved further up and to the right. The left eye glint 411 is on the sclera 320, while the right eye glint 413 is now on the edge of the cornea 310. Since the right eye glint 413 is about to break away from the cornea 310, the geometry of the eye causes multiple glints to appear at the right eye 414 instead of just one glint.

[0132] As described above, the position of the glint on the cornea may be different between the two eyes. This is a factor that may affect the reliability of gaze tracking data (such as gaze direction) determined based on the glint on the left eye and the right eye, respectively, and is listed as Factor A in Table 1 below. The factor A may be expressed, for example, as the distance between the glint and the center 313 of the cornea. Since the spherical area 311 of the cornea 310 is located at the center of the cornea 310 near the pupil, the position of the glint at the cornea 310 may also be expressed as the position of the glint relative to the pupil of the eye (or relative to the center 350 of the pupil). Factor A may be expressed, for example, as the distance between the glint at the eye and the center 350 of the pupil of the eye.

[0133] Pupil / iris contrast

[0134] The contrast between the pupil and the iris depends both on the reflection of the iris and on the bright pupil effect of the user. The bright pupil effect can vary significantly depending on age, ethnicity, pupil size, and eye tracker geometry, but can also vary for several other reasons. This means that the pupil / iris contrast may, for example, always be poor at one eye, or the pupil / iris contrast may be good at both eyes but may suddenly disappear for one eye. The bright pupil effect may, for example, disappear within a certain angle. When there is low contrast (or no contrast) between the pupil and the iris, it may be difficult to detect the pupil and thus the gaze direction cannot be correctly determined. If this happens to one eye, the gaze tracking data (e.g., gaze direction) for that eye should be given a lower weight (or confidence) than the gaze point data for the other eye. In other words, the pupil / iris contrast is a factor that affects the reliability of the gaze tracking data (such as the determined gaze direction) for the left and right eyes, and is listed as Factor B in Table 1 below.

[0135] Figure 5An example is shown, where the pupil 511 of the right eye 510 is clearly visible as a bright spot, whereas in the left eye 520 it is almost impossible to determine the pupil center in the left eye 520 due to the low pupil / iris contrast.

[0136] Flash intensity

[0137] If the pupil or glint is determined incorrectly, the estimated gaze point on the screen may be off. Even small errors can result in a poor eye tracking experience. In order to accurately determine the location of the glint, it is important that the glint is not over-saturated in the image and that the intensity of the glint is not too low. Saturation may cause the glint to appear as a large white spot in the image. The center of the glint may be difficult to estimate because saturation may cause the entire area to have the same color as the center of the glint. Possible causes of low / high intensity glints may be eye surgery or changes in the thickness of the tear film on the eye. The glint intensity is a factor that affects the reliability of gaze tracking data (such as the determined gaze direction) for the left and right eyes, and is listed as Factor C in Table 1 below.

[0138] Flashes on the edge of the pupil

[0139] Figure 6 An eye 600 is shown with a glint 610 from an illuminator located at the edge 620 of a pupil 630. The glint 610 overlaps the pupil edge 620 and thus causes a smaller portion of the edge 620 to be visible in the image, making the estimate of the pupil center less reliable. The size of the glint 610 relative to the pupil 630 may be different in different situations. If the glint 610 is large relative to the pupil 630, it may cover a significant portion of the edge 620 of the pupil 630.

[0140] Furthermore, in a bright pupil image, if the glint 610 is located at the edge 620 of the pupil 630, the calculation of the center of the glint 610 may be affected by the bright pupil 630. The calculation of the center of the glint 610 may be performed, for example, similarly to the calculation of the centroid (but using light intensity instead of mass). The calculated glint position may thus be "dragged" into the pupil 630, which may result in an incorrectly determined user position and / or gaze angle.

[0141] In view of the above, if one of the eyes has glint 610 on pupil edge 620 (or has glint 610 overlapping pupil edge 620), then the gaze tracking data from that eye should be weighted less than the gaze tracking data from the other eye, especially if glint 610 covers a large portion of pupil edge 620. Glitter on the pupil edge is a factor that affects the reliability of gaze tracking data (such as determined gaze direction) for the left and right eyes, and is listed as Factor D in Table 1 below.

[0142] Glasses reflection

[0143] Wearing glasses may cause large reflections in the eye image. If such reflections cover the eye in a bad way (e.g., covering certain areas such as the pupil), it may be difficult or even impossible to find the pupil center and / or the glint in the image. If this happens for one eye, it may be desirable to completely ignore gaze tracking data from that eye. On the other hand, in some cases, the glasses reflection may cover only a portion of the eye, so that the pupil and glint can still be detected, but with lower accuracy than normal. In such a case, it may be desirable to use gaze tracking data from that eye, but with a lower weight compared to the other eyes. The eye tracker can be designed so that the probability of both eyes having reflections from the glasses at the same time is very low (e.g., by arranging the illuminator and camera at appropriate locations). Therefore, the impact of reflections from the glasses can be reduced by appropriately weighting the gaze tracking data from the left and right eyes (such as the gaze direction). The presence of glasses reflections is a factor that affects the reliability of gaze tracking data (such as the determined gaze direction) for the left and right eyes, and is listed as Factor E in Table 1 below.

[0144] Occluding objects

[0145] Objects such as eyelashes or eyelids may in some cases cover critical portions of any eye, making it difficult to determine the gaze point (or gaze direction) of that eye. In such cases, it may be desirable to give gaze tracking data (e.g., gaze direction) from that eye a lower weight (or lower confidence) than gaze tracking data from the other eye. In some cases where critical areas of an eye are occluded, it may even be desirable to completely ignore gaze tracking data (or gaze direction) from that eye and use gaze tracking data (or gaze direction) from the unobstructed eye instead. The presence of occluding objects is a factor that affects the reliability of gaze tracking data (such as the determined gaze direction) for the left and right eyes, and is listed as Factor F in Table 1 below.

[0146] Machine Learning Confidence

[0147] A black box machine learning algorithm may be employed to generate confidence values ​​for each eye. A properly designed machine learning algorithm may potentially generate appropriate confidence values ​​if a sufficient amount of training data is available to the algorithm. Such confidence values ​​may be used as a factor indicative of the reliability of gaze tracking data (such as the determined gaze direction) for the left and right eyes, and are listed as Factor G in Table 1 below. For example, a machine learning algorithm may be trained based on a comparison between the determined gaze tracking data and the corresponding reference point / position of the user's actual gaze.

[0148] The number of pupil edge pixels

[0149] The pupil center can be calculated using those parts of the pupil edge that are visible in the image. The images are typically digital images, for which the position of the pupil center is estimated using pixels positioned along (or covering) the pupil edge. The more pupil edge pixels that can be obtained in the image, the easier it is to determine the pupil center. Many factors may affect the number of pupil edge pixels, such as pupil size, occlusion, etc. Basically, it is desirable to use an image in which nothing covers / occludes the pupil, and it is desirable to have as many pupil edge pixels as possible. If one eye has fewer pupil edge pixels, it may therefore be desirable to give the gaze tracking data of that eye (e.g., the determined gaze direction) a lower weight (or lower confidence) than the gaze tracking data from the other eye. The number of pupil edge pixels is a factor that affects the reliability of gaze tracking data (such as the determined gaze direction) for the left eye and the right eye, respectively, and is listed as factor H in Table 1 below.

[0150] Spatial distribution of pupil edge pixels

[0151] The spatial distribution of pupil edge pixels may also affect how well the pupil center position, pupil size, and / or pupil shape can be estimated. For example, x pupil edge pixels located only at the left half of the pupil may result in less reliable gaze tracking data (e.g., less reliable gaze direction) than if x / 2 pupil edge pixels were available at the left half of the pupil and x / 2 pupil edge pixels were available at the right half of the pupil. In other words, the degree of spatial distribution of pupil edge pixels in one or more images obtained from the respective eye is therefore a factor that affects the reliability of gaze tracking data (such as the determined gaze direction) for the left and right eyes, respectively, and is listed as factor Q in Table 1 below. Factor Q may also be expressed as the degree of spatial distribution of the available pupil edge pixels along the pupil edge of the eye, or as the angular distribution of pupil edge pixels about the pupil center.

[0152] Calibration quality (residual)

[0153] In order to provide a good eye tracking experience, calibration is required. Figure 7a-7b The calibration of the eye tracker is shown. During the calibration process, the user looks at test points on the screen, and an algorithm optimizes the eye model to minimize the error at each of these points. Figure 7a As shown in , the user looks at a plurality of test points 701. A gaze point 702 is estimated by the eye tracker for each test point 701. In this example, ten consecutive gaze points 702 are generated for each test point 701. The eye model is then calibrated to minimize the deviation between the test point 701 and the associated estimated gaze point 702 (e.g., to minimize the sum of the squares of the deviations). The calibrated result is shown in Figure 7b , where the new gaze point 703 is closer to the test point 701. The remaining deviation (or error) may be referred to as a residual, or simply a residual. The size of the residual may be considered a measure of the quality of the calibration. Calibration is performed for each eye, and the residual is typically different for each eye. If gaze data for one eye (such as the determined gaze direction) is associated with a large residual, this may indicate that the gaze tracking data for this eye is less reliable, and it may be desirable to give the gaze tracking data for this eye a lower weight (or lower confidence) than for the other eye. In other words, the size of the residual (or calibration quality) is a factor that affects the reliability of gaze tracking data (such as the determined gaze direction) for the left eye and the right eye, respectively, and is listed as Factor I in Table 1 below.

[0154] In Table 1, factor I is represented as a parameter indicating the size of the residual remaining after calibration of the eye model used to calculate the gaze tracking data of the eye. The residual can be of different types. Figure 7a and 7b In the example described, the deviation between the test point 701 and the estimated gaze point 702 is minimized, and the residual is a measure of the remaining deviation (e.g., a sum of squares). Other possible quantities used in the calibration include the deviation between the actual eye position and the estimated eye position, or the deviation between the actual pupil size and the estimated pupil size. If the calibration is performed to minimize these other quantities, the residual obtained can be formed, for example, as the sum of squares of these quantities (or deviations) remaining after the calibration.

[0155] As described above, the size of the residual remaining after calibration for an eye (such as the left eye and / or the right eye) can indicate the reliability of gaze tracking data obtained during gaze tracking of the eye after calibration. For example, calculating a confidence value for an eye (such as the left eye and / or the right eye) can include associating a first residual size with a lower reliability than a reliability associated with a second residual size. The first residual size can be higher than the second residual size.

[0156] Flash association confidence

[0157] Eye trackers using multiple illuminators, such as those described above Figure 1 The described eye tracking system 100 may use a flash from a first illuminator 111 to predict where a flash from a second illuminator 112 should appear. The eye tracker 100 may, for example, illuminate one eye (or both eyes) using the first illuminator 111 during a first image frame, and illuminate one eye (or both eyes) using the second illuminator 112 during a second image frame, which is a frame subsequent to the first image frame. The image captured by the light sensor 113 at the first image frame will include a flash at the eye caused by reflection of light from the first illuminator 111. Since such a short time has passed between the first image frame and the second image frame, the user typically does not move much distance between the frames. Since the position of the second illuminator 112 relative to the first illuminator 111 and the light sensor 113 is known, it is possible to predict where the flash caused by reflection of light from the second illuminator 112 should appear in the image captured in the next image frame by using the position of the flash from the first illuminator 111 in the first image frame. This prediction may also be based on a geometric model of the eye and an estimated position of the eye relative to the illuminators 111 and 112 and the light sensor 113 .

[0158] Several flashes may sometimes appear in an image, and there is a risk that a wrong flash is detected in the image, resulting in an error in the estimated gaze point. The ability to predict the flash position allows the eye tracker to check whether the detected flash (or the determined flash position) appears to be the correct one. In other words, the closer the detected flash is to the predicted position, the greater the chance that it is the correct flash. The difference between the predicted position and the position actually detected (or actually determined or estimated) in the image is a factor representing the reliability of the gaze tracking data (such as the determined gaze direction) for the left eye and the right eye respectively, and is listed as Factor J in Table 1 below.

[0159] It should be understood that even if the correct flash has been detected, the detected position may still deviate from the predicted position. If such a deviation is large, it may indicate that something else is wrong and / or that the gaze tracking data for that eye is unreliable. Therefore, the gaze tracking data for an eye with a large deviation between the predicted flash position and the detected flash position may be given a lower (confidence) weight than the gaze tracking data for the other eye.

[0160] It will also be appreciated that the two illuminators 111 and 112 may, for example, be used to illuminate an eye in the same image frame. The position of the flash from one of the illuminators in the image may then be used to predict the position of the flash from the other illuminator in the same image. In other words, it is not necessary to perform a "prediction" from one image to a subsequent image. In fact, the prediction may also be performed in reverse from one image to a previous image.

[0161] Eye dominance

[0162] If the user has a dominant eye, gaze tracking data from that eye may be given a higher weight (or higher confidence) than gaze tracking from the other eye. For example, dominant eye may be detected by analyzing how the left and right eyes move and / or by analyzing the actual gaze points of the left and right eyes when the user looks at a test point. Eye dominance is a factor that indicates the reliability of gaze tracking data (such as the determined gaze direction) for the left and right eyes, respectively, and is listed as factor K in Table 1 below.

[0163] Even though the dominant eye can be detected by analyzing images captured by an eye tracker, the dominant eye may be known in advance. For example, the user may manually input this information.

[0164] Gaze accuracy (concentration)

[0165] Precision is a measure of statistical variability, or how closely the gaze points are concentrated / converged. Figure 8 The left side of shows an example of poor accuracy (or high statistical variability, or large random error), where the estimated gaze points 801 are distributed over a relatively large area, rather than concentrated / clustered in a small area. Figure 8 The right side of FIG. 1 shows an example with better accuracy, where the estimated gaze points 803 are closer (or concentrated / converged) to each other. The estimated gaze points 801 and 803 are relative to the user's true gaze point (in Figure 8 The position of the digits (indicated by 802 and 804) is not relevant for accuracy.

[0166] During gaze tracking, the gaze accuracy of each eye (i.e., the accuracy of the gaze direction determined for the left eye and the right eye respectively) can be continuously calculated to monitor whether there is anything in the image that makes it difficult for the algorithm to correctly find the pupil and the glint. If the accuracy of one eye becomes worse than the accuracy of the other eye, there may be something in the image that the algorithm cannot handle. Therefore, it may be desirable to apply a lower weight (or confidence) to the gaze tracking data for that eye, which is compared to the gaze tracking data for the other eye. Gaze accuracy is a factor that indicates the reliability of the gaze tracking data for the left eye and the right eye respectively, and is listed as Factor L in Table 1 below. Factor L can also be expressed as the degree of concentration of the determined gaze locations.

[0167] When the user is gazing at a fixed point, the determined gaze direction of both eyes should be constant. During a saccade (when the user moves their gaze to a new point), the determined gaze direction of both eyes should change. The change in the determined gaze direction caused by a saccade may be mistaken for poor gaze accuracy of both eyes. However, if the change in the gaze direction of the first eye determined while the gaze direction of the determined second eye is constant, this can be an indication of poor accuracy of the first eye.

[0168] Gaze accuracy

[0169] Accuracy is a measure of statistical deviation (or systematic error). Fig. 9 The left side of shows an example of poor accuracy (or large statistical deviation), where the estimated gaze point 901 is distributed in a region above the user's true gaze point 902 (the average of the estimated gaze point 901 is above the true gaze point 902). Fig. 9 An example of higher accuracy (or small statistical deviation) is shown on the right side of , where the estimated gaze point 903 is distributed around the user's true gaze point 904, so that the average of the estimated gaze point 903 is located close to the true gaze point 904. If the accuracy for one eye is worse, it may be desirable to apply a lower weight (or confidence) to the gaze tracking data (or determined gaze direction) of that eye than to the gaze tracking data (or determined gaze direction) of the other eye with higher accuracy. Gaze accuracy is a factor that represents the reliability of gaze tracking data (such as determined gaze direction) for the left eye and the right eye respectively, and is listed as factor M in Table 1 below. Factor M in Table 1 can, for example, be represented as the distance between a predetermined position (such as a reference position or a true gaze position) and the average of the determined gaze positions for the eyes.

[0170] Gaze accuracy may be determined continuously during gaze tracking or during calibration.

[0171] Noise in user distance

[0172] Flash position can be used to calculate user distance. Errors in flash position can cause the calculated user position in space to be noisy, which will also cause noise in the estimated gaze point (or gaze direction). In other words, a high noise level in the calculated user distance is an indication that the gaze tracking data is unreliable. For each eye, the user position can be calculated continuously. The noise level in the user distance calculated for each eye can be monitored during gaze tracking. If the noise level is high for the user distance calculated for one eye, it may be desirable to apply a lower weight (or confidence) to the gaze tracking data (or determined gaze direction) for that eye than to the gaze tracking data (or determined gaze direction) from the other eye. The noise level in the calculated user distance is a factor indicating the reliability of gaze tracking data (such as the determined gaze direction) for the left eye and the right eye respectively, and is listed as factor N in Table 1 below.

[0173] Flash shape

[0174] If a circular illuminator (or an illuminator that can be considered a point source) is used to illuminate the eye, reflections at the spherical part of the cornea will generally result in a circular glint in the image. If the shape of the glint is not circular, this may indicate that the cornea has non-spherical structures (such as scars) or that the glint does not exist at all on the cornea. Both of these situations can lead to a poor estimate of the user's position and a poor eye tracking experience.

[0175] The shape of the optics and / or illuminator of the light sensor (or camera) can affect the shape of the glint in the image. However, the shape of the glint in the image can be predicted at least to some extent (e.g., if it is assumed that the glint is located in the spherical part of the cornea, the glint shape can be predicted).

[0176] For the left eye and the right eye, the similarity between the shape of the detected flash and the expected flash shape can be monitored. If an unexpected flash shape is detected for the first eye but not for the second eye, it may be desirable to apply a lower weight (or confidence) to the gaze tracking data (or determined gaze direction) for the first eye than to the gaze tracking data (or determined gaze direction) for the second eye. Flash shape is a factor that indicates the reliability of gaze tracking data (such as determined gaze direction) for the left eye and the right eye respectively, and is listed as Factor O in Table 1 below.

[0177] Number of flashes

[0178] When a large number of potential flashes are found, the risk of selecting the wrong flash increases. This can occur, for example, when there are other light sources with infrared content in the environment, or when the flash is about to break away from the cornea. Therefore, an unexpectedly large number of flashes can be an indication that the gaze tracking data from the eye is unreliable.

[0179] In some eye trackers (e.g. in virtual reality eye trackers), multiple illuminators may be used simultaneously so that several flashes intentionally appear at the eye. If some expected flashes are missing in the image of the eye, the reliability of the gaze tracking data may be reduced.

[0180] In view of the above, it may be desirable to provide a higher weight (or higher confidence) to gaze tracking data (or determined gaze direction) from an eye whose detected number of flashes matches the expected number of flashes than to gaze tracking data (or determined gaze direction) from an eye whose detected number of flashes differs from the expected number of flashes. The number of detected flashes is one factor indicative of the reliability of gaze tracking data (such as the determined gaze direction) for the left eye and the right eye, respectively, and is listed as factor P in Table 1 below.

[0181] Exemplary Embodiments

[0182] Fig.10 is a flow chart of method 1000, which may be performed as described above with reference to Figure 1 The described eye tracking system 100 is performed, or is performed by circuitry 120 of the eye / gaze tracking system 100 .

[0183] The method 1000 includes obtaining 1001 one or more images of a user's left eye and one or more images of a user's right eye. The images may be obtained by an image sensor 113.

[0184] The method 1000 includes determining 1002 a gaze direction of a left eye of a user based on at least one obtained left-eye image, and determining 1003 a gaze direction of a right eye of a user based on at least one obtained right-eye image.

[0185] The method 1000 includes determining 1004 a first confidence value based on one or more obtained left-eye images. The first confidence value represents an indication of the reliability of the determined left-eye gaze point.

[0186] The method 1000 comprises determining 1005 a second confidence value based on the one or more obtained right eye images. The second confidence value represents an indication of the reliability of the determined right eye gaze point.

[0187] The method 1000 includes determining 1006 a final (or combined) gaze direction based on the determined gaze directions of the left eye and the right eye and based on the first confidence value and the second confidence value. Step 1006 may, for example, determine a combined gaze point based on the determined gaze directions of the left eye and the right eye and based on the first confidence value and the second confidence value, instead of determining the final gaze direction.

[0188] In steps 1004 and 1005, a first confidence value and a second confidence value are determined (or calculated) based on one or more parameters representing respective factors indicating the reliability of the determined gaze tracking data (or gaze directions) of the left eye and the right eye.

[0189] The parameters used to determine the confidence value at steps 1004 and 1005 may include all parameters AQ listed in Table 1 below, or may include a subset of the parameters AQ.

[0190] Table 1

[0191]

[0192]

[0193] The parameters used to determine the confidence value at steps 1004 and 1005 may, for example, include the following parameters:

[0194] At least one of the parameters AQ

[0195] At least two of the parameters AQ

[0196] At least three of the parameters AQ

[0197] At least four of the parameters AQ

[0198] At least five of the parameters AQ

[0199] At least one of A and parameter AQ

[0200] At least one of B and parameters AQ

[0201] At least one of H and parameter AQ

[0202] A and B

[0203] A and H

[0204] A, B and H

[0205] The first confidence value and the second confidence value may be calculated in different ways. The confidence value of an eye (such as a left eye and / or a right eye) may, for example, be calculated as a weighted average of different parameters calculated for that eye. A specific example will now be described. In this example, parameters A, B and H are used.

[0206] refer to Figure 3 The calculation of parameter A related to the position of the glint relative to the cornea is described. In this example, the value of parameter A is assigned to the left eye and the right eye independently of each other. If the glint is in the spherical region 311 of the cornea 310, parameter A is assigned a value of 1. If the glint is in the non-spherical region 312 of the cornea 310, parameter A is assigned a value that gradually decreases from 1 to 0 as the glint moves closer to the edge 315 of the cornea 310. When the glint is at the sclera 320, parameter A is assigned a value of 0.

[0207] The calculation of parameter B associated with pupil / iris contrast can be performed as follows. In this example, values ​​of parameter B are assigned to the left eye and the right eye independently of each other. When the pupil intensity in a bright pupil image is not greater than 1.3 times the iris intensity, the value assigned to parameter B is 0. When the pupil intensity in a bright pupil image increases to 2 times the iris intensity, the value assigned to parameter B increases linearly from 0 to 1. For higher contrast values, the value assigned to parameter B remains at 1. When the iris intensity in a dark pupil image is not greater than 1.3 times the pupil intensity, the value assigned to parameter B is 0. When the iris intensity in a dark pupil image increases to 2 times the pupil intensity, the value assigned to parameter B increases linearly from 0 to 1. For higher contrast values, the value assigned to parameter B remains at 1.

[0208] The calculation of the parameter H related to the number of pupil edge pixels may be performed as follows. In this example, the values ​​of parameter H are assigned to the left and right eyes relative to each other (i.e., the values ​​of parameter H assigned to the left and right eyes are not independent of each other). The value assigned to the left eye may be formed as the number of pupil edge pixels for the left eye divided by the number of pupil edge pixels for the right eye:

[0209]

[0210] Similarly, the value assigned to the right eye can be formed as the number of pupil edge pixels for the right eye divided by the number of pupil edge pixels for the left eye:

[0211]

[0212] The confidence value for the left eye can then be calculated as:

[0213]

[0214] Similarly, the confidence of the right eye can be calculated as:

[0215]

[0216] As can be seen from the equation above, the confidence values ​​for the left and right eyes are normalized so that the sum of the confidence values ​​is 1. This makes the confidence values ​​suitable as weights if a weighted average of gaze tracking data (such as gaze direction or gaze point) for the left and right eyes is to be formed.

[0217] As shown in the above examples, the value of parameter AQ can be calculated independently of each other for the left eye and the right eye (such as parameters A and B in the above examples), or can be calculated relative to each other for the left eye and the right eye (such as parameter H in the above examples).

[0218] It should be appreciated that the example given above is just one of many different ways to calculate a confidence value using the parameters from Table 1 above.

[0219] The step 1006 of determining the final gaze direction may, for example, include forming a weighted combination of the gaze direction of the left eye and the gaze direction of the right eye. When forming the weighted combination, the gaze directions of the respective eyes are weighted by the respective associated confidence values. In other words, the gaze direction of the right eye is weighted by the right eye confidence value, and the gaze direction of the left eye is weighted by the left eye confidence value. In this way, a combined (or final) gaze direction is obtained as a weighted combination of the estimated gaze directions of the left eye and the right eye.

[0220] Because the weighted combination takes into account one or more of the factors AQ for both the left eye and the right eye, the negative impact of errors present in the gaze tracking data (or the determined gaze direction) of only one eye can be more effectively mitigated compared to the case where the left eye and right eye gaze tracking data (or gaze direction) are combined without taking these factors into account.

[0221] The confidence values ​​for the left eye and the right eye determined at steps 1004 and 1005 may be output by the eye tracking system 100, for example, together with the gaze tracking data (or gaze direction) for the left eye and the right eye. Fig.12 As described, a system receiving such a signal can then itself determine (or calculate) the combined gaze direction.

[0222] Rather than determining 1004 both the first confidence value and 1005 the second confidence value based on one or more of the parameters AQ, such parameters AQ may be output by the eye tracking system 100 together with the gaze tracking data (or gaze directions) for the left and right eyes. Fig.11 As described, a system receiving such a signal can then itself determine (or calculate) confidence values ​​for the left and right eyes, and then determine (or calculate) a combined gaze direction.

[0223] Fig.11 is a flowchart of method 1100, which can be obtained from the above reference Figure 1 The gaze tracking system 100 described above is executed by a system that receives data. The method 1100 may be executed by, for example, an operating system or one or more processors.

[0224] The method 1100 includes receiving 1101 gaze tracking data indicating a gaze direction and / or a gaze point of a left eye, and receiving 1102 gaze tracking data indicating a gaze direction and / or a gaze point of a right eye.

[0225] Method 1100 includes receiving 1103 a plurality of parameters indicating different factors of reliability of gaze tracking data for a left eye, and receiving 1104 a plurality of parameters indicating different factors of reliability of gaze tracking data for a right eye. The plurality of parameters received in steps 1103 and 1104 may be any of the parameters AQ in Table 1 above.

[0226] Method 1100 includes calculating 1105 (or determining) a combined reliability parameter (or a first confidence value) of gaze tracking data for the left eye based on at least one of the received parameters associated with the left eye, and calculating 1106 a combined reliability parameter (or a second confidence value) of gaze tracking data for the right eye based on at least one of the received parameters associated with the right eye. Steps 1105 and 1106 of method 1100 may be, for example, similar to those described above with reference to Fig.10 Steps 1004 and 1005 of the described method 1000 are identical.

[0227] A system performing method 1100 can, for example, select which of the parameters received in steps 1103 and 1104 to use when calculating confidence values ​​for the left eye and the right eye in steps 1105 and 1106. For example, any eye tracking system 100 may provide all parameters AQ, but a system performing method 1100 may choose to use only a subset of these parameters, such as parameters A, B, and H, when forming confidence values ​​for the left eye and the right eye.

[0228] Optionally, method 1100 may include forming 1107 (or determining) a weighted combination of gaze tracking data for the left eye (or the determined gaze direction) and gaze tracking data for the right eye (or the determined gaze direction). When forming the weighted combination, the gaze tracking data for the respective eyes are weighted based on the respective associated confidence values. Step 1107 of method 1100 may be, for example, the same as described above with reference to Fig.10 The same is true for step 1006 of the described method 1000. Step 1107 may, for example, generate a combined gaze direction or a combined gaze point.

[0229] Fig.12is a flowchart of method 1200, which can be obtained from the above reference Figure 1 The method 1200 may be performed by a system that receives data from the gaze tracking system 100. The method 1200 may be performed by, for example, an operating system or one or more processors.

[0230] As mentioned above Fig.11 The method 1200 comprises receiving 1101 gaze tracking data indicating a gaze direction and / or a gaze point of a left eye, and receiving 1102 gaze tracking data indicating a gaze direction and / or a gaze point of a right eye.

[0231] Instead of receiving parameters from Table 1 above, method 1200 includes receiving 1203 a confidence value (or combined reliability parameter) for gaze tracking data of the left eye (e.g., calculated in step 1004 of method 1000), and receiving 1204 a confidence value (or combined reliability parameter) for gaze tracking data of the right eye (e.g., calculated in step 1005 of method 1000).

[0232] As mentioned above Fig.11 As with method 1100 described above, method 1200 includes forming 1107 (or determining, or calculating) a weighted combination of gaze tracking data (or gaze direction) for the left eye and gaze tracking data (or gaze direction) for the right eye.

[0233] Other matters

[0234] Those skilled in the art will recognize that the present invention is by no means limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims. For example, those skilled in the art will recognize that the eye / gaze tracking method described herein may be implemented by Figure 1 Many other eye / gaze tracking systems (e.g., using multiple illuminators and multiple cameras) other than the example eye / gaze tracking system 100 shown may be implemented. It should be understood that any other combination of parameters AQ in Table 1, other than those explicitly mentioned herein, may also be used to determine the confidence values ​​(or combined reliability parameters) for the left and right eyes.

[0235] In addition, by studying the drawings, the disclosure and the appended claims, those skilled in the art can understand and implement changes to the disclosed embodiments when implementing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The division of tasks between the functional units mentioned in the present disclosure does not necessarily correspond to the division of different physical units; on the contrary, one physical component can have multiple functions, and one task can be performed in a distributed manner by multiple physical components that cooperate. The computer program can be stored / distributed on a suitable non-transitory medium, such as an optical storage medium or solid-state medium provided with or as part of other hardware, but can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. The fact that certain measures / features are stated in mutually different dependent claims does not mean that the combination of these measures / features cannot be used to advantage. The method steps are not necessarily performed in the order in which they appear in the claims or in the embodiments described herein, unless a specific order is explicitly described. Any figure mark in the claims should not be interpreted as limiting the scope.

[0236] Special computer systems

[0237] Fig.13 1 is a block diagram illustrating a special purpose computer system 1300 in which embodiments of the present invention may be implemented. The example illustrates that, for example, a special purpose computer system 1300 may be used in whole, in part, or with various modifications to provide the functionality of the various components described herein.

[0238] The special-purpose computer system 1300 is shown to include hardware elements that can be electrically connected via a bus 1390. The hardware elements can include one or more central processing units 1310, one or more input devices 1320 (e.g., a mouse, keyboard, eye tracking device, etc.), and one or more output devices 1330 (e.g., a display device, a printer, etc.). The special-purpose computer system 1300 can also include one or more storage devices 1340. By way of example, the storage device 1340 can be a disk drive, an optical storage device, a solid-state storage device such as a random access memory (RAM) and / or a read-only memory (ROM), which can be programmable, flash-updatable, and the like.

[0239] The special-purpose computer system 1300 may also include a computer-readable storage medium reader 1350, a communication system 1360 (e.g., a modem, a network card (wireless or wired), an infrared communication device, a Bluetooth TMDevice, cellular communication device, etc.), and working memory 1380, which may include RAM and ROM devices as described above. In some embodiments, the special-purpose computer system 1300 may also include a processing acceleration unit 1370, which may include a digital signal processor, a special-purpose processor, etc.

[0240] Computer-readable storage media reader 1350 may further be connected to computer-readable storage media, which together (and optionally, in conjunction with storage device 1340) broadly represent remote, local, fixed and / or removable storage devices plus storage media for temporarily and / or more permanently containing computer-readable information. Communication system 1360 may allow data to be exchanged with the aforementioned networks, systems, computers and / or other components.

[0241] The special purpose computer system 1300 may also include software elements, shown as currently located in the working memory 1380, including an operating system 1384 and / or other code 1388. It should be understood that the optional embodiments of the special purpose computer system 1300 may have many variations different from the above-described embodiments. For example, customized hardware may also be used and / or specific elements may be implemented in hardware, software (including portable software, such as applets), or both hardware and software. In addition, connections with other computing devices (such as network input / output and data acquisition devices) are also possible.

[0242] The software of the special purpose computer system 1300 may include code 1388 for implementing any or all of the functionality of the various elements of the architecture described herein. For example, software stored on and / or executed by a special purpose computer system such as the special purpose computer system 1300 may provide the functionality of components of the present invention such as those described above. Methods that may be implemented by software on some of these components have been discussed in more detail above.

[0243] List of some exemplary embodiments

[0244] Embodiment 1: A method (1000) comprises: obtaining (1001) one or more images of a left eye and one or more images of a right eye; calculating (1002) gaze tracking data of the left eye indicating an estimated gaze direction of the left eye based on the one or more images of the left eye; calculating (1003) gaze tracking data of the right eye indicating an estimated gaze direction of the right eye based on the one or more images of the right eye; calculating a plurality of parameters indicating different factors of reliability of the gaze tracking data indicating the left eye based on the one or more images of the left eye; and calculating a plurality of parameters indicating different factors of reliability of the gaze tracking data indicating the right eye based on the one or more images of the right eye.

[0245] Embodiment 2: The method according to Embodiment 1 further comprises: calculating (1004) a combined reliability parameter of gaze tracking data for the left eye based on multiple parameters calculated for the left eye; and calculating (1005) a combined reliability parameter of gaze tracking data for the right eye based on multiple parameters calculated for the right eye.

[0246] Embodiment 3: The method according to embodiment 2 further comprises: calculating (1006) a combined gaze direction or gaze point based on the gaze tracking data of the left eye and the right eye and the associated combined reliability parameter.

[0247] Embodiment 4: A method according to embodiment 3, wherein calculating a combined gaze direction or gaze point comprises forming a weighted combination of gaze tracking data for a left eye and gaze tracking data for a right eye, wherein the gaze tracking data for the corresponding eyes are weighted based on corresponding associated combination reliability parameters.

[0248] Embodiment 5: A method according to any of the preceding embodiments, wherein calculating gaze tracking data of an eye (200) comprises: estimating the position of the pupil (210) of the eye and the position of the flash (220) at the eye based on one or more images of the eye; and calculating the gaze tracking data of the eye based on the estimated pupil position and flash position of the eye.

[0249] Embodiment 6: A method according to any of the preceding embodiments, wherein the multiple parameters calculated for the eye (300) include: the position of the glitter (220) at the eye relative to the cornea (310) of the eye; the distance between the glitter (220) at the eye and the center (313) of the cornea (310) of the eye; the position of the glitter (220) at the eye relative to the pupil (210) of the eye; the distance between the glitter (220) at the eye and the center (350) of the pupil of the eye; and / or the position of the glitter (610) at the eye relative to the edge (620) of the pupil (630) of the eye.

[0250] Embodiment 7: The method according to embodiment 6 comprises: calculating (1004) a combined reliability parameter of gaze tracking data for the left eye based on multiple parameters calculated for the left eye; and calculating (1005) a combined reliability parameter of gaze tracking data for the right eye based on multiple parameters calculated for the right eye, wherein calculating the combined reliability parameter of the gaze tracking data of the eyes comprises: associating a first value of the distance between the flash at the eye and the center of the eye's cornea with a lower reliability compared to the reliability associated with a second value of the distance, wherein the first value of the distance is higher than the second value of the distance; and / or associating a situation where the flash at the eye is located at the edge of the pupil of the eye with a lower reliability compared to the reliability associated with other positions of the flash relative to the pupil.

[0251] Embodiment 8: The method according to any of the preceding embodiments, wherein the multiple parameters calculated for the eye 510 include: a contrast level between the pupil 511 of the eye and the iris of the eye.

[0252] Embodiment 9: The method according to embodiment 8 comprises: calculating (1004) a combined reliability parameter of the gaze tracking data of the left eye based on a plurality of parameters calculated for the left eye; and calculating (1005) a combined reliability parameter of the gaze tracking data of the right eye based on a plurality of parameters calculated for the right eye, wherein calculating the combined reliability parameter of the gaze tracking data of the eyes comprises: associating a first value of the contrast level with a lower reliability compared to the reliability associated with a second value of the contrast level, wherein the first value of the contrast level is lower than the second value of the contrast level.

[0253] Embodiment 10: A method according to any of the preceding embodiments, wherein the multiple parameters calculated for the eye (200) include: the intensity of the flash (220) used to calculate the gaze tracking data of the eye; the shape of the flash (220) used to calculate the gaze tracking data of the eye; and / or multiple flashes (220) detected in one or more images of the eye used to obtain the gaze tracking data of the eye.

[0254] Embodiment 11: The method according to embodiment 10 comprises: calculating (1004) combined reliability parameters of gaze tracking data of the left eye based on multiple parameters calculated for the left eye; and calculating (1005) combined reliability parameters of gaze tracking data of the right eye based on multiple parameters calculated for the right eye, wherein calculating the combined reliability parameters of the gaze tracking data of the eyes comprises: estimating the similarity between the shape and the expected flash shape; and associating a first level of similarity between the shapes with a lower reliability, wherein the lower reliability is compared to the reliability associated with a second level of similarity between the shapes, wherein the first level of similarity is lower than the second level of similarity.

[0255] Embodiment 12: The method according to any one of embodiments 10-11 comprises: calculating (1004) combined reliability parameters of gaze tracking data for the left eye based on parameters calculated for the left eye; and calculating (1005) combined reliability parameters of gaze tracking data for the right eye based on parameters calculated for the right eye, wherein calculating the combined reliability parameters of the gaze tracking data of the eyes comprises: comparing the number of flashes detected in one or more images of the eye used to obtain the gaze tracking data of the eyes with the expected number of flashes; and associating a situation in which the number of detected flashes is consistent with the expected number of flashes with a higher reliability, wherein the higher reliability is compared to the reliability associated with a situation in which the number of detected flashes deviates from the expected number of flashes.

[0256] Embodiment 13: A method according to any of the preceding embodiments, wherein the multiple parameters calculated for the eye include: parameters indicating the presence of reflections from glasses in one or more images of the eye; and / or parameters indicating whether a certain area of ​​the eye is at least partially obscured in one or more images of the eye.

[0257] Embodiment 14: A method according to any of the preceding embodiments, wherein the multiple parameters calculated for the eye (600) include: parameters indicating the number of pupil edge pixels in one or more images of the eye, wherein these pupil edge pixels are image pixels positioned along one or more portions of an edge (620) of the pupil (630) that are visible in one or more images of the eye.

[0258] Embodiment 15: The method according to embodiment 14 comprises: calculating (1004) combined reliability parameters of gaze tracking data of the left eye based on multiple parameters calculated for the left eye; and calculating (1005) combined reliability parameters of gaze tracking data of the right eye based on the parameters calculated for the right eye, wherein calculating the combined reliability parameters of the gaze tracking data of the eyes comprises: associating a first number of pupil edge pixels with a reliability lower than the reliability associated with a second number of pupil edge pixels, wherein the first number of pupil edge pixels is lower than the second number of pupil edge pixels.

[0259] Embodiment 16: The method according to any of the preceding embodiments, wherein the plurality of parameters calculated for the eye include: a parameter indicating the size of a residual remaining after calibration of an eye model used to calculate gaze tracking data for the eye.

[0260] Embodiment 17: A method according to any of the preceding embodiments, comprising, for each of a left eye and a right eye: estimating, based on one or more images of the eye, a position of a first flash at the eye caused by reflection of light from a first illuminator (111); predicting, based on the position of the first flash and based on knowledge of how a second illuminator (112) is arranged relative to the first illuminator, a position of a second flash at the eye caused by reflection of light from a second illuminator; and detecting, in one or more images of the eye, a position of the second flash at the eye caused by reflection of light from the second illuminator, wherein the plurality of parameters calculated for the eye include: a difference between the predicted position and the detected position of the second flash.

[0261] Embodiment 18: A method according to any of the preceding embodiments, comprising: performing gaze tracking for a left eye and a right eye separately for a series of images; monitoring the variability of a gaze tracking position estimated for the left eye; and monitoring the variability of a gaze tracking position estimated for the right eye, wherein the multiple parameters calculated for the eyes include: a parameter indicating the monitored variability of the gaze tracking position estimated for the eyes.

[0262] Embodiment 19: A method according to any of the preceding embodiments, comprising: performing gaze tracking for a left eye for a series of images; estimating the distance to the left eye for a series of images for gaze tracking of the left eye; estimating the noise level in the estimated distance to the left eye; performing right eye gaze tracking for a series of image sequences; estimating the distance to the right eye for a series of images for gaze tracking of the right eye; and estimating the noise level in the estimated distance to the right eye, wherein the multiple parameters calculated for the eye include: a parameter indicating the noise level in the estimated distance to the eye.

[0263] Embodiment 20: A method according to any of the preceding embodiments, wherein the plurality of parameters calculated for the eye comprises: a parameter indicating a distance between a reference position and an average of gaze tracking positions estimated for the eye.

[0264] Embodiment 21: A method according to any of the preceding embodiments, wherein the plurality of parameters calculated for an eye include: a parameter indicating whether the eye is dominant relative to the other eye.

[0265] Embodiment 22: A computer program product comprising one or more computer readable media having instructions for executing the method according to any one of embodiments 1-21.

[0266] Embodiment 23: A system (100), comprising: one or more processors (120) configured to perform the method of any one of embodiments 1-21.

[0267] Embodiment 24: The system according to embodiment 23 further comprises: one or more illuminators (111, 112) for illuminating the eye; and one or more cameras (112) for capturing images of the eye.

[0268] Embodiment 25: A method (1100) comprising: receiving (1101) gaze tracking data indicating a gaze direction of a left eye; receiving (1102) gaze tracking data indicating a gaze direction of a right eye; receiving (1103) multiple parameters indicating different factors indicating the reliability of the gaze tracking data of the left eye; receiving (1104) multiple parameters indicating different factors indicating the reliability of the gaze tracking data of the right eye; calculating (1105) a combined reliability parameter for the gaze tracking data for the left eye based on at least one of the parameters associated with the left eye; and calculating (1106) a combined reliability parameter for the gaze tracking data for the right eye based on at least one of the parameters associated with the right eye.

[0269] Embodiment 26: The method according to embodiment 25 further comprises: calculating (1107) a combined gaze direction or gaze point based on the gaze tracking data of the left eye and the right eye and the associated combined reliability parameter.

[0270] Embodiment 27: A method according to embodiment 26, wherein calculating a combined gaze direction or gaze point comprises forming a weighted combination of gaze tracking data for a left eye and gaze tracking data for a right eye, wherein the gaze tracking data for the corresponding eyes are weighted based on a reliability parameter of the corresponding associated combination.

[0271] Embodiment 28: A method according to any one of embodiments 25-27, wherein calculating a combined reliability parameter of the eye's gaze tracking data based on at least one of the received parameters associated with the eye comprises: selecting one or more of a plurality of parameters associated with the eye; and calculating a combined reliability parameter of the eye's gaze tracking data based on the selected one or more parameters.

[0272] Embodiment 29: The method according to any one of Embodiments 25-28, wherein the plurality of parameters respectively representing different factors indicating the reliability of the gaze tracking data of the left eye and the right eye include one or more of the parameters AQ listed in Table 1.

[0273] Embodiment 30: A method (1200), comprising: receiving (1101) gaze tracking data indicating a gaze direction and / or gaze point of a left eye; receiving (1102) gaze tracking data indicating a gaze direction and / or gaze point of a right eye; receiving (1203) combined reliability parameters of the gaze tracking data of the left eye; receiving (1204) combined reliability parameters of the gaze tracking data of the right eye; and calculating (1107) a combined gaze direction or gaze point based on the gaze tracking data of the left eye and the right eye and the associated combined reliability parameters.

[0274] Embodiment 31: A method according to embodiment 30, wherein calculating a combined gaze direction or gaze point comprises forming a weighted combination of gaze tracking data for a left eye and gaze tracking data for a right eye, wherein the gaze tracking data for the corresponding eyes are weighted based on corresponding associated combined reliability parameters.

[0275] Embodiment 32: A computer program product comprising one or more computer readable media having instructions for performing the method of any of Embodiments 25-31.

[0276] Embodiment 33: A system comprising: one or more processors configured to perform the method according to any one of embodiments 25-31.

Claims

1. An eye / gaze tracking system (100), comprising a circuit (120), wherein the circuit is configured to: Obtaining one or more images of a left eye (412; 520) of a user (410) and one or more images of a right eye (414; 510) of the user; determining a gaze direction of the left eye of the user based on the obtained at least one image of the left eye; determining a gaze direction of the right eye of the user based on the obtained at least one image of the right eye; determining a first confidence value based on the obtained one or more images of the left eye, the first confidence value representing an indication of the reliability of the determined gaze direction of the left eye; determining a second confidence value based on the obtained one or more images of the right eye, the second confidence value representing an indication of the reliability of the determined gaze direction of the right eye; and determining a final gaze direction based at least in part on the first confidence value and the second confidence value, Wherein determining the first confidence value based on the obtained one or more images of the left eye further comprises: determining the first confidence value based on a contrast between a pupil (210; 630) and an iris of the left eye in the obtained one or more images of the left eye; and associating a first level of the contrast with a lower confidence value than a confidence value associated with a second level of the contrast, wherein the first level of contrast is lower than the second level of contrast; and Determining the second confidence value based on the obtained one or more images of the right eye further comprises: determining the second confidence value based on a contrast between a pupil (210; 511; 630) and an iris of the right eye in the obtained one or more images of the right eye; and A first level of the contrast is associated with a lower confidence value than a confidence value associated with a second level of the contrast, wherein the first level of contrast is lower than the second level of contrast.

2. An eye / gaze tracking system according to claim 1, wherein determining the final gaze direction is further based on the determined gaze direction of the left eye and the determined gaze direction of the right eye, or based on a weighted combination of the determined gaze direction of the left eye and the determined gaze direction of the right eye.

3. The eye / gaze tracking system of claim 1 , wherein the circuit is further configured to: determining one or more flash locations in a predetermined area of ​​the left eye based on the obtained one or more images of the left eye; and Based on the obtained one or more images of the right eye, one or more flash positions in a predetermined area of ​​the right eye are determined, wherein the predetermined area of ​​the left eye or the right eye preferably includes at least one of the following: a first region extending from a center (313) of the cornea to an edge (314) of a spherical region (311; 314) of the cornea (310); a second region extending from the edge of the spherical region of the cornea to an edge (315) of the cornea (310); or A third area located outside the edge of the cornea, the third area is preferably located at the sclera, and the center of the cornea is preferably a position at the spherical area of ​​the cornea, the position being the intersection of a virtual line extending from the center of the eyeball (330) through the center of the pupil (350) and the spherical area of ​​the cornea.

4. The eye / gaze tracking system of claim 1 , wherein: Determining the first confidence value based on the obtained one or more images of the left eye includes determining the first confidence value based on a distance between a corneal center (313) of the left eye and one or more glints (411) at the left eye; and Determining the second confidence value based on the one or more images obtained of the right eye includes determining the second confidence value based on a distance between a corneal center (313) of the right eye and one or more glints (413) at the right eye, in: Determining the first confidence value based on the obtained one or more images of the left eye preferably further comprises: associating a glint located in a first area of ​​the left eye with a higher confidence value than a confidence value associated with a glint located in a second area of ​​the left eye, wherein the first area of ​​the left eye extends from a center of the cornea (313) to an edge (315) of a spherical area (311; 314) of the cornea, and wherein the second area of ​​the left eye extends from the edge of the spherical area of ​​the cornea to the edge of the cornea; and associating the glint located in the second area of ​​the left eye with a higher confidence value than a confidence value associated with the glint located in a third area of ​​the left eye, wherein the third area of ​​the left eye is located outside the edge (315) of the cornea (310); and Determining the second confidence value based on the obtained one or more images of the right eye further comprises: associating a glint located in a first area of ​​the right eye with a higher confidence value than a confidence value associated with a glint located in a second area of ​​the right eye, wherein the first area of ​​the right eye extends from a center of the cornea (313) to an edge (314) of a spherical area (311; 314) of the cornea, and wherein the second area of ​​the right eye extends from the edge of the spherical area of ​​the cornea to an edge (315) of the cornea (310); and Glitter located in the second area of ​​the right eye is associated with a higher confidence value than a confidence value associated with glint located in a third area of ​​the right eye, wherein the third area of ​​the right eye is located outside the edge of the cornea.

5. The eye / gaze tracking system of claim 1 , wherein the circuit is further configured to: The pupil center (350) of the left eye, the pupil center (350) of the right eye, the eyeball center (330) of the left eye, and the eyeball center (330) of the right eye are determined.

6. The eye / gaze tracking system of claim 1, wherein: Determining the first confidence value based on the obtained one or more images of the left eye further comprises: determining the first confidence value based on a determined first concentration of the gaze position of the left eye relative to a predetermined position at a display device, and Determining the second confidence value based on the obtained one or more images of the right eye further comprises: The second confidence value is determined based on a second degree of concentration of the determined gaze position of the right eye relative to a predetermined position on the display device.

7. The eye / gaze tracking system of claim 1 , wherein: Determining the first confidence value based on the obtained one or more images of the left eye further comprises: determining the first confidence value based on a first distance between a predetermined position at a display device and an average of the determined gaze positions of the left eye, and Determining the second confidence value based on the obtained one or more images of the right eye further comprises: The second confidence value is determined based on a second distance between a predetermined position at the display device and an average of the determined gaze positions of the right eye.

8. The eye / gaze tracking system of claim 1 , wherein the determination of the first confidence value or the second confidence value, respectively, is based on at least one of: a flash intensity at the left eye or the right eye, respectively; a flash shape at the left eye or the right eye, respectively; or The number of flashes at the left eye or the right eye, respectively.

9. The eye / gaze tracking system of claim 1 , wherein: Determining the first confidence value based on the obtained one or more images of the left eye further comprises: determining the first confidence value based on an evaluation of a similarity between an expected flash shape and a shape of a flash detected at the left eye; and associating a first level of glitter shape similarity between the shapes with a lower confidence value than a confidence value associated with a second level of glitter shape similarity between the shapes, wherein the first level of glitter shape similarity is lower than the second level of glitter shape similarity; and Determining the second confidence value based on the obtained one or more images of the right eye further comprises: determining the second confidence value based on an evaluation of a similarity between an expected flash shape and a shape of a flash detected at the right eye; and A first level of flash shape similarity between the shapes is associated with a lower confidence value than a confidence value associated with a second level of flash shape similarity between the shapes, wherein the first level of flash shape similarity is lower than the second level of flash shape similarity.

10. The eye / gaze tracking system of claim 1, wherein: Determining the first confidence value based on the obtained one or more images of the left eye further comprises: The first confidence value is determined based on a comparison between the number of flashes detected at the left eye and a predetermined number of flashes, wherein associating a situation where the detected number of flashes is consistent with the predetermined number of flashes with a higher confidence value than a situation where the detected number of flashes deviates from the predetermined number of flashes; and Determining the second confidence value based on the obtained one or more images of the right eye further comprises: The second confidence value is determined based on a comparison between the number of flashes detected at the right eye and a predetermined number of flashes, wherein A higher confidence value is associated with a situation where the detected number of flashes is consistent with the predetermined number of flashes than a situation where the detected number of flashes deviates from the predetermined number of flashes.

11. The eye / gaze tracking system of claim 1 , wherein the determination of the first confidence value or the second confidence value is based at least in part on: a parameter for the left eye or the right eye, respectively, indicating a number of pupil edge pixels in one or more acquired images of the eye, wherein the pupil edge pixels are image pixels located along one or more portions of an edge of the pupil visible in one or more acquired images of the left eye or the right eye, respectively; and A first number of pupil edge pixels is associated with a lower confidence value than a confidence value associated with a second number of pupil edge pixels, wherein the first number of pupil edge pixels is lower than the second number of pupil edge pixels.

12. The eye / gaze tracking system of claim 1 , wherein the circuit is further configured to perform the following actions for the left eye and the right eye, respectively: determining a first flash position at the eye caused by reflection of light from a first illuminator (111) based on one or more images of the eye; predicting a second flash position at the eye caused by reflection of light from the second illuminator based on the first flash position and a spatial relationship between the first illuminator and a second illuminator (112); and determining a second glint position at the eye caused by reflection of light from the second illuminator based on the one or more images of the eye, Wherein the determination of the first confidence value or the second confidence value, respectively, is preferably at least partially based on: The difference between the predicted second flash position and the determined second flash position.

13. The eye / gaze tracking system of claim 1 , wherein the determination of the first confidence value and the second confidence value are each based at least in part on: A parameter indicating whether the eye is dominant relative to the other eye.

14. One or more non-transitory computer-readable storage media storing computer-executable instructions that, when executed by a computing system implementing eye / gaze data processing, cause the computing system to perform a method (1000), the method comprising: Obtaining (1001) one or more images of a left eye (412; 520) of a user (410) and one or more images of a right eye (414; 510) of the user; determining (1002) a gaze direction of the left eye of the user based on the obtained at least one image of the left eye; determining (1003) a gaze direction of the right eye of the user based on the obtained at least one image of the right eye; determining (1004) a first confidence value based on the obtained one or more images of the left eye, the first confidence value representing an indication of the reliability of the determined gaze direction of the left eye; determining (1005) a second confidence value based on the obtained one or more images of the right eye, the second confidence value representing an indication of the reliability of the determined gaze direction of the right eye; and determining (1006) a final gaze direction based at least in part on the first confidence value and the second confidence value, Wherein, determining the first confidence value based on the obtained one or more images of the left eye further comprises: determining the first confidence value based on a contrast between a pupil and an iris of the left eye in the obtained one or more images of the left eye; and associating a first level of the contrast with a lower confidence value than a confidence value associated with a second level of the contrast, wherein the first level of contrast is lower than the second level of contrast; and Determining the second confidence value based on the obtained one or more images of the right eye further comprises: determining the second confidence value based on a contrast between a pupil and an iris of the right eye in the obtained one or more images of the right eye; and A first level of the contrast is associated with a lower confidence value than a confidence value associated with a second level of the contrast, wherein the first level of contrast is lower than the second level of contrast.

15. A method (1000) performed by a computing system, comprising: Acquiring (1001) one or more images of a left eye (412; 520) of a user (410) and one or more images of a right eye of the user; determining (1002) a gaze direction of the left eye of the user based on the obtained at least one image of the left eye; determining (1003) a gaze direction of the right eye of the user based on the obtained at least one image of the right eye; determining (1004) a first confidence value based on the obtained one or more images of the left eye, the first confidence value representing an indication of the reliability of the determined gaze direction of the left eye; determining (1005) a second confidence value based on the obtained one or more images of the right eye, the second confidence value representing an indication of the reliability of the determined gaze direction of the right eye; and determining (1006) a final gaze direction based at least in part on the first confidence value and the second confidence value, Wherein, determining the first confidence value based on the obtained one or more images of the left eye further comprises: determining the first confidence value based on a contrast between a pupil and an iris of the left eye in the obtained one or more images of the left eye; and associating a first level of the contrast with a lower confidence value than a confidence value associated with a second level of the contrast, wherein the first level of contrast is lower than the second level of contrast; and Determining the second confidence value based on the obtained one or more images of the right eye further comprises: determining the second confidence value based on a contrast between a pupil and an iris of the right eye in the obtained one or more images of the right eye; and A first level of the contrast is associated with a lower confidence value than a confidence value associated with a second level of the contrast, wherein the first level of contrast is lower than the second level of contrast.

Images