Method and system for gaze tracking using primary corneal reflections

Abstract

Methods and systems are described for estimating a gaze direction of an individual using, as input, a location of a single corneal reflex, a location of a pupil center, and location information of an individual head. An image is acquired from an infrared (IR) camera, and the position of corneal reflections in the image is estimated. Position information of the user's head is determined from the image and a 3D position of the corneal center of the user's eyes is estimated from the position of the corneal reflections, the position of the IR light source, and the position information of the individual's head. A position of a 3D pupil center of the user's eye is then estimated, and a 3D gaze vector representing a gaze direction is estimated from the position of the corneal center and the position of the 3D pupil center.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method and device for estimating a sight line direction. Background Art

[0002] Gaze tracking is a useful indicator of human visual attention and has a wide range of applications in areas such as human-computer interaction, automotive safety, medical diagnostics, and accessibility interfaces. Eye tracking or gaze estimation systems track eye movements and estimate the viewpoint on a display or in the surrounding environment. To exploit the benefits of gaze tracking, the monitoring system should ideally operate with high accuracy, be robust to head movements, and be minimally affected by image noise.

[0003] A commonly used method for gaze estimation is video-based eye tracking. In such a system, a camera is used to capture eye images. The camera can be an infrared (IR) camera (capturing IR data) or an RGB camera (capturing visible spectrum data). Typically, an IR camera is used in conjunction with a light-emitting diode (LED) for eye illumination, as this allows for highly accurate gaze tracking. In such IR-based systems, two or more IR LEDs are typically used to illuminate the eye. Using two or more IR LEDs for gaze estimation builds redundancy into the system, ensuring that at least two corneal reflections are generated in the captured image within a range of eye positions. However, there is a benefit to using the minimum number of IR LEDs that can be used for eye tracking, as hardware circuit complexity and image noise can increase with the number of LEDs, negatively impacting the performance of the gaze estimation system.

[0004] Therefore, it would be useful to provide a solution for 3D line of sight estimation using a single corneal reflection. Summary of the invention

[0005] In various examples, the present invention describes methods and systems for estimating the gaze direction of an individual using the position of a single corneal reflection, the position of a pupil center, and the position information of the individual's head as input. Specifically, an image of a user including at least a portion of the image of the user's head is acquired from an infrared (IR) camera, and the 2D position of the corneal reflection is estimated in the image. The position information of the user's head is determined from the IR camera image. Then, the 3D position of the corneal center of the user's eye is estimated based on the 2D position of the corneal reflection, the position of an IR light source (e.g., an IR LED), and the position information of the user's head in the image. The 2D position of the pupil center of the user's eye is estimated in the IR camera image, and then the 3D position of the pupil center is estimated based on the 2D position of the pupil center and the 3D position of the corneal center. Finally, a 3D gaze vector representing the gaze direction is estimated based on the 3D position of the corneal center and the 3D position of the pupil center. The disclosed system can help overcome challenges associated with the gaze estimation performance of systems that require two or more corneal reflections, for example, under conditions of extreme head movement.

[0006] The disclosed systems and methods enable 3D line of sight estimation using a single corneal reflection. This enables improved line of sight estimation performance in conditions where it may be difficult to detect two or more corneal reflections (e.g., under conditions of extreme movement or during extreme head positions, among other possible situations).

[0007] In various examples, the present invention provides a technical effect of estimating the gaze direction in the form of a 3D gaze vector. Input obtained from an IR camera in the form of a face or eye image and position information of a person's head is input to a gaze estimation system to estimate the viewpoint on the screen or in the surrounding environment.

[0008] In some examples, the present invention provides the technical advantage of using only one corneal reflection instead of two or more corneal reflections to estimate the gaze direction.

[0009] Examples of the present invention can achieve improved gaze estimation performance under conditions of extreme head movement (e.g., where it may be difficult to accurately capture two or more corneal reflections in an image). Thus, requiring only one corneal reflection in the input image can reduce noise and data loss, for example, reducing the frequency of situations where the image cannot be used to calculate gaze direction. In an example, more than one IR LED may be included in an eye tracking system, however, images where only one corneal reflection is visible (e.g., at extreme head positions) can be used to calculate gaze direction, in which case they would not be usable for systems that require two or more corneal reflections.

[0010] In some examples, the present invention provides the technical advantage of tracking only one corneal reflection instead of multiple reflections, which means that the hardware configuration required to estimate the gaze direction is simpler, for example, the hardware circuit complexity associated with synchronizing the IR LED with the camera shutter is reduced. The system configuration may also be more flexible, for example, the IR LED can be placed in a wider range of positions. Similarly, the computing resources required for estimating the gaze direction are less complex.

[0011] In an exemplary aspect, the present invention describes a method. The method includes: acquiring an image of a user, the image including an image of the user's eye; estimating a position of a corneal reflection based on the image; determining position information of the user's head based on the image; and estimating a position of a corneal center of the user's eye based on the position of the corneal reflection and the position information.

[0012] Optionally, the image is acquired by using an IR camera and at least one IR LED, and the image is an IR camera image.

[0013] In the above exemplary aspect of the method, the method further comprises estimating the position of the 3D pupil center of the user's eye based on the IR camera image and the position of the cornea center.

[0014] In the above exemplary aspect of the method, the method further includes: estimating a sight vector representing a sight direction of the user based on the position of the cornea center and the position of the 3D pupil center.

[0015] In the above-mentioned exemplary aspect of the method, estimating the gaze vector representing the gaze direction of the user includes: estimating the optical axis of the user's eye based on the position of the cornea center and the position of the 3D pupil center; estimating the gaze vector based on the optical axis and multiple calibration parameters.

[0016] In any of the above exemplary aspects of the method, the position information of the user's head includes the user's head posture.

[0017] In the above-mentioned exemplary aspects of the method, obtaining the head posture of the user includes: estimating one or more facial key points corresponding to the face of the user based on the IR camera image; fitting the estimated one or more facial key points to a 3D face model; and estimating the head posture based on the 3D face model.

[0018] In any of the above exemplary aspects of the method, the IR camera is located at a distance from the user, and the IR camera image is a facial image.

[0019] In some exemplary aspects of the method, the IR camera image is an eye image, and the position information of the user's head is the distance from the user's eyes to the IR camera, and the distance from the user's eyes to the IR camera is obtained from a head-mounted device.

[0020] In an exemplary aspect of the method, the IR camera image of the user includes a bright pupil.

[0021] In some example aspects of the method, the estimated gaze vector is a 3D gaze vector.

[0022] In some aspects, the present invention describes an eye tracking system. The system includes: one or more processors; a memory storing machine executable instructions, when the instructions are executed by the processor device, the system performs the following operations: acquiring an image of a user, the image including an image of the user's eye; estimating the position of a corneal reflection based on the image; determining the position information of the user's head based on the image; estimating the position of the corneal center of the user's eye based on the position of the corneal reflection and the position information.

[0023] Optionally, the system further includes: an IR camera; a first IR LED; the image is acquired by the IR camera and the first IR LED; and the image is an IR camera image.

[0024] In some exemplary aspects, the present invention describes a non-transitory computer-readable medium having instructions stored thereon. When the instructions are executed by a processor, the processor is caused to perform any of the above exemplary aspects of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Reference will now be made by way of example to the accompanying drawings which show exemplary embodiments of the present application, in which:

[0026] Figure 1 is a schematic block diagram of an eye tracking system suitable for implementing the examples described herein;

[0027] Figure 2 A block diagram of an exemplary hardware structure of a computing system provided for an example of the present invention, the computing system being operable to implement a method for estimating a gaze vector representing a gaze direction;

[0028] Figure 3 A block diagram of an exemplary architecture for a line of sight estimation system provided for exemplary embodiments of the present invention;

[0029] Figure 4 A block diagram of an exemplary architecture for a head pose estimator provided for exemplary embodiments of the present invention;

[0030] Figure 5 A block diagram of an exemplary architecture for a line of sight estimator provided for exemplary embodiments of the present invention;

[0031] Fig. 6A An exemplary embodiment of an eye tracking system provided by an example of the present invention is shown;

[0032] Figure 6B and Figure 6C The example of the present invention provides a Fig. 6A An exemplary IR camera image acquired by an embodiment of the present invention;

[0033] Fig. 7A An exemplary embodiment of an eye tracking system provided by an example of the present invention is shown;

[0034] Figure 7B and Figure 7C The example of the present invention provides a Fig. 7A An exemplary IR camera image acquired by an embodiment of the present invention;

[0035] Fig. 8A An exemplary embodiment of an eye tracking system provided by an example of the present invention is shown;

[0036] Figure 8B , Figure 8C and Fig.8D The example of the present invention provides a Fig. 8A An exemplary IR camera image acquired by an embodiment of the present invention;

[0037] Fig. 9 Combinations provided for exemplary embodiments of the present invention Fig. 8A A block diagram of an exemplary architecture of a line of sight estimation system used in an embodiment of the present invention;

[0038] Fig.10 A flow chart of steps of an exemplary method for estimating a gaze vector representing a user's gaze direction is provided for examples of the present invention. DETAILED DESCRIPTION

[0039] The exemplary technical solutions of the present invention are described below in conjunction with the accompanying drawings.

[0040] In various examples, the present invention describes methods and systems for estimating the gaze direction of an individual using the position of a single corneal reflection, the position of a pupil center, and the position information of the individual's head as input. Specifically, an image of a user including at least a portion of the image of the user's head is acquired from an infrared (IR) camera, and the 2D position of the corneal reflection in the image is estimated. The position information of the user's head is determined from the IR camera image. Then, the 3D position of the corneal center of the user's eye is estimated based on the 2D position of the corneal reflection in the image, the position of an IR light source (e.g., an IR LED), and the position information of the user's head. The 2D position of the pupil center of the user's eye in the IR camera image is estimated, and then the 3D position of the pupil center is estimated based on the 2D position of the pupil center and the 3D position of the corneal center. Finally, a 3D gaze vector representing the gaze direction is estimated based on the 3D position of the corneal center and the 3D position of the pupil center. The disclosed system can help overcome challenges associated with the gaze estimation performance of systems that require two or more corneal reflections, for example, under conditions of extreme head movement.

[0041] To help understand the present invention, some existing technologies for gaze tracking are now discussed.

[0042] Existing camera-based eye tracking systems are typically used to estimate the gaze direction of a user by capturing an image and providing the image to a gaze estimation algorithm for estimating the gaze direction. The gaze estimate can be determined as a 2D viewpoint (e.g., on a display screen) or a 3D gaze vector. The 2D viewpoint can be estimated by intersecting the 3D gaze vector with the plane of the display screen. 3D gaze estimation systems provide additional benefits over 2D eye tracking systems because they can be used to track gaze direction in environments without a display screen (e.g., in a vehicle).

[0043] Typical hardware configurations for eye tracking systems include remote systems and head-mounted systems. In remote eye tracking systems, the hardware components, including the camera and any LEDs, are placed away from the user, while in head-mounted eye tracking systems, the hardware components are placed inside a head-mounted device (such as an augmented reality (AR) or virtual reality (VR) device) so that the camera and any LEDs are placed very close to the eyes. Cameras used in eye tracking systems may include infrared (IR) cameras (capturing IR data) or RGB cameras (capturing visible spectrum data). IR-based cameras and LEDs are generally preferred over visible light because IR light does not interfere with human vision and IR-based systems can operate in most environments, including at night. For these and other reasons, IR-based eye tracking systems operate with high accuracy.

[0044] The main purpose of IR LEDs in eye tracking systems is to provide uniform illumination on the captured image, thereby improving the signal to noise ratio (SNR) in the captured image. A high SNR means that the captured image is of good quality, thereby improving the performance of the gaze algorithm. Another important use of IR LEDs in eye tracking systems is to generate corneal reflections. Corneal reflections are virtual images of the reflection of the IR LEDs on the cornea of ​​the eye, formed behind the corneal surface and captured as flashes in the IR image. The position of the corneal reflection in the captured image is an important eye feature used by the latest gaze algorithms. The number of corneal reflections generated for the user's eyes depends on the number of IR LEDs, and various gaze algorithms require different numbers of corneal reflections in the image to determine the gaze direction.

[0045] Gaze estimation algorithms can be classified based on the type of algorithm used to estimate the viewpoint or gaze direction and the number of corneal reflections used in the process: (1) appearance-based, (2) feature-based, (3) geometric eye model-based, and (4) cross-ratio-based. Appearance-based methods use the appearance of the face or eyes in an image to learn a direct mapping between an input image and gaze direction or viewpoint. Appearance-based methods compute gaze positions on images captured by an eye tracking system by leveraging machine learning (ML) techniques. Appearance-based methods do not require any information about corneal reflections. As described in WuZhengyang et al., “Magiceyes: A large scale eye gaze estimation dataset for mixed reality” arXiv preprint arXiv:2003.08806 (2020), the best appearance-based methods can perform within 2 to 3 degrees of accuracy. The accuracy of appearance-based methods can be improved by retraining the ML network for each object, but this may not be practical. Appearance-based techniques also require large training datasets.

[0046] Feature-based methods use the spatial position of features extracted from facial images (e.g., a vector between a corneal reflection and the center of the pupil) to estimate gaze direction. An example of a feature-based method is described in Zhu Zhiwei and Qiang Ji, "Novel eye gaze tracking techniques under natural head movement," IEEE Transactions on Biomedical Engineering, 54, 12 (2007): 2246-2260. Estimating eye features from facial images is a challenging process, especially at extreme head angles. Feature-based methods can only estimate 2D gaze and require a display screen to be present in the system. An example of a cross-ratio-based method is described in Fan, Shuo, Jiannan Chi, and Jiahui Liu, "A Novel Cross-Ratio Based Gaze Estimation Method," 2022 International Conference on Machine Learning and Knowledge Engineering (MLKE), IEEE, 2022, which uses 4 or more corneal reflections and the center of the pupil to estimate gaze direction, but they are sensitive to head movement.

[0047] Geometric model-based methods use a 3D model of the eye to estimate the gaze direction (e.g., visual axis), where the visual axis of the eye is determined as the vector connecting the node of the eye and the fovea, and the viewpoint is the intersection of the visual axis of the eye and the scene of interest. Typically, geometric eye model methods use the coordinates of the pupil and two corneal reflections to calculate the 3D gaze direction. Examples of geometric model-based methods are described in: Guestrin, ED and Eizenman, M. (2006), General Theory of Remote Gaze Estimation Using Pupil Center and Corneal Reflections, IEEE Transactions on Biomedical Engineering, 53(6), 1124-1133, the entire contents of which are incorporated herein by reference. Geometric eye model-based methods are insensitive to head movement and can achieve better accuracy than the previously described classes of gaze methods, for example, achieving better than 1 degree accuracy on desktops, smartphones, and head-mounted systems. Such methods typically use more than two IR LEDs to ensure that at least 2 corneal reflections are present in the captured image over the entire range of possible eye positions. However, minimizing the number of IR LEDs has benefits, for example, in reducing the hardware complexity of the eye tracking system and reducing the computational resources required to implement the gaze tracking algorithm.

[0048] The present invention describes examples that may help address some or all of the above-mentioned shortcomings of the prior art.

[0049] In the present invention, references to different coordinate systems can identify specific local or global coordinate systems. For example, coordinates expressed in the image coordinate system, the camera coordinate system, and the eye coordinate system can all describe positions relative to the local coordinate system, which can be converted into coordinates that describe positions relative to the world coordinate system.

[0050] In the present invention, "sight vector" or "visual axis" may refer to: a vector indicating the gaze direction of an individual. In the method based on the geometric eye model, the visual axis of the eye is the vector connecting the eye node and the fovea. In the example, the 3D gaze vector may be represented by the eye coordinate system.

[0051] In the present invention, "Point-of-gaze (POG)" may refer to an object or location within a scene of interest that a person is viewing, or more specifically, the intersection of the visual axis of the eye with the scene of interest. In other examples, POG may correspond to a location on a display screen where the visual axis intersects a 2D display screen.

[0052] In the present invention, the "optical axis" is a line connecting the center of the pupil and the center of the cornea in the 3D geometric eye model. In an example, the optical axis can be represented by a world coordinate system.

[0053] In the present invention, "corneal reflection (CR)" is a virtual image reflected by an IR LED on the cornea, wherein the CR is formed behind the corneal surface and captured in the IR camera image. In an example, the CR may be a set of 2D coordinates describing the position of the CR in the IR camera image and expressed in an image coordinate system. In an example, the CR may also be referred to as a "flicker".

[0054] In the present invention, the "cornea center" is the 3D position of the center point of the cornea. In an example, the cornea center can be a 3D coordinate set describing the position of the cornea center in the world coordinate system. In an example, the cornea center can also be used as the origin of the eye coordinate system.

[0055] In the present invention, the "3D pupil center" is the 3D position of the pupil center point. In an example, the 3D pupil center may be a set of 3D coordinates describing the position of the pupil center in a world coordinate system.

[0056] In the present invention, the "2D pupil center" is the 2D position of the pupil center point in the IR camera image. In an example, the 2D pupil center can be a set of 2D coordinates describing the position of the pupil center in the image coordinate system.

[0057] For purposes of the present invention, a "remote eye tracking system" may refer to an eye tracking system in which the hardware components including the camera and any LEDs are placed away from the user.

[0058] In the present invention, a "head-mounted eye tracking system" may refer to an eye tracking system in which the hardware components including the camera and any LEDs are placed in a head-mounted device such as an augmented reality (AR) or virtual reality (VR) device, such that the camera and any LEDs are positioned very close to the eyes.

[0059] Other terms used in the present invention may be introduced and defined in the following description.

[0060] Figure 1An eye tracking system 100 suitable for implementing the examples described herein is shown, the eye tracking system 100 including an eye tracking device and a 3D geometric eye model of a user's eye. Examples of 3D geometric eye models that may be implemented in example embodiments are described in: Guestrin, ED and Eizenman, M. (2006), General Theory of Remote Gaze Estimation Using Pupil Center and Corneal Reflection, IEEE Transactions on Biomedical Engineering, 53(6), 1124-1133, the entire contents of which are incorporated herein by reference. According to examples of the present invention, the eye tracking system 100 is an illustrative example of a system to which the systems, methods, and processor-readable media described herein may be applied.

[0061] In some embodiments, for example, the eye tracking system 100 includes an image capture device, such as an infrared (IR) camera 104, for capturing an IR camera image 302 of a user, and an illumination device, such as an IR LED 106, for illuminating at least a portion of a user's head (including the user's eye 102). In an example, the IR camera image 302 may include an image of at least a portion of the user's head (including the user's eye 102). In an example, the IR camera 104 may communicate with the gaze estimation system 300 to be able to estimate a gaze vector 120 representing a gaze direction of the user. Optionally, the display screen 108 may include a display of an electronic device, such as a desktop or laptop computer, a mobile communication device, a virtual reality / augmented reality (VR / AR) device, etc.

[0062] In some embodiments, for example, the user's eye 102 may be represented as a 3D geometric model of the eye, including a corneal surface 110, a location of a corneal center c 112, and a location of a pupil center p 114. An optical axis 116 of the eye 102 may be an axis passing through a rotation center of the eye 118, the corneal center 112, and the pupil center 114. In an example, a visual axis representing a line of sight direction (e.g., a line of sight vector 120) may be an axis connecting the corneal center 112 with the center of the fovea 122. In the present invention, the term visual axis may be interchangeable with the term line of sight vector.

[0063] In some embodiments, for example, IR LED 106 can illuminate eye 102 and generate corneal reflection 124 (e.g., flicker). When IR LED 106 is modeled as a point light source i and IR camera 104 is modeled as a pinhole camera j, the light from IR LED 106 is i The incident light 130 of 106 may be reflected at a point q along the corneal surface 110. i132, and the reflected light 134 can pass through the node o 136 of the IR camera 104 to reach the point u ij 138 intersects the image plane of the IR camera 104. In an example, the corneal reflection 124 can be a virtual image reflected by the IR LED 106 on the corneal surface, where point u ij 138 represents the location of the corneal reflection in the captured IR camera image 302 .

[0064] Figure 2 is a block diagram of an exemplary hardware structure of a computing system 200 suitable for implementing the embodiments described herein. Examples of the present invention may be implemented in other computing systems that may include components different from those discussed below. The computing system 200 may be used to execute instructions for estimating a gaze vector 120 representing a gaze direction using any of the examples described herein. The computing system 200 may also be used to train blocks of the gaze estimation system 300, or blocks of the gaze estimation system 300 may be trained by another computing system.

[0065] Although Figure 2 A single instance of each component is shown, but multiple instances of each component may exist in the computer system 200. In addition, although the computing system 200 is shown as a single block, the computing system 200 can be a single physical machine or device (e.g., implemented as a single computing device, such as a single workstation, a single end-user device, a single server, etc.), and can include mobile communication devices (smartphones), laptops, tablet computers, desktop computers, vehicle driver assistance systems, smart appliances, wearable devices, assistive technology devices, medical diagnostic devices, virtual reality devices, augmented reality devices, Internet of Things (IoT) devices, interactive kiosks, advertising and interactive signage, and educational tools, etc.

[0066] The computing system 200 includes at least one processor 202, such as a central processing unit, a microprocessor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a dedicated logic circuit, a dedicated artificial intelligence processor unit, a graphics processing unit (GPU), a tensor processing unit (TPU), a neural processing unit (NPU), a hardware accelerator, or a combination thereof.

[0067] The computer system 200 may include input / output (I / O) interfaces 204 that may support connections to input devices 206 and / or optional output devices 210. In the example shown, the input devices 206 (e.g., keyboard, mouse, microphone, touch screen, and / or keypad) may also include the IR camera 104. In the example shown, the optional output devices 210 (e.g., display, speaker, and / or printer) are shown as optional and external to the computer system 200. In other exemplary embodiments, there may not be any input devices 206 and output devices 210, in which case the I / O interfaces 204 may not be required.

[0068] The computing system 200 may include an optional communication interface 212 for wired or wireless communication with other computing systems (e.g., other computing systems in a network). The communication interface 212 may include a wired link (e.g., an Ethernet cable) and / or a wireless link (e.g., one or more antennas) for communication within and / or between networks.

[0069] The computing system 200 may include one or more memories 214 (collectively referred to as "memory 214"), which may include volatile or non-volatile memory (e.g., flash memory, random access memory (RAM) and / or read-only memory (ROM). The non-transitory memory 214 may store instructions executed by one or more processes 202, for example, to perform the examples described in the present invention. For example, the memory 214 may store instructions for implementing any of the networks and methods disclosed herein. The memory 214 may include other software instructions to implement an operating system (OS), as well as other applications / functions. The instructions may include instructions for implementing and operating the following references. Figure 3 , Figure 4 and Figure 5 Instructions 300 -I of the line of sight estimation system 300 are described.

[0070] The memory 214 may also store other data 216 , information, rules, policies, and machine-executable instructions described herein, including IR camera images 302 captured by the IR camera 104 or eye tracking system calibration parameters 520 of a user.

[0071] In some examples, the computing system 200 may also include one or more electronic storage units (not shown), such as a solid-state drive, a hard disk drive, a magnetic disk drive, and / or an optical disk drive. In some examples, data and / or instructions may be provided by an external memory (e.g., an external drive that communicates with the computing system 200 by wire or wirelessly), or by a transient or non-transient computer-readable medium. Examples of non-transient computer-readable media include RAM, ROM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, CD-ROM, or other portable memory. Storage units and / or external memory may be used in conjunction with the memory 214 to implement data storage, retrieval, and caching functions of the computing system 200. For example, the components of the computing system 200 may communicate with each other via a bus.

[0072] Figure 3 A block diagram of an exemplary architecture of a gaze estimation system 300 provided for an exemplary embodiment of the present invention is provided. The gaze estimation system 300 may be used to implement a method for estimating a gaze vector 120 representing a gaze direction.

[0073] In some examples, the gaze estimation system 300 receives an IR camera image 302 (e.g., an IR facial image 304) and outputs a gaze vector 120 including an estimate of a gaze angle, wherein the gaze vector 120 represents the gaze direction of the user. The IR camera image 302 may be captured by an IR camera 104 on the computing system 200, or the IR camera image 302 may be a digital image captured by another IR camera on another electronic device and transmitted to the computing system 200 (e.g., in the case where the computing system 200 provides gaze estimation services to other devices). In an example, the IR camera image 302 may be extracted from a video image captured by the IR camera 104. In some embodiments, the IR camera image 302 may be obtained from a remote IR camera 104 that is far from the user 600, or the IR camera image 302 may be obtained by an IR camera 104 in a head mounted device (HMD) 910 that is close to the user. In some embodiments, for example, IR camera image 302 may be IR facial image 304, where IR facial image 304 is an IR image that includes a user's face, including features such as eyes, nose, mouth, and chin. In other embodiments, IR camera image 302 may be IR eye image 306, where IR eye image 306 is an IR image that includes a user's eyes.

[0074] In some embodiments, for example, head pose estimator 310 of gaze estimation system 300 may receive IR camera image 302 as IR facial image 304 and may output position information for the user's head as head pose 315, as described below. Figure 4 as described in the discussion.

[0075] Figure 4 A block diagram of an exemplary architecture for a head pose estimator 310 is provided for exemplary embodiments of the present invention. In an example, a facial key point detector 410 of the head pose estimator 310 receives an IR facial image 304 and outputs 2D facial image key points 420. In an example, the facial key point detector 410 can be any suitable pre-trained machine learning based facial key point detection algorithm, such as Google Media Pipe, etc. The facial key point detector 410 can identify 2D key points in the IR facial image 304 and can apply contours around key points such as eyes, nose, mouth, and chin. In some embodiments, for example, the 2D facial image key points 420 can be used with a 3D face model 430, for example, by fitting the 3D face model 430 to the 2D facial image key points 420, so as to estimate the head pose 315 of the user's head in 3D space. In an example, the head pose 315 may include a head translation vector (t x ,t y ,t z )440 and the head rotation vector (θ x ,θ y ,θ z )450 both. In the example, the distance k representing the distance between the user's eye and the IR camera 104 q The translation vector 440 may be determined in the world coordinate system as shown in Equation 1 below.

[0076]

[0077] Return to Figure 3In some embodiments, the eye region extractor 320 of the gaze estimation system 300 may also receive the IR camera image 302 as the IR facial image 304, and may extract a portion of the IR facial image 304 corresponding to the eye region of the user as an eye region image 325. In an example, the eye region extractor 320 may use any suitable pre-trained machine learning-based facial landmark detection algorithm, such as Google MediaPipe, etc., to identify one or both eyes in the IR facial image 304. In an example, the eye region extractor 320 may apply a contour around the user's eyes in the IR facial image 304, and may crop the eye region image 325 including an image of one eye of the user from the IR facial image 304.

[0078] In an example, the eye region image 325 may be input to a corneal reflection estimator 330 to obtain at least one position of a corneal reflection 124 within the eye region image 325, wherein the corneal reflection 124 is a virtual image of a reflection of a corresponding IR LED 106 on a corneal surface of an eye of the user, and wherein the number of corneal reflections 124 depends on the number of IR LEDs 106. An example of a corneal reflection estimator 330 that may be implemented in an exemplary embodiment is described in: Chugh, S. et al., “Detection and Correspondence Matching of Corneal Reflections for Eye Tracking Using Deep Learning,” in the Proceedings of the 2020 IEEE 25th International Conference on Pattern Recognition (ICPR), the contents of which are incorporated herein by reference.

[0079] In some embodiments, the position of at least one corneal reflection 124 may be input to a corneal center estimator 340. In an example, the corneal center estimator 340 may also receive position information as a head pose 315 and a position of a light source (IRLED 106). In an example, the corneal center estimator 340 may determine the position of the corneal center c 112, for example as a 3D coordinate (x, y, z), based on the position of at least one corneal reflection 124, the light source position, and the position information. In an example, the corneal center estimator 340 may be a block that calculates the position of the corneal center c 112 according to equations 2 to 5 below.

[0080] For some k qi ,

[0081] Among them, k qi Reflection point q i 132 and the distance between the node o 136 of the IR camera 104, as calculated in equation 1 above, and u i represents the position of the corneal reflection 124 in the image acquired by the IR camera 104. For any point falling on the cornea, the following conditions can be satisfied:

[0082] R=||q i -c|| (3)

[0083] Where R is the radius of the cornea. In an example, R can be estimated to be 7.8 mm, as described in: Guestrin, ED and Eizenman, M. (2006), A General Theory of Remote Gaze Estimation Using Pupil Center and Corneal Reflection, IEEE Transactions on Biomedical Engineering, 53(6), 1124-1133. In other examples, R can be determined during a calibration process, for example, by referring to Figure 5 In this example, the IR LED i The incident light 130 of 106, the reflected light 134 from the corneal surface and the i The normal at 132 is in the same plane, so Equation 4 is expressed as:

[0084] (l i -o)×(q i -o)·(co)=0 (4)

[0085] Among them, l i is the location of the center of the light source (e.g., IR LED 106), and o is the node 136 of the IR camera 104. Finally, at the reflection point q i At 132, the angle between the incident light and the normal to the corneal surface is equal to the angle between the reflected light 134 and the normal to the corneal surface, so equation 5 is expressed as:

[0086]

[0087] In some embodiments, for example, the corneal center estimator 340 may solve for the corneal center c 112 using Equations 2 to 5 and given k as calculated in Equation 1 qi The 3D coordinates of the corneal center c 112 are calculated using the corneal radius R.

[0088] In some embodiments, for example, the eye region image 325 and the location of the corneal center 112 may be input to a pupil center estimator 350 to generate the location of the 3D pupil center 114. An example of a through-aperture center estimator 350 that may be implemented in an exemplary embodiment is described in: Guestrin, ED and Eizenman, M. (2006), A General Theory of Remote Gaze Estimation Using Pupil Center and Corneal Reflection, IEEE Transactions on Biomedical Engineering, 53(6), 1124-1133, which is incorporated herein by reference in its entirety.

[0089] In some embodiments, for example, corneal center 112 and 3D pupil center 114 may be input to gaze estimator 360 to generate gaze vector 120, as described below. Figure 5 as described in the discussion of .

[0090] Figure 5 A block diagram of an exemplary architecture for a gaze estimator 360 is provided for an exemplary embodiment of the present invention. In an example, the gaze estimator 360 receives the position of the corneal center 112 and the position of the 3D pupil center 114 and outputs a gaze vector 120 representing a gaze direction. In some embodiments, for example, the gaze estimator may include an optical axis estimator 510 block for calculating the optical axis 116 and a visual axis estimator 530 block for calculating the gaze vector 120 based on the optical axis and a plurality of object-specific calibration parameters 520. An example of a gaze estimator 360 that may be implemented in an exemplary embodiment is described in Guestrin, E.D. and Eizenman, M. (2006), A General Theory of Remote Gaze Estimation Using Pupil Center and Corneal Reflection, IEEE Transactions on Biomedical Engineering, 53(6), 1124-1133, the entire contents of which are incorporated herein by reference.

[0091] In an example, calibration parameters 520 may be acquired and stored for a user during a one-time calibration process, for example, where the user views one or more target points with known positions on the display screen 108 while capturing multiple IR camera images 302 for each corresponding target point. In some embodiments, for example, the gaze estimation system 300 may predict multiple gaze vectors 120 from the multiple IR camera images 302 acquired during the calibration process, and may compare the predicted gaze vectors 120 with each of the one or more target points with known positions. In an example, an optimization algorithm may be used to acquire multiple user-specific calibration parameters 520 that minimize the error between the predicted gaze vectors 120 and the target points. In an example, the optimization algorithm may include a least squares method or other optimization method. In some examples, the multiple calibration parameters 520 may include an angular offset between an optical axis and a visual axis of the user's eye, and / or the calibration parameters 520 may include parameters representing a radius R of the user's eye, a distance between a 3D pupil center 114 and a corneal center 112, or an effective refractive index of the aqueous humor and cornea of ​​the user's eye, as well as other subject-specific parameters. In an example, the gaze vector 120 may be calculated from the optical axis 116 using an angular offset between the optical axis and the user's eye and the visual axis.

[0092] Fig. 6A An exemplary embodiment of an eye tracking system 100 provided by an example of the present invention is shown, wherein the eye tracking system 100 is a remote eye tracking system and includes an IR LED 106 located at a distance from an IR camera 104. In this embodiment, a user 600 interacts with the remote eye tracking system 100, and a gaze vector 120 represents the direction of the user's gaze. An optional display screen 108 is shown to demonstrate screen-based interaction. In this embodiment, the IR LED 106 is positioned away from the IR camera 104 for generating a corneal reflection 124. Figure 6B and Figure 6C The example of the present invention provides a Fig. 6A 3. An exemplary IR camera image 302 corresponding to, for example, a user's eye is obtained in an embodiment of the present invention. In an example, the IR camera image 302 may include features of the eye, such as a pupil (e.g., a dark pupil 602) and an iris 604. Figure 6C As shown, one or more corneal reflections 124 may also be visible in the IR camera image 302 .

[0093] FIG7 shows an exemplary embodiment of an eye tracking system 100 provided by an example of the present invention, wherein the eye tracking system 100 is a remote eye tracking system and includes a first IR LED 106 located at a distance from an IR camera 104 and a second IR LED 107 located near the IR camera 104. In this embodiment, a user 600 interacts with the remote eye tracking system 100, and a gaze vector 120 represents the direction of the user's gaze. An optional display screen 108 is shown to demonstrate screen-based interaction. In this embodiment, the first IR LED 106 is located at a position away from the IR camera 104 for generating a corneal reflection 124, and the second IR LED 107 is located at a position near the IR camera 104 for creating a bright pupil effect in the IR camera image 302. Figure 7B and Figure 7C The example of the present invention provides a Fig. 7A 3. An exemplary IR camera image 302 corresponding to, for example, a user's eye, is obtained in accordance with an embodiment of the present invention. In an example, the IR camera image 302 may include features of the eye, such as a pupil (e.g., a bright pupil 702) and an iris 604. In an example, an advantage of using the second IR LED 107 to create a bright pupil effect may be to improve the accuracy of the gaze estimation system 300 in locating the pupil region in the image. Figure 7C As shown, one or more corneal reflections 124 on the corneal surface of the user's eye may also be visible in the IR camera image 302 .

[0094] FIG8 shows an exemplary embodiment of an eye tracking system 100 provided by an example of the present invention, wherein the eye tracking system 100 is located in a head mounted device (HMD) 910. In this embodiment, the user 600 interacts with the HMD eye tracking system by looking at the display screen 108 in the HMD 910 through a convex lens 802. The gaze vector 120 represents the direction of the user's gaze. In this embodiment, the first IR LED 106a, the second IR LED 106b, and the IR camera 104 are located near the user's eyes. In the example, the first IR LED 106a and the second IR LED 106b can each generate a corneal reflection 124. In this embodiment, for example, because the hardware of the eye tracking system 100 is very close, it may be necessary to use two IRLEDs to illuminate the user's eyes to ensure uniform illumination in the captured IR camera image 302 and to ensure that at least one corneal reflection 124 is captured in the IR camera image 302 over the entire range of eye movement and position. FIG. 8B to FIG. 8D The range of eye positions is shown as Fig. 8A An exemplary IR camera image 302 captured by an embodiment of the present invention. Figure 8BAn exemplary IR camera image 302 is shown in which the user's eyes are looking forward and two corneal reflections 124a and 124b are visible in the IR camera image 302, each corresponding to a respective IR LED. Figure 8C An exemplary IR camera image 302 is shown in which the user's eye is looking far to the right, but only one corneal reflection 124 is captured. A secondary reflection (e.g., scleral reflection 804) is visible on the sclera of the user's eye. Similarly, Fig.8D An exemplary IR camera image 302 is shown in which the user's eye is gazing to the left, but only one corneal reflection 124 is captured, while a secondary scleral reflection 804 is also visible.

[0095] Fig. 9 FIG. 1 is a block diagram of an exemplary architecture of a sight line estimation system 300 provided for an exemplary embodiment of the present invention. The sight line estimation system 300 may be used in conjunction with Fig. 8A The embodiment of the present invention implements a method for estimating a sight vector 120 representing a sight direction.

[0096] In some examples, the gaze estimation system 300 receives an IR camera image 302 (e.g., an IR eye image 306) and outputs a gaze vector 120 including an estimate of a gaze angle, wherein the gaze vector 120 represents the gaze direction of the user. The IR camera image 302 may be captured by the IR camera 104 on the computing system 200, or the IR camera image 302 may be a digital image captured by another IR camera on another electronic device and transmitted to the computing system 200 (e.g., in the case where the computing system 200 provides gaze estimation services to other devices). In an example, the IR camera image 302 may be extracted from a video image captured by the IR camera 104. In this embodiment, the IR camera image 302 may be an IR eye image 306 acquired by the IR camera 104 within the head mounted device (HMD) 910 when approaching the user, wherein the IR eye image 306 is an IR image including the user's eyes.

[0097] In some embodiments, for example, the HMD 910 of the gaze estimation system 300 may output position information of the user's head as the eye-to-camera distance 920. In an example, the HMD 910 includes an adjustable headset worn by the user, wherein the IR camera 104 is fixed within the headset, and the distance from the IR camera 104 to the user's eyes may be determined based on the physical information provided by the HMD 910. In an example, when the user adjusts the headset of the HMD 910, the HMD 910 may obtain the measurement of the user's adjustment or fit setting and use it to determine the eye-to-camera distance 920. Optionally, the pupil distance (PD) (e.g., the distance measured in millimeters between the 2D pupil centers of the eyes) may also be obtained by the HMD 910 and used to determine the eye-to-camera distance 920.

[0098] In an example, the IR eye image 306 may be input to a corneal reflection estimator 330 to obtain at least one position of a corneal reflection 124 within the IR eye image 306, wherein the corneal reflection 124 is a virtual image of a reflection of a corresponding IR LED 106 on a corneal surface of an eye of the user, and wherein the number of corneal reflections 124 depends on the number of IR LEDs 106. An example of a corneal reflection estimator 330 that may be implemented in an exemplary embodiment is described in Chugh, S. et al., “Detection and Correspondence Matching of Corneal Reflections for Eye Tracking Using Deep Learning,” in Proceedings of the 2020 IEEE 25th International Conference on Pattern Recognition (ICPR), the contents of which are incorporated herein by reference.

[0099] In some embodiments, the position of at least one corneal reflection 124 may be input to the corneal center estimator 340. In an example, the corneal center estimator 340 may also receive the position information as the distance 920 from the eye to the camera, and may output the position of the corneal center c 112 based on the at least one corneal reflection 124 and the position information, for example as a 3D coordinate (x, y, z). In an example, the corneal center estimator 340 may be based on the previous description of Figure 3 Equations 2 to 5 describe the calculation of the location of the corneal center c 112 in blocks, where k qi Indicates the distance from the eye to the camera 920.

[0100] In some embodiments, for example, the IR eye image 306 and the location of the corneal center 112 may be input to a pupil center estimator 350 to generate the location of the 3D pupil center 114. An example of a through-aperture center estimator 350 that may be implemented in an exemplary embodiment is described in: Guestrin, ED and Eizenman, M. (2006), A General Theory of Remote Gaze Estimation Using Pupil Center and Corneal Reflection, IEEE Transactions on Biomedical Engineering, 53(6), 1124-1133, which is incorporated herein by reference in its entirety.

[0101] In some embodiments, for example, corneal center 112 and 3D pupil center 114 may be input to gaze estimator 360 along with subject-specific calibration parameters 520 to generate gaze vector 120, as previously described in Figure 5 as described in the discussion of .

[0102] Fig.10 A flowchart of the steps of an exemplary method 1000 for estimating a gaze vector 120 representing a user's gaze direction is provided for an example of the present invention. The method 1000 may be performed by the computing system 200. For example, the processor 202 may execute computer-readable instructions (which may be stored in the memory 214) to cause the computing system 200 to perform the method 1000. The method 1000 may be performed using a single physical machine (e.g., a workstation or server), multiple physical machines working in coordination (e.g., a server cluster), or cloud-based resources (e.g., using virtual resources on a cloud computing platform).

[0103] The method 1000 begins at step 1002, where an IR camera image 302 of a user is acquired using the IR camera 104 and at least one IR LED 106, where the IR camera image 302 includes an image of the user's eyes. The IR camera image 302 may be captured by the IR camera 104 on the computing system 100, or may be a digital image captured by another IR camera on another electronic device and transmitted to the computing system 100.

[0104] In step 1004, the position of the corneal reflection 124 is estimated within the IR camera image 302 using a corneal reflection estimator. In an example, the corneal reflection 124 is a virtual image of the reflection of the corresponding IR LED 106 on the corneal surface of the user's eye captured in the IR camera image 302.

[0105] In step 1006, the position information of the user's head may be obtained. In some embodiments, the position information of the user's head may be the head posture 315, or in other embodiments, the position information of the user's head may be the distance 920 from the eyes to the camera.

[0106] In step 1008 , the position of the cornea center 112 of the user's eye is estimated using the cornea center estimator 340 based on the position and position information of the cornea reflection 124 .

[0107] In step 1010 , the position of the 3D pupil center 114 is estimated using the pupil center estimator 350 based on the IR camera image 302 and the position of the corneal center 112 .

[0108] Finally, in step 1012, the gaze vector 120 representing the gaze direction may be estimated using the gaze estimator 360 based on the position of the corneal center 112 and the position of the 3D pupil center 114. The estimated gaze vector 120 may include two angles describing the gaze direction, namely yaw and pitch.

[0109] In some examples, the estimated sight direction can be output to an application on the electronic device (e.g., a software application executed by the computing system 200) to estimate the point on the screen of the electronic device that the individual is watching. For example, if the application on the electronic device is an auxiliary tool that enables speech generation, obtaining an accurate estimate of the viewpoint on the screen can enable non-verbal individuals to communicate by spelling words or composing sentences by looking at specific areas of the screen. In another example, if the application on the electronic device is an educational application, collecting data about the location and time of certain areas of the screen that the user watches can provide feedback to the provider of the educational application about the effectiveness of the educational content, which content attracts the user's attention, and which content is missed. Similarly, if the application on the electronic device contains advertising or marketing content, data about the effectiveness of the content can be collected by checking whether the individual watches the advertisement and how long the advertisement is watched. Data can be collected to understand the best placement of content on the screen, or to identify effective content that attracts the individual's attention more frequently and keeps their attention.

[0110] In other examples, the estimated gaze direction may be output to an application to be executed by the onboard computing system to assess the viewpoint of the individual operating the vehicle. In situations where the individual operating the vehicle appears distracted or inattentive, e.g., frequently or for extended periods of time looking away from the road, the vehicle safety system may provide a notification or warning to the operator of the vehicle to remind them to pay attention to the road ahead.

[0111] Although the present invention describes the method and process in a specific order, one or more steps of the method and process may be omitted or modified as appropriate. One or more steps may be performed in sequence, but not in the order described (as appropriate).

[0112] Although the present invention has been described, at least in part, in terms of methods, it will be understood by those skilled in the art that the present invention also relates to various components for performing at least some aspects and features of the described methods by hardware components, software, or any combination of the two. Accordingly, the technical solution of the present invention can be embodied in the form of a software product. Suitable software products can be stored in a pre-recorded storage device or other similar non-volatile or non-transient computer-readable medium, such as a DVD, CD-ROM, USB flash drive, removable hard disk or other storage medium. The software product includes instructions stored thereon, which enable a processing device (such as a personal computer, server or network device) to perform examples of the methods disclosed herein. Machine executable instructions can be in the form of code sequences, configuration information, or other data, which, when executed, cause a machine (e.g., a processor or other processing device) to perform steps in the method according to the examples of the present invention.

[0113] The present invention may be embodied in other specific forms without departing from the subject matter of the claims. The described exemplary embodiments are merely illustrative and not restrictive in all respects. Selected features from one or more of the above embodiments may be combined to create alternative embodiments not explicitly described, and it is understood that features suitable for such combinations are within the scope of the present invention.

[0114] All values ​​and subranges within the disclosed range are also disclosed. In addition, although the systems, devices, and processes disclosed and shown herein may include a specific number of elements / components, the systems, devices, and assemblies may be modified to include more or fewer elements / components in such elements / components. For example, although any element / component disclosed may be cited as a singular, the embodiments disclosed herein may be modified to include a plurality of such elements / components. The subject matter described herein is intended to cover and encompass all appropriate technical changes.

Claims

1. A method, characterized in that include: Acquire an image of a user, the image including an image of the user's eyes; estimating a position of a corneal reflection in the image; Determine the position information of the user's head according to the image; The position of the cornea center of the user's eye is estimated according to the position of the cornea reflection in the image and the position information.

2. The method according to claim 1, characterized in that The image of the user is acquired by using an IR camera and at least one IR LED, wherein Estimating the position of the cornea center of the user's eye includes: determining a distance between the user's eyes and the IR camera based on the position information; The position of the cornea center is estimated based on the distance between the user's eye and the IR camera, the radius of the user's cornea, the position of the IR camera, the position of the at least one IR LED, and the position of the corneal reflection in the image.

3. The method according to claim 2, characterized in that Also includes: estimating the position of the 2D pupil center in the image; estimating a position of a 3D pupil center of the user's eye based on the position of the 2D pupil center in the image and the position of the cornea center; A sight line vector indicating a sight line direction of the user is estimated based on the position of the cornea center and the position of the 3D pupil center.

4. The method according to claim 3, characterized in that Estimating the sight vector representing the sight direction of the user comprises: estimating the optical axis of the user's eye based on the position of the cornea center and the position of the 3D pupil center; The line of sight vector is estimated based on the optical axis and a plurality of calibration parameters.

5. The method according to any one of claims 1 to 4, characterized in that The position information of the user's head includes the user's head posture.

6. The method according to claim 5, characterized in that Acquiring the head posture of the user includes: estimating one or more facial landmarks corresponding to the user's face based on the image; Fitting the estimated one or more facial key points to a 3D face model; The head posture is estimated based on the 3D face model.

7. The method according to any one of claims 2 to 6, characterized in that The IR camera is located at a certain distance from the user, and the image is a facial image.

8. The method according to any one of claims 2 to 4, characterized in that The image is an eye image, and the position information of the user's head is the distance from the user's eyes to the IR camera, and the distance from the user's eyes to the IR camera is obtained from a head-mounted device.

9. The method according to claim 2, characterized in that: The image of the user includes bright pupils.

10. The method according to any one of claims 1 to 4, characterized in that The estimated viewing vector is a 3D viewing vector.

11. An eye tracking system, characterized in that: include: one or more processors; A memory storing machine executable instructions, which, when executed by the processor device, cause the system to perform the following operations: Acquire an image of a user, the image including an image of the user's eyes; estimating a position of a corneal reflection in the image; Determine the position information of the user's head according to the image; The position of the cornea center of the user's eye is estimated according to the position of the cornea reflection in the image and the position information.

12. The system according to claim 11, characterized in that The system also includes an IR camera and a first IRLED, wherein The image of the user is acquired through the IR camera and the first IR LED, When the machine executable instructions are executed by the one or more processors to estimate the position of the corneal center of the user's eye, the system is further caused to perform the following operations: determining a distance between the user's eyes and the IR camera based on the position information; The position of the cornea center is estimated based on the distance between the user's eye and the IR camera, the radius of the user's cornea, the position of the IR camera, the position of the at least one IR LED, and the position of the corneal reflection in the image.

13. The system according to claim 12, characterized in that When the machine executable instructions are executed by the one or more processors, the system is further caused to perform the following operations: estimating the position of the 2D pupil center in the image; estimating a position of a 3D pupil center of the user's eye based on the position of the 2D pupil center in the image and the position of the cornea center; A sight line vector indicating a sight line direction of the user is estimated based on the position of the cornea center and the position of the 3D pupil center.

14. The system according to claim 13, characterized in that When the machine executable instructions are executed by the one or more processors to estimate the sight vector representing the sight direction of the user, the system is further caused to perform the following operations: estimating the optical axis of the user's eye based on the position of the cornea center and the position of the 3D pupil center; The line of sight vector is estimated based on the optical axis and a plurality of calibration parameters.

15. The system according to any one of claims 11 to 14, characterized in that The position information of the user's head includes the user's head posture.

16. The system according to claim 15, characterized in that When the machine executable instructions are executed by the one or more processors to obtain the head posture of the user, the system is further caused to perform the following operations: estimating one or more facial landmarks corresponding to the user's face based on the image; Fitting the estimated one or more facial key points to a 3D face model; The head posture is estimated based on the 3D face model.

17. The system according to any one of claims 12 to 16, characterized in that The IR camera is located at a certain distance from the user, and the image is a facial image.

18. The system according to any one of claims 12 to 14, characterized in that The IR camera image is an eye image, and the position information of the user's head is the distance from the user's eyes to the IR camera, and the distance from the user's eyes to the IR camera is obtained from a head-mounted device.

19. The system according to claim 12, characterized in that Also included is a second IR LED, wherein the first IR LED generates the corneal reflection in the image and the second IR LED generates a bright pupil in the image.

20. A non-transitory computer readable medium, characterized in that The non-transitory computer-readable medium stores machine-executable instructions, and when the machine-executable instructions are executed by a processor of a device, the device performs the following operations: Acquire an image of a user, the image including an image of the user's eyes; estimating a position of a corneal reflection in the image; Determine the position information of the user's head according to the image; The position of the cornea center of the user's eye is estimated according to the position of the cornea reflection in the image and the position information.

Images