Eye tracking device and method

Abstract

This invention discloses an eye-tracking device and method. The eye-tracking device includes a dense display device with a first display area configured to emit infrared vector structured light toward a first eye; a first detection component including at least two first probes configured to receive light reflected or scattered by the first eye to acquire first detection image information, the first detection image information including a first pupil center point and at least one first flicker point; and a control component electrically connected to the dense display device and the first detection component, configured to drive the dense display device to emit light and calculate the gaze direction of the first eye based on the first detection image information; wherein the control component is configured to calculate the gaze direction of the first eye using a corneal curvature algorithm model and / or a pupil center corneal reflection algorithm model. This invention can simultaneously meet the requirements of simple structure and high tracking accuracy in eye-tracking devices.

Description

Technical Field

[0001] This invention relates to the field of eye-tracking technology, and more particularly to an eye-tracking device and method. Background Technology

[0002] Eye-tracking technology has wide applications in modern society, serving as a technical means for visual and human behavior analysis in fields such as psychology, neuromarketing, neurocognition, user experience, basic research, and market research. Eye-tracking applications fall into five main categories: desktop computers, televisions and display panels, head-mounted displays, automotive devices, and handheld devices. Desktop computer applications primarily involve using eye gaze for text input and login system recognition; televisions and display panels are used for selection, navigation, and channel switching; head-mounted displays are mainly used in virtual reality for research on the relationship between user gaze and cognition, as well as psychoanalysis; automotive applications primarily involve tracking the driver's eyes to assess drowsiness and connect to driver assistance systems; and handheld devices can track the user's gaze to activate the system for authentication and interactive displays.

[0003] However, existing eye-tracking devices using eye-tracking technology cannot simultaneously meet the requirements of simple structure, high tracking accuracy, and no need for external calibration, which limits the application of eye-tracking devices. Summary of the Invention

[0004] This invention provides an eye-tracking device and method that balances the requirements of simple structure and high tracking accuracy in eye-tracking devices.

[0005] According to one aspect of the present invention, an eye-tracking device is provided, the eye-tracking device comprising:

[0006] A dense display device is provided with a first display area, the first display area being configured to emit infrared vector structured light toward a first eye;

[0007] A first detection component includes at least two first probes, the first detection component being configured to receive light reflected or scattered by a first eye to obtain first detection image information, the first detection image information including a first pupil center point and at least one first scintillation point;

[0008] A control component, electrically connected to the dense display device and the first detection component, is configured to drive the dense display device to emit light and calculate the gaze direction of the first eye based on the first detected image information;

[0009] The control component is configured to calculate the gaze direction of the first eye using a corneal curvature algorithm model and / or a pupil center corneal reflection algorithm model.

[0010] Optionally, the control component is configured to calculate a first calculation result of the gaze direction of the first eye using a pupil center corneal reflection algorithm model; and to learn a corneal curvature algorithm model using a machine learning method and the first calculation result, and to calculate a second calculation result of the gaze direction of the first eye using the learned corneal curvature algorithm model and the first scintillation point data.

[0011] Optionally, the control component is further configured to calculate a second calculation result of the gaze direction of the first eye using the corneal curvature algorithm model when no new first calculation result is calculated using the pupil-center corneal reflection algorithm model; and to calibrate the corneal curvature algorithm model using the new first calculation result after a new first calculation result is calculated using the pupil-center corneal reflection algorithm model.

[0012] Optionally, the eye-tracking device further includes at least one first reflector; the dense display device also includes at least one second display area, the second display area being configured to emit infrared vector structured light toward the first eye through reflection from the corresponding first reflector.

[0013] Optionally, the eye-tracking device further includes: a semi-reflective lens and a second detection component; the dense display device also has a third display area, which is configured to emit infrared vector structured light to the second eye through the semi-reflective lens;

[0014] The second detection component includes at least two second probes, and the second detection component is configured to receive light reflected or scattered by the second eye to obtain second detection image information, the second detection image information including the center point of the second pupil and at least one second scintillation point;

[0015] The control component is electrically connected to the second detection component and is configured to drive the dense display device to emit light and calculate the gaze direction of the second eye based on the second detection image information;

[0016] The control component is configured to calculate the gaze direction of the second eye using a corneal curvature algorithm model and / or a pupil center corneal reflection algorithm model.

[0017] Optionally, the eye-tracking device further includes at least one second reflector, and the dense display device further includes at least one fourth display area, the fourth display area being configured to emit infrared vector structured light toward the second eye through the semi-reflective lens and the reflection of the corresponding second reflector.

[0018] Optionally, the control component may further include a learning machine configured to learn a corneal curvature algorithm model.

[0019] Optionally, the first probe is an optical fiber probe, and the detection component further includes a first photodetector; the first photodetector is optically connected to the optical fiber probe and electrically connected to the control component;

[0020] Alternatively, the first probe may be a photodetector, and the first probe may be electrically connected to the control component.

[0021] According to another aspect of the present invention, an eye-tracking method is provided, executed by the aforementioned eye-tracking device, the eye-tracking method comprising:

[0022] The control component drives the dense display device to emit light and calculates the gaze direction of the first eye based on the first detected image information;

[0023] The control component is configured to calculate the gaze direction of the first eye using a corneal curvature algorithm model and / or a pupil center corneal reflection algorithm model.

[0024] Optionally, a first calculation result of the gaze direction of the first eye is calculated using a pupil center corneal reflection algorithm model; and a corneal curvature algorithm model is learned using machine learning methods and the first calculation result, and a second calculation result of the gaze direction of the first eye is calculated using the learned corneal curvature algorithm model and the first scintillation point data.

[0025] Optionally, the control component is further configured to calculate a second calculation result of the gaze direction of the first eye using the corneal curvature algorithm model when no new first calculation result is calculated using the pupil-center corneal reflection algorithm model; and to calibrate the corneal curvature algorithm model using the new first calculation result after a new first calculation result is calculated using the pupil-center corneal reflection algorithm model.

[0026] Optionally, before the control component drives the dense display device to emit light and calculates the gaze direction of the first eye based on the first detected image information, the method further includes:

[0027] The control component drives the dense display device to perform a global scan, obtains a first eye detection map formed by the global scan, and calculates the pupil position and the corresponding first flicker point position of the first eye based on the first eye detection map.

[0028] Optionally, the eye-tracking device further includes: a semi-reflective lens and a second detection component; the dense display device further includes a third display area, the third display area being configured to emit infrared vector structured light to the second eye through the semi-reflective lens; the second detection component includes at least two second probes, the second detection component being configured to receive light reflected from the second eye to obtain second detection image information, the second detection image information including a second pupil center point and at least one second scintillation point; the control component is electrically connected to the second detection component;

[0029] The eye-tracking method further includes: the control component driving the dense display device to emit light, and calculating the gaze direction of the second eye based on the second detected image information; wherein, the control component is configured to calculate the gaze direction of the second eye using a corneal curvature algorithm model and / or a pupil center corneal reflection algorithm model;

[0030] Before the control component drives the dense display device to emit light and calculates the gaze direction of the second eye based on the second detected image information, the following steps are also included:

[0031] The control component drives the dense display device to perform a global scan, obtains a second eye detection map formed by the global scan, and calculates the pupil position and the corresponding second flicker point position of the second eye based on the second eye detection map.

[0032] Optionally, the control component drives the dense display device to emit light, and calculates the gaze direction of the first eye based on the first detected image information, including:

[0033] The control component determines the pupil region displacement range for the next frame based on the pupil region of the current frame and a first preset formula, determines the first flashing point displacement range for the next frame based on the position of the first flashing point of the current frame and a second preset formula, and drives the dense display device to scan only the pupil region displacement range and / or only the first flashing point displacement range in the next frame.

[0034] Optionally, the driving dense display device scans only the pupil region displacement range and the first flicker point displacement range, including:

[0035] The dense display device is driven to emit a first structured light calculated based on the pupil region of the current frame and scan the displacement range of the pupil region in the next frame; wherein, the first structured light is a circular structured light, or a structured light whose contour corresponds to the pupil contour of the first eye in the current frame;

[0036] The driving dense display device scans only the displacement range of the first flicker point, including:

[0037] The dense display device is driven to emit line structured light to scan the displacement range of the first scintillation point; wherein the cross-section of the line structured light consists of two mutually perpendicular line segments.

[0038] Optionally, the eye-tracking device further includes a first reflector; the dense display device also includes a second display area, the second display area being configured to emit infrared vector structured light toward the first eye through reflection from the first reflector;

[0039] The control component is also configured to drive the first display area and the second display area to emit light in a time-division manner.

[0040] The technical solution of this invention employs a dense display device, which can typically generate multiple first flashing points simultaneously or asynchronously on the first eye. Among these multiple flashing points, some will fall on the corneal area, and others will fall on non-corneal areas. Furthermore, the coordinates of the first pupil's center point can be obtained through the first detected image information. Then, at least one of a corneal curvature algorithm model and a pupil center corneal reflection algorithm model can be used to calculate the gaze direction of the first eye. Since two algorithm models can be used to calculate the gaze direction of the first eye, the calculation accuracy can be greatly improved, and compatibility is also stronger. It should be noted that there is no need to determine whether the flashing point falls on the corneal area during calculation. The control component is equipped with machine learning capabilities, and by increasing the learning amount, it can identify flashing points falling on the corneal area and flashing points falling on non-corneal areas during subsequent calculations. The eye-tracking device of this embodiment has the advantages of simple device structure and easy integration. Moreover, it can use at least one of a corneal curvature algorithm model and a pupil center corneal reflection algorithm model to calculate the gaze direction, offering the advantage of high calculation accuracy. In other words, this embodiment makes the eye-tracking device compatible with both simple structure and high tracking accuracy.

[0041] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic diagram of the structure of an eye-tracking device provided in an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of a first detection image information provided in an embodiment of the present invention;

[0045] Figure 3 A schematic diagram illustrating the principle of a corneal curvature algorithm model provided in an embodiment of the present invention;

[0046] Figure 4 A schematic diagram showing the change in eye gaze direction versus beam elevation angle;

[0047] Figure 5 A flowchart of an eye-tracking method provided in an embodiment of the present invention;

[0048] Figure 6 A flowchart illustrating yet another eye-tracking method provided in an embodiment of the present invention;

[0049] Figure 7 Schematic diagram for determining the displacement range of the pupil region;

[0050] Figure 8 A schematic diagram for determining the displacement range of the first flashing point. Detailed Implementation

[0051] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0052] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0053] Figure 1 This is a schematic diagram of an eye-tracking device provided in an embodiment of the present invention, with reference to... Figure 1The eye-tracking device includes: a dense display device 11, having a first display area configured to emit infrared vector structured light toward a first eye; a first detection component 12, including at least two first probes 121, configured to receive light reflected or scattered by the first eye to obtain first detection image information, the first detection image information including a first pupil center point and at least one first flashing point; and a control component 13, electrically connected to the dense display device 11 and the first detection component 12, configured to drive the dense display device 11 to emit light and calculate the gaze direction of the first eye based on the first detection image information; wherein the control component 13 is configured to calculate the gaze direction of the first eye using a corneal curvature algorithm and / or a pupil center corneal reflection algorithm model.

[0054] Specifically, such as Figure 1As shown, the eye-tracking device can be installed on the glasses. It should be noted that after the eye-tracking device is installed, its position can be calibrated before it is fixed in place. Furthermore, the positions of the lenses, the dense display device, and the first detection component of the glasses are calculated and designed according to the glasses specifications, and their positions are calibrated before the glasses leave the factory. That is to say, the relative positions of the lenses, the dense display device, and the first detection component are known. The dense display device 11 can be, for example, an LED (Light Emitting Diode) chip, typically an LED chip containing four thousand pixels. This LED chip has a scanning rate of 200K when all four thousand pixels are lit simultaneously, and can reach 800M frames / second for single-point scanning. For example, the LED chip is a high-speed projection chip that emits infrared vector structured light. The aperture of the projection chip lens must be small enough, with a diameter of approximately 0.2mm, to ensure a sufficiently large depth of field, sufficiently fine light emission, and sufficiently high resolution. The projection chip can be driven to project structured light in a time-division manner to scan the human eye, and can be dynamically driven to project any one or several light paths to a designated position. It can also perform time-division single-point and structured light scanning. The dense display device 11 includes multiple display areas, each containing multiple pixels for emitting infrared vector structured light in different directions. For example, the first display area on the dense display device 11 can emit infrared vector structured light towards the first eye. The first eye is, for example, the left or right eye of a living organism (such as a human), and this embodiment does not specifically limit this. Preferably, the position of the eye-tracking device can be calibrated (e.g., by adjusting the position of the glasses) so that the infrared vector structured light emitted by the first display area illuminates the area of ​​the eyeball in the first eye as much as possible. The first detection component 12 can receive light reflected or scattered by the first eye, so that the eye-tracking device can obtain first detection image information based on the light received by the first detection component 12. Different first probes 121 are responsible for detecting different areas of the first eye; by integrating the information detected by multiple first probes 121, the first detection image information can be obtained. Of course, the more probes (121) there are, the finer the detection zones, and the higher the detection accuracy. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of a first detection image information provided in an embodiment of the present invention. It should be noted that the first detection image information in this embodiment may also be only... Figure 2A portion of the complete image (described later); the first detection image information includes at least a first pupil center point P and at least one first scintillation point A (exemplarily including two first scintillation points A1 and A2 in the figure). The first pupil center point P is also the pupil center corresponding to the first eye; the first scintillation point is a scintillation point—the Purkinje image—generated by the first display area of ​​the dense display device illuminating the cornea. This scintillation point (Corneal Reflection, CR) is generated by the reflection of light entering the pupil on the outer surface of the cornea. Since existing methods using the Pupil Center Corneal Reflection (PCCR) principle to track the human eye assume the eyeball is approximately spherical, the position of the scintillation point illuminating the eyeball does not change significantly with eye movement. Furthermore, since the first display area contains multiple pixels, which is equivalent to multiple light sources illuminating the first eye in the first display area, at least one first scintillation point will be generated in the first detection image information. The control component 13 can be integrated into the lens frame and configured to send a drive signal to the dense display device 11 to drive the dense display device 11 to emit light. Furthermore, the control component 13 can also calculate the gaze direction of the first eye based on the first detected image information. Specifically, the gaze direction of the first eye can be calculated using at least one of a corneal curvature algorithm model and a pupil-center corneal reflex algorithm model. The principles of the corneal curvature algorithm model and the pupil-center corneal reflex algorithm model will be introduced below.

[0055] Figure 3 This is a schematic diagram illustrating the principle of a corneal curvature algorithm model provided in an embodiment of the present invention. (Refer to...) Figure 3 Assume that eyeball 51 is an approximately spherical structure, while cornea 52 has a different curvature and a certain degree of convexity compared to eyeball 51. Eyeball 51 and cornea 52 are two spherical surfaces with different curvatures. Assume that the center of eyeball 51 when looking straight ahead is 0, and the center of cornea is o. The center of eyeball remains unchanged when the gaze direction changes. The cornea 52 after the gaze direction changes is represented by a solid line, which is the line connecting the center of eyeball 0 and the center of cornea o' at this time. Direction of eye movement The included angle is This can be confirmed as the deflection angle of the human eye's gaze direction. Assume S... i For dense display devices, i.e., transmitters, i>0, P j For a device (i.e., a first probe) to receive the first reflected or scattered signal from the eye, j>0; The signal emitted by a certain transmitter can be detected by a certain probe P j The elevation angle of the received beam. α is the horizontal light emission angle, β is the vertical light emission angle, and L ij For transmitter Si To P j distance, S represents the coordinate position of the received light beam on the surface of the eyeball when the eye is looking straight ahead. i (x,y,z), P j (x, y, z) is a fixed device, and its position is known. Then the distance L is... ij It is certain that it was P j The received beam of light can also be determined, that is... It is also determinable. Therefore, we know that there must exist a formula f1 that can calculate the reflection point. The coordinates of the location There also exists a formula f, which can be derived from... Determine the gaze direction from the changes Therefore, the gaze direction can be learned by acquiring the coordinates of multiple blinking points and using machine learning. like Figure 4 Indication, Figure 4 This is a schematic diagram illustrating the change in eye gaze direction versus beam elevation angle. Assuming... Given the beam elevation angle when looking straight ahead and the eyeball can rotate horizontally 180 degrees, then through the transmitter S... i With the device P that receives reflected infrared light j It can track the beam elevation angle. The curve showing the change. Based on the characteristics of the eye's structure, we know that regardless of whether the eye is looking left or right, up or down, the elevation angle of the tracked beam... It may undergo a sudden change at a certain moment, or it may remain basically unchanged after rotating to a certain angle. It can reflect changes in the direction of human eye gaze. Of course, it should be noted that... Figure 4 This is just an illustration; each eye is different, and the actual curve corresponding to each eye is not exactly the same. Using our eye-tracking device, without needing to know how f1 and f are calculated, we can combine standard signal input with the learning machine to allow the machine to learn the calculation formula f. Therefore, by tracking the position of the blinking point, we can output the eye's gaze direction. This corneal curvature algorithm has high output efficiency.

[0056] The specific principle of the pupil-center corneal reflex algorithm model is as follows:

[0057] An eye is illuminated by a light source to produce a noticeable reflection, and a camera captures images of the eye showing these reflections. These images are then used to identify the reflections of the light source on the pupil and cornea, i.e., the scintillation points. The eye movement vector can then be calculated by determining the angle between the pupil center and the scintillation point. The direction of this vector is then combined with the geometric features of other reflections to calculate the direction of the gaze. In other words, the pupil-center corneal reflection algorithm model requires the coordinates of the scintillation point on the cornea and the coordinates of the pupil center.

[0058] In this embodiment, since a dense display device is used as a projection device, it can generate at least one first flickering point on the first eye. In the prior art, when calculating using the corneal curvature algorithm model, the first flickering point must fall on the corneal area for the calculation result to be reliable. When calculating using the pupil center corneal reflection algorithm model, the first flickering point must fall on the non-corneal area for the calculation result to be reliable. In this embodiment, the dense display device can generally generate multiple first flickering points on the first eye. Among these multiple first flickering points, there will inevitably be flickering points that fall on the corneal area and flickering points that fall on the non-corneal area. Furthermore, the coordinates of the center point of the first pupil can be obtained through the first detection image information. Therefore, at least one of the corneal curvature algorithm model and the pupil center corneal reflection algorithm model can be used to calculate the gaze direction of the first eye. Since two algorithm models can be used to calculate the gaze direction of the first eye, the calculation accuracy of the gaze direction can be greatly improved, and the compatibility is also stronger. It should be noted that there is no need to determine whether the flashing point falls on the corneal area during the calculation. The control component 13 is equipped with machine learning capabilities. By increasing the amount of learning, it can identify flashing points that fall on the corneal area and flashing points that fall on non-corneal areas during subsequent calculations.

[0059] The eye-tracking device of this embodiment has the advantages of simple structure and easy integration. Furthermore, it can use at least one of the corneal curvature algorithm model and the pupil center corneal reflex algorithm model to calculate the gaze direction, offering high calculation accuracy. In other words, this embodiment makes the eye-tracking device compatible with both simple structure and high tracking accuracy.

[0060] Optionally, continue to refer to Figure 1 The eye-tracking device includes at least one first reflector 14, and the dense display device 11 is further provided with at least one second display area, which is configured to emit infrared vector structured light toward the first eye through reflection from the corresponding first reflector.

[0061] Specifically, in this embodiment, at least one first reflecting mirror can be used to virtually create a first virtual image 111 of multiple dense display devices, which is equivalent to using multiple light sources to emit infrared vector structured light towards the first eye from multiple angles. Since traditional methods using a single light source require external calibration to ensure a unique solution for the gaze direction when using a single pupil-center corneal reflection algorithm model, this embodiment uses the first reflecting mirror 14 to virtually create the first virtual image 111, allowing scanning of the first eye from multiple angles. This results in multiple flickering points in the first eye. The probability of all flickering points falling on the cornea is relatively small, but if even one flickering point does not fall on the cornea, that point can serve as a calibration benchmark, ensuring a unique solution for the gaze direction calculation and achieving calibration-free operation. In the above embodiment, the distance between the flickering point and the pupil center point can be used to determine whether the flickering point falls outside the cornea. For example, if the distance between the flickering point and the pupil center point is greater than a preset value, it indicates that the flickering point is outside the cornea. This preset value can be determined based on the actual situation; alternatively, the point farthest from the pupil center point can be directly considered as the point that does not fall on the cornea. Figure 2 As shown, the first detection image information will also include the pupil center point P' and the flashing point A' (A1' and A2' in this embodiment) corresponding to the first virtual image 111. Furthermore, time-division multiplexing (which will be explained later) can be used to ensure that the dense display device and the first virtual image 111 do not conflict with each other when searching for the pupil center point and the flashing point.

[0062] Alternatively, a corneal curvature algorithm model can be used alone to calculate the gaze direction. The dense display device 11 can project multi-point structured light onto the first eye, and through the virtual first image, it can achieve multi-angle scanning of the first eye. This scanning method can obtain a sufficient amount of scintillation point data. For example, for the first eye, two first probes respectively detect and return light signals to the control component. The control component can then record the position of the return signal in the scanned area and generate a scintillation point role distribution map. The scintillation point role distribution map records all known information such as the area of ​​the dense display device emitting structured light, the position of the first probe receiving the reflected light, and the elevation angle of the emitted light. Since the emitted beam is a vector, this vector information has four dimensions: up, down, left, and right. Since the corneal curvature algorithm model requires scintillation points to fall on the cornea for gaze direction resolution, projecting multi-point structured light from multiple angles ensures that a sufficient number of scintillation points fall on the cornea. Therefore, sufficient data can be input into the corneal curvature algorithm model to output the gaze direction of the first eye. It is understandable that, since the curvature of the eyes varies among different organisms, the same corneal curvature algorithm model is not applicable to all organisms. Therefore, when using a corneal curvature algorithm model alone to track the direction of eye gaze, it is preferable to use external calibration before use.

[0063] In the above embodiments, because the beam of the dense display device has a certain divergence angle, a large amount of beam covers the part outside the eyeball when scanning the eye, resulting in wasted resources. Preferably, when installing the dense display device, the angle of the dense display device can be adjusted so that the light emitted from the first display area and the light reflected by the first reflector from the second display area can both cover the first eye, thereby scanning the first eye from different angles. A corresponding first probe is set in each area covered by the beam, so that the probe in the corresponding area is only responsible for receiving the reflected or scattered light of the scanning beam in that area.

[0064] Alternatively, the control component in this embodiment can be configured to calculate a first calculation result of the gaze direction of the first eye using a pupil center corneal reflection algorithm model; and to learn a corneal curvature algorithm model using a machine learning method and the first calculation result, and to calculate a second calculation result of the gaze direction of the first eye using the learned corneal curvature algorithm model and the first scintillation point data.

[0065] Specifically, the control component can use either a corneal curvature algorithm model or a pupil center corneal reflex algorithm model individually, or a combination of both to calculate the gaze direction of the first eye. Traditional corneal curvature algorithm models require massive amounts of manually labeled data for training to achieve robustness, while this embodiment combines the two algorithms for self-learning and self-calibration without data labeling. Since no manual labeling is used initially, the specific formula for the corneal curvature algorithm f is unavailable. However, the pupil center corneal reflex algorithm model can be used to calculate the first result of the gaze direction, which is then output as standard information to the learning machine (located in the control component or in the cloud). The learning machine receives the standard information and extracts the scintillation point distribution map from the corresponding frame for machine learning, thereby learning the corneal curvature algorithm f and establishing a connection between corneal scintillation point coordinate changes and the gaze direction. This allows subsequent tracking to directly use the scintillation point distribution data into the corneal curvature algorithm f to quickly calculate the gaze direction of the eye. This embodiment can utilize two algorithm models to supplement each other, which can greatly reduce the amount of information calculation and improve the refresh rate.

[0066] Furthermore, in the above embodiments, the control component is also configured to calculate a second calculation result of the gaze direction of the first eye using the corneal curvature algorithm model when no new first calculation result is calculated using the pupil center corneal reflection algorithm model; and to calibrate the corneal curvature algorithm model using the new first calculation result after a new first calculation result is calculated using the pupil center corneal reflection algorithm model.

[0067] Specifically, when using the central pupil corneal reflex algorithm model, it is necessary to obtain the coordinates of the blinking point and the pupil center point. However, when using the corneal curvature algorithm model, only the blinking point coordinates are needed. Since obtaining the blinking point coordinates is faster than obtaining the pupil center point coordinates (approximately ten times faster), this embodiment sets the corneal curvature algorithm model to calculate the second calculation result of the gaze direction as soon as new blinking point coordinates are obtained, and outputs this second calculation result as the gaze direction. After calculating the new pupil center point coordinates, the central pupil corneal reflex algorithm model is used to calculate the first calculation result of the gaze direction, which is then output as the gaze direction of the first eye. This first calculation result is used as standard information and input into the learning machine. The learning machine extracts the blinking point role distribution data from the corresponding frame position of the standard information for learning, thereby updating the corneal curvature algorithm model and making it increasingly accurate with use. Through this method, real-time calibration and real-time learning of the corneal curvature algorithm model can be achieved without external calibration, resulting in a more accurate calculated gaze direction.

[0068] In the above embodiments, the first eye is used as an example for illustration. It can be understood that an organism generally contains two eyes, a first eye and a second eye. Therefore:

[0069] Further reference Figure 1 The eye-tracking device further includes: a semi-reflective mirror 15 and a second detection component 16; the dense display device 11 is also provided with a third display area, which is configured to emit infrared vector structured light to the second eye through the semi-reflective mirror; the second detection component 16 includes at least two second probes, which are configured to receive light reflected or scattered by the second eye to obtain second detection image information, which includes the center point of the second pupil and at least one second scintillation point; the control component 13 is electrically connected to the second detection component 16 and is configured to drive the dense display device to emit light and calculate the gaze direction of the second eye based on the second detection image information; wherein, the control component 13 is configured to calculate the gaze direction of the second eye using a corneal curvature algorithm model and / or a pupil center corneal reflection algorithm model.

[0070] Specifically, the second detection component 16 is similar to the first detection component, the second detection image information is similar to the first detection image information, and the calculation process of the gaze direction of the control component 13 for the second eye is the same as the calculation process for the gaze direction of the first eye, which will not be repeated here. It should be noted that in this embodiment, no additional light source is set for the second eye. Instead, a semi-reflective lens is used to separate the dense display device into a second dense display device 112, so that the light emitted from different display areas in the dense display device is emitted to different eyes. The light emitted from the third display area is emitted to the second eye through the semi-reflective lens, thereby reducing the need for a light source and allowing light that originally illuminated the non-eyeball part of the first eye to be emitted to the second eye, thereby improving the utilization rate of the dense display device.

[0071] Further reference Figure 1 The eye-tracking device also includes at least one second reflector 17, and the dense display device is further provided with at least one fourth display area, which is configured to emit infrared vector structured light toward the second eye through the reflection of the semi-reflective lens 15 and the corresponding second reflector 17.

[0072] Specifically, similar to the function of the first reflector, by setting the second reflector 17, the semi-reflective mirror can further virtualize the second dense display device 112 into a second virtual image 113, thereby enabling multi-angle scanning of the second eye and further improving the accuracy of calculating the gaze direction of the second eye.

[0073] In the above embodiment, the semi-reflective mirror 15 is positioned on the glasses at the middle position corresponding to both eyes, and is placed as far away from the frame as possible to be close to the eyes without affecting the display.

[0074] When installing dense display devices, fiber optic probes, semi-reflective mirrors, and reflectors, the following principles should be followed: place them as close to the eye as possible without affecting the display; disperse them as much as possible to achieve multi-angle detection; ensure that the dense display device can generate a corresponding detection map when it is illuminated, that is, the detection map contains the pupil, the blinking point, and the virtual image corresponding to the pupil and the blinking point.

[0075] Optionally, continue to refer to Figure 1In the above embodiments, the first probe 121 and the second probe 161 can be fiber optic probes. In this case, the first detection component may further include a first photodetector 122, and the second detection component may further include a second photodetector 162. The first photodetector 122 is optically connected to the fiber optic probe and electrically connected to the control component; the second photodetector 162 is optically connected to the second probe and electrically connected to the control component 13. One end of the fiber optic probe (including the first and second probes) is fixedly distributed inside the frame, acting as a sensor to receive light reflected or scattered from the eye. A device to help couple spatial light into the fiber optic probe can be provided at the top of the probe to improve the signal intensity, such as a coupling lens; alternatively, a color filter can be added to reduce interference. When setting up the fiber optic probes, they should be as close to the eye as possible without affecting the display; and they should be set up as dispersedly as possible to achieve multi-angle detection. Each fiber optic probe has a corresponding detection area, and distance factors can be considered during setup to ensure that the received projection signals do not overlap. The coupling between the fiber optic probe and the photodetector can be achieved in various ways (e.g., N groups of one-fiber coupling, N groups of two-fiber coupling, or multiple fiber optic connectors before being connected to the photodetector). Combining the optical signals allows for better bonding and installation of multiple fibers, saving on photodetectors and reducing their configuration, thus optimizing costs. It also optimizes circuit design, simplifying the design and making installation easier. The photodetector converts the combined optical signal into a single electrical signal, amplifies it, and transmits it to the control components. Alternatively, the first and second probes can be directly configured as photodetectors, but this would require a larger number of photodetectors in the eye-tracking device, increasing costs.

[0076] Preferably, the control component 13 includes a learning machine configured to learn a corneal curvature algorithm model. Specifically, the corneal curvature algorithm model needs to be acquired through learning. The learning machine in the control component 13 can be a remote learning machine or a local learning machine; the present invention does not impose any particular limitation on this.

[0077] Figure 5 A flowchart of an eye-tracking method provided in an embodiment of the present invention is shown below. Figure 5 Eye-tracking methods include:

[0078] In step S301, the control component drives the dense display device to emit light and calculates the gaze direction of the first eye based on the first detected image information; wherein, the control component is configured to calculate the gaze direction of the first eye using a corneal curvature algorithm model and / or a pupil center corneal reflection algorithm model.

[0079] Specifically, the control component drives the dense display device to emit light, causing the first display area to emit infrared vector structured light to the first eye, and the first detection component detects the light reflected or scattered by the first eye. The control component generates first detection image information based on the light detected by the first detection component. The first detection image information includes the center point of the first pupil and at least one first flicker point. The dense display device can generate at least one first flicker point on the first eye. When calculating using the corneal curvature algorithm model, the first flicker point must fall in the corneal area for the calculation result to be reliable. When calculating using the pupil center corneal reflection algorithm model, the first flicker point must fall in the non-corneal area for the calculation result to be reliable. In this embodiment, the dense display device can generally generate multiple first flicker points on the first eye. Among the multiple first flicker points, there must be flicker points that fall in the corneal area and flicker points that fall in the non-corneal area. Furthermore, the coordinates of the center point of the first pupil can be obtained through the first detection image information, and then at least one of the corneal curvature algorithm model and the pupil center corneal reflection algorithm model can be used to calculate the gaze direction of the first eye. Because two algorithm models can be used to calculate the gaze direction of the first eye, the accuracy and efficiency of gaze direction calculation can be greatly improved, and the compatibility is also stronger. It should be noted that there is no need to determine whether the flickering point falls on the corneal area during the calculation. The control component 13 is equipped with machine learning capabilities, and by increasing the amount of learning, it can identify flickering points that fall on the corneal area and flickering points that fall on non-corneal areas during subsequent calculations.

[0080] Optionally, Figure 6 A flowchart of another eye-tracking method provided in an embodiment of the present invention is shown below. Figure 6 Eye-tracking methods may specifically include:

[0081] Step S401: The control component drives the dense display device to perform a global scan, obtains the global scan to form a first eye detection map, and calculates the pupil position and the corresponding first flashing point position of the first eye based on the first eye detection map.

[0082] Specifically, when the eye-tracking device also includes a semi-reflective mirror and a second detection component, this step may further include: controlling the component to drive the dense display device to perform a global scan, acquiring a second eye detection map formed by the global scan, and calculating the pupil position and the corresponding second flicker point position of the second eye based on the second eye detection map. The first and second detection image information can be acquired simultaneously. After the eye-tracking device is powered on, a blind scan can be initiated first, that is, a global scan of the first and second eyes can be performed using a single-point scan method. For example, it can be a point-by-point blind scan method, recording the brightness information fed back by each point after it is scanned. Taking the first eye as an example, the first eye detection map formed after the point-by-point blind scan can also be referenced. Figure 2 , Figure 2The image contains both bright and dark areas AA. Based on the characteristics of the eye, the darkest area AA in the detection image can be considered the pupil area to be tracked, and the brightest point is the flicker point. It should be noted that when the light beam scans an area outside the pupil, the beam will be directly reflected, and the detection component will receive relatively strong brightness information, thus forming a bright area in the detection image. However, when the light beam scans the pupil area, the beam is reflected multiple times within the pupil, resulting in weaker brightness information or the beam not being received by the detection component, thus forming a dark area. Image processing techniques can be used to calculate the major axis D0 of the dark area AA, the ratio β0 of the major axis to the minor axis, and the angle θ0 between the first eye and the dense display device. The coordinates P(x0, y0) of the center of the dark area (pupil center) and the flicker point A1(x0, y0) can also be determined in the detection image. A1 ,y A1 ) and A2(x A2 ,y A2 The coordinates of the pupil are shown in the image. Simultaneously, the coordinates of the virtual pupil center P'(x'0,y'0) projected by the reflector, and the virtual blinking point A1'(x'0,y'0) can also be seen in the detection image. A1 ,y' A1 ) and A2'(x' A2 ,y' A2 The coordinates of ).

[0083] In step S402, the control component determines the pupil region displacement range of the next frame based on the pupil region of the current frame and the first preset formula, determines the first flashing point displacement range of the next frame based on the position of the first flashing point of the current frame and the second preset formula, and drives the dense display device to scan only the pupil region displacement range and / or only the first flashing point displacement range in the next frame.

[0084] Specifically, after obtaining the detection map through global scanning and locating the coordinates of the pupil center and corneal scintillation point, the system enters the structured light tracking mode. The advantage of structured light scanning is that it can scan only the "useful" information areas, i.e., only tracking the positions within the pupil's movement range and the corneal scintillation point's movement range, and establishing a data structure change relationship between the structured light and the pupil center and scintillation point. This method converts image processing into digital signal processing, greatly reducing computational pressure and the number of scans, and improving processing speed; it can also improve the signal-to-noise ratio by increasing the number of scans. After each global scan, a detection map of the eye is obtained. Through image processing techniques, a set of data structures M0 and a virtual data structure M'0 within the reflector are obtained. We define i as the previous frame, i+1 as the current frame, and quantities without subscripts are the current measurement values, which can be calculated using image analysis methods. After global scanning, the following data structures M0 and M'0 can be obtained through image processing.

[0085]

[0086]

[0087] Where PC is the center of the pupil and CR is the scintillation point.

[0088] Subsequently, based on the i-frame data, the displacement range of the pupil region and the displacement range of the flickering point in frame i+1 can be determined. For example... Figure 7 As shown, Figure 7 The schematic diagram for determining the pupil region displacement range uses the known viewing angle and coordinates of a dense display device as a reference, setting Δt as the interval between scan frames, and ε... max ω represents the maximum moving pixel at the center of the pupil within Δt. max ε is the fastest rotational speed of the eyeball. max =α1×Δt×ω max α1 is the structure coefficient, which is related to the relative positions of the sensing device, the dense display device, and the eye, and can be obtained through static correction. To ensure that the pupil center remains within the scanning range of the dense display device within the Δt interval, a rectangle along the major axis of the pupil in frame i is used to define the scanning range for the first structured light tracking of the pupil in frame i+1. The center of the rectangle is the same as the pupil center, and the major axis of the rectangle is D. i +2ε max -d, the minor axis of the rectangle is The first preset formula includes the formula corresponding to the major axis and the formula corresponding to the minor axis.

[0089] After determining the displacement range of the pupil region, the first structured light 50 can be used to track and scan within the pupil displacement range. The first structured light 50 can be a circular structured light with a diameter of d. When d = 0, it reverts to single-point (pixel) scanning mode, and the dark area AA in the obtained detection image is the pupil region. When d > 0, to ensure that the center of the first structured light is always within the pupil range, the diameter d of the assumed circular structured light is much smaller than the pupil size, and the minor axis of the pupil region in the i-frame detection image is used as the reference. The control component drives the corresponding pixels in the dense display device to emit light, thereby emitting a circular structured light with a diameter of d. Of course, in other embodiments, other forms of structured light, such as elliptical structured light, or other superior structured light, can also be used for scanning. When recording the brightness information returned by each pixel in the circular structured light, the coordinates of the center point of the circular structured light are also recorded. In this case, in the detection map scanned using a circular structured light with a diameter of d (d>0), the boundary of the dark region AA2 is not the actual pupil boundary; the actual pupil boundary is a larger region that is coaxial and concentric with the dark region AA2. Figure 7 (The dashed area is 70°). Assuming the length of the major axis of the dark region is ΔD, image processing techniques can be used to determine ΔD and θ. i+1Based on the relationship between circular structured light and the pupil, the major axis D of the pupil in the current frame... i+1 =ΔD+d, Pupil center P i+1 The coordinates are updated to (x i+1 ,y i+1 ),x i+1 =Δx,y i+1 =Δy. By tracking and scanning the pupil, the coordinates of the pupil center point in frame i+1 can be updated in real time. Traditional single-point scanning methods require scanning... In this embodiment, the number of scans required within the pupil region displacement range is proportionally reduced to the scan area compared to the traditional full scan method; and with circular structured light scanning, the number of scans is further reduced to [number missing]. Furthermore, as long as d > 0, the number of scans can be greatly reduced.

[0090] The scanning range of the first flashing point in frame i+1 is determined based on the position of the first flashing point in frame i. The principle for determining the scanning range of the first flashing point is as follows: Due to the structural characteristics of the eye (the curvature of the cornea is different from that of the eyeball), the flashing point on the cornea changes when the eye rotates or when the relative position between the dense display device and the eye changes. Unlike pupil movement, the change in the corneal flashing point is very small when the relative position between the dense display device and the eye does not change. The maximum movement of the corneal flashing point within Δt is set to N. max N max =α2×Δt×ω max N max The corresponding formula is the second preset formula, where α2 is the structural coefficient, which is related to the relative positions of the sensing device, the dense display device, and the eye, and can be obtained through static correction. The flicker point A(x) is used as an example. A ,y A For example, taking the known coordinates of corneal scintillation point A as the center, the scanning range for tracking scintillation point A is 2N centered on scintillation point A. max ×2N max Rectangle, (x A± N max ,y A± N max ),like Figure 8 As shown, Figure 8 The schematic diagram is shown for determining the displacement range of the first scintillation point. After determining the displacement range of the first scintillation point, the dense display device can be driven to emit line structured light to scan within the displacement range of the first scintillation point. The cross-section of the line structured light consists of two mutually perpendicular line segments. It should be noted that the scanning range of the line structured light cannot exceed the detection area of ​​the original fiber optic probe (or photodetector).

[0091] Specifically, using line structured light to trace and scan flashing points can be done as follows: within a defined flashing point scanning range of frame i+1, a length L = 2N concentric with the flashing points is used. max The short-line structured light (second structured light) tracks and scans the flashing point A in the 0-degree (x-axis) and 90-degree (y-axis) directions, recording the brightness information and position returned by each pixel of the short-line structured light, and generating a brightness information curve. The position corresponding to the pixel with the highest brightness in the x-axis and y-axis directions is the flashing point A. i+1 The coordinates of the scintillation points are determined. Brightness information is detected by the fiber optic probe in the corresponding area and transmitted back to the photodetector, where it is converted into an electrical signal for processing by the control components. The coordinates of the scintillation points are the coordinates of the highest brightness recorded in the horizontal and vertical directions within the scanning range of each scintillation point. The scintillation point coordinates of frame i are used to determine the center of the scintillation point scanning range for frame i+1. The data structure updates in the virtual image follow the same principle.

[0092] If a single-point scanning method is used within a defined flash point scanning range, it is necessary to scan (2N). max ) 2 The number of scans decreased again to 4N when using short-line structured light scanning; however, using short-line structured light scanning further reduced the number of scans to 4N. max Of course, short-line structured light is not necessarily the optimal structured light. This embodiment is just an example. There are also better structured light and algorithms to improve scanning efficiency and reduce the number of scans.

[0093] In this embodiment, since only the pupil displacement range or the first flash point displacement range is scanned, the amount of information calculation can be greatly reduced, thereby reducing the computational pressure and improving the output efficiency.

[0094] Furthermore, it should be noted that in this embodiment, when the dense display device emits the first and second structured light, each pixel corresponding to the structured light emits light simultaneously within a frame, rather than using a point-by-point scanning method. This significantly improves the scanning speed compared to point-by-point scanning. Moreover, using a multi-point simultaneous emission method for scanning results in a higher signal-to-noise ratio than single-point scanning, making the received signal closer to a real analog signal with clearer boundaries. Simultaneously, excessively high single-point emission power can easily damage the human eye. Under the same power conditions, multi-point structured light scanning can disperse the power of a single point, allowing for scanning a larger area while maintaining the same single-point power, thus increasing the overall power.

[0095] In addition, since the pupil center and blinking point are tracked in real time, the pupil center or blinking point may be lost in some cases. Once lost, it is necessary to return to step S401 and re-scan the global area to ensure the accuracy of the tracking.

[0096] In addition, in this embodiment, when the pupil center corneal reflection algorithm model is used for calculation, it can be applied to dense display devices with low gain; while when the corneal curvature algorithm model is used, it can be applied to dense display devices with high gain.

[0097] Preferably, since structured light of a corresponding shape is used for scanning and the dense display device contains multiple pixels, the control component can drive the dense display device, so that the corresponding structured light can be switched to the defined displacement range at any time, making the tracking more efficient.

[0098] Step S302: Calculate the first calculation result of the gaze direction of the first eye using the pupil center corneal reflection algorithm model; and learn the corneal curvature algorithm model using machine learning methods and the first calculation result, and calculate the second calculation result of the gaze direction of the first eye using the learned corneal curvature algorithm model and the first scintillation point data.

[0099] Specifically, the control component employs a combination of two algorithms to calculate the gaze direction of the first eye. Traditional corneal curvature algorithm models require massive amounts of manually labeled data for training to achieve robustness. However, this embodiment's combination of two algorithms enables self-learning and self-calibration without the need for data labeling. When the specific formula for the corneal curvature algorithm f is unavailable, the first calculation result of the gaze direction calculated by the pupil center corneal reflection algorithm model can be output as standard information to the learning machine. The learning machine receives this standard information and extracts the scintillation point distribution map from the corresponding frame for machine learning, thereby learning the corneal curvature algorithm f. This establishes a connection between changes in corneal scintillation point coordinates and the gaze direction, allowing the scintillation point distribution map to be substituted into the corneal curvature algorithm f during subsequent tracking, quickly calculating the gaze direction of the eye. This embodiment utilizes two algorithm models to supplement the calculations, significantly reducing the amount of information computation and improving the refresh rate.

[0100] Step S303: Does the control component calculate a new first calculation result? If yes, then proceed to step S304; if no, then proceed to step S305. After step S304, return to step S303. Step S304 is to calibrate the corneal curvature algorithm model using the new first calculation result. Step S305 is to calculate the second calculation result of the gaze direction of the first eye using the corneal curvature algorithm model.

[0101] Specifically, when using the central pupil corneal reflex algorithm model, it is necessary to obtain the coordinates of the blink point and the pupil center point. When using the corneal curvature algorithm model, only the blink point coordinates are needed. Since obtaining the blink point coordinates is faster than obtaining the pupil center point coordinates, this embodiment sets the process such that as soon as new blink point coordinates are available, the corneal curvature algorithm model is used to calculate the second calculation result of the gaze direction, and this second calculation result is output as the gaze direction. After calculating the pupil center point coordinates, the central pupil corneal reflex algorithm model is used to calculate the first calculation result of the gaze direction, and this first calculation result is output as the gaze direction. Simultaneously, this result is fed into the learning machine to learn and update the corneal curvature algorithm model, making it more accurate. Subsequently, the new corneal curvature algorithm model is used to calculate the second calculation result of the gaze direction for the first eye. Through this method, real-time calibration of the corneal curvature algorithm model can be achieved, resulting in a more accurate calculated gaze direction.

[0102] Preferably, in the above embodiment, the control component is configured to drive the first display area and the second display area in a time-division manner. Since both the first and second display areas illuminate the first eye, simultaneous illumination might prevent the control component from distinguishing which display area of ​​the light source produced the flickering point and the pupil center point. Therefore, this embodiment employs a time-division driving method, ensuring that the first and second display areas illuminate the first eye at different times, preventing interference. It is also understood that the control component time-division drives the corresponding area on the dense display device, ensuring that the second dense display device and the second virtual image do not interfere with each other when illuminating the second eye.

[0103] For example, assuming the resolution of the dense display device is 384*384 and the field of view is 50mm*50mm, the fastest rotational speed of the human eye is ω. max The rotation speed is 720° / s, and the radius of eyeball rotation is 13.5 mm. Assume D = 40 and β = 2. A circular structured light source requires πd points (31 points) to illuminate one ring. This is calculated using the formula for the number of scans. Tracking a single pupil center (PC) requires approximately 1250 scans per frame, and scanning both PCs for the left and right eyes requires a total of 2500 scans. The virtual PC' image projected by the first mirror is smaller, and the maximum pixel movement is also smaller. Assuming the virtual image in the mirror is one-third the size of the real image, then D' = 13, ε' max =4, d'=2, completing the tracking of PC' in the two virtual images requires approximately 400 scans per frame. Therefore, it can be calculated that when a dense display device allocates 400K frames / second to track all PCs and PC', the refresh rate can reach approximately 130Hz.

[0104] Similarly, when tracking blinking points (CR), assume Nmax =5, calculated according to the CR scan count formula 4N max Each frame requires 20 scans to complete one CR scan, while completing all CR tracking requires 2 × 2 × 4 × 20 = 320 scans. When a dense display device tracks all CRs and CR's at a rate of 300K frames per second, the refresh rate can reach 1000Hz.

[0105] Furthermore, assuming that the dense display device performs a global scan of the human eye at a rate of 300K (384×384), completing the scan of both eyes requires 384×384=300K scans per frame. With a refresh rate of approximately 2Hz, it can guarantee that two global scans can be completed per second to obtain the human eye image. The gaze direction of the human eye can then be directly calculated using the pupil-center corneal reflection algorithm model. It can also be used as gaze output in cases of lost tracking or eyelid occlusion, or as standard and calibration information for learning. Input is given to the learning machine.

[0106] Of course, it should be noted that the rate allocation in software time-division multiplexing is only an example, and there may be better allocation methods. The refresh rate of dense display devices can also be further improved through chip upgrades and sensor transmission rate upgrades. This dense display device supports switching between three driving modes (serial, narrow parallel, and wide parallel) to achieve higher refresh rates; in this example, only the lowest refresh rate driving mode (serial) is used for estimation.

[0107] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0108] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An eye tracking device, characterized in that, The eye movement tracking device comprises: A dense display device provided with a first display area configured to emit infrared vector structured light to the first eye; A first detection assembly comprising at least two first probes, the first detection assembly being configured to receive light reflected or scattered by the first eye to obtain first detection image information, the first detection image information comprising a first pupil center point and at least one first glint point; wherein the first glint point is a glint point generated by the first display area of the dense display device irradiating on the cornea; the first pupil center point is the pupil center corresponding to the first eye; A control assembly electrically connected with the dense display device and the first detection assembly, configured to drive the dense display device to emit light, and calculate the gaze direction of the first eye according to the first detection image information; Wherein, the control assembly is configured to calculate the gaze direction of the first eye by using a corneal curvature algorithm model and a pupil center corneal reflection algorithm model; the control assembly is configured to calculate a first calculation result of the gaze direction of the first eye by using the pupil center corneal reflection algorithm model; and learn a corneal curvature algorithm model by using a machine learning method and the first calculation result, and calculate a second calculation result of the gaze direction of the first eye by using the learned corneal curvature algorithm model and first glint point data.

2. The eye movement tracking device according to claim 1, wherein The control assembly is further configured to calculate a second calculation result of the gaze direction of the first eye by using the corneal curvature algorithm model when a new first calculation result is not calculated by using the pupil center corneal reflection algorithm model; After a new first calculation result is calculated by using the pupil center corneal reflection algorithm model, the new first calculation result is used to calibrate the corneal curvature algorithm model.

3. The eye-tracking apparatus of claim 1, wherein, The eye movement tracking device further comprises at least one first mirror; the dense display device is further provided with at least one second display area, the second display area being configured to emit infrared vector structured light to the first eye through reflection of the corresponding first mirror.

4. The eye-tracking apparatus of claim 1, wherein, The eye movement tracking device further comprises a half mirror and a second detection assembly; the dense display device is further provided with a third display area, the third display area being configured to emit infrared vector structured light to the second eye through the half mirror; The second detection assembly comprises at least two second probes, the second detection assembly being configured to receive light reflected or scattered by the second eye to obtain second detection image information, the second detection image information comprising a second pupil center point and at least one second glint point; The control assembly is electrically connected with the second detection assembly, configured to drive the dense display device to emit light, and calculate the gaze direction of the second eye according to the second detection image information; Wherein, the control assembly is configured to calculate the gaze direction of the second eye by using the corneal curvature algorithm model and the pupil center corneal reflection algorithm model.

5. The eye-tracking apparatus of claim 4, wherein, The eye movement tracking device further comprises at least one second mirror, and the dense display device is further provided with at least one fourth display area, which is configured to transmit infrared vector structured light to the second eye through the half-mirror and reflection of the corresponding second mirror.

6. The eye-tracking apparatus of claim 1, wherein, The control component further comprises a learning machine configured to learn a corneal curvature algorithm model.

7. The eye-tracking apparatus of claim 1, wherein The first probe is an optical fiber probe, and the detection component further comprises a first photodetector optically connected to the optical fiber probe and electrically connected to the control component. Alternatively, the first probe is a photodetector, and the first probe is electrically connected to the control component.

8. An eye tracking method performed by the eye tracking device of any one of claims 1-7, characterized by, The eye movement tracking method comprises: The control component drives the dense display device to emit light and calculates the gaze direction of the first eye according to the first detection image information; The control component is configured to calculate the gaze direction of the first eye by using a corneal curvature algorithm model and a pupil center corneal reflection algorithm model, calculate a first calculation result of the gaze direction of the first eye by using the pupil center corneal reflection algorithm model, learn a corneal curvature algorithm model by using a machine learning method and the first calculation result, and calculate a second calculation result of the gaze direction of the first eye by using the learned corneal curvature algorithm model and the first glint point data.

9. The eye movement tracking method of claim 8, wherein The control component is further configured to calculate a second calculation result of the gaze direction of the first eye by using the corneal curvature algorithm model when the pupil center corneal reflection algorithm model does not calculate a new first calculation result; After calculating a new first calculation result by using the pupil center corneal reflection algorithm model, the control component calibrates the corneal curvature algorithm model by using the new first calculation result.

10. The eye-tracking method of claim 8, wherein, Before the control component drives the dense display device to emit light and calculates the gaze direction of the first eye according to the first detection image information, the method further comprises: The control component drives the dense display device to perform global scanning and obtains a first eye detection image formed by the global scanning, and calculates the pupil position of the first eye and the corresponding first glint point position according to the first eye detection image.

11. The eye-tracking method of claim 10, wherein, The eye movement tracking device further comprises a half-mirror and a second detection component, and the dense display device is further provided with a third display area configured to transmit infrared vector structured light to the second eye through the half-mirror; the second detection component comprises at least two second probes, and the second detection component is configured to receive light reflected by the second eye to obtain second detection image information comprising a second pupil center point and at least one second glint point; and the control component is electrically connected to the second detection component. The eye movement tracking method further comprises that the control component drives the dense display device to emit light and calculates the gaze direction of the second eye according to the second detection image information; and the control component is configured to calculate the gaze direction of the second eye by using a corneal curvature algorithm model and a pupil center corneal reflection algorithm model. Before the control component drives the dense display device to emit light and calculates the gaze direction of the second eye according to the second detection image information, the method further comprises: The control component drives the dense display device to perform global scanning, obtains a second eye detection image formed by the global scanning, and calculates the pupil position of the second eye and the corresponding second glint position according to the second eye detection image.

12. The eye-tracking method of claim 10, wherein, The control component drives the dense display device to emit light and calculates the gaze direction of the first eye according to the first detection image information comprises: The control component determines the displacement range of the pupil region of the next frame according to the current frame pupil region and a first preset formula, determines the displacement range of the first glint of the next frame according to the current frame first glint position and a second preset formula, and drives the dense display device to only scan the displacement range of the pupil region and / or only scan the displacement range of the first glint in the next frame.

13. The eye-tracking method of claim 12, wherein, Driving the dense display device to only scan the displacement range of the pupil region and the displacement range of the first glint comprises: Driving the dense display device to emit the first structured light calculated according to the pupil region of the current frame to scan the displacement range of the pupil region of the next frame; wherein the first structured light is a positive circular structured light, or a structured light with an outline corresponding to the pupil outline of the first eye of the current frame; Driving the dense display device to only scan the displacement range of the first glint comprises: Driving the dense display device to emit a line structured light to scan the displacement range of the first glint; wherein the cross section of the line structured light is two line segments perpendicular to each other.

14. The eye-tracking method of claim 8, wherein, The eye movement tracking device further comprises a first reflector; the dense display device is further provided with a second display region, and the second display region is configured to emit infrared vector structured light to the first eye through reflection of the first reflector. The control component is further configured to drive the first display region and the second display region to emit light in time sharing mode.

Images