Gaze tracking error compensation or parameter calibration method and system based on gaze trajectory

Abstract

The invention belongs to the field of computer vision technology, and discloses a gaze tracking error or parameter calibration compensation method, system, device and medium based on gaze trajectory, which are used for gaze landing point error compensation or mapping model parameter calibration of a gaze tracking system. By tracking a moving point of a given trajectory, human eye images of different time series are selected to build a mapping model, and the shape information of the gaze trajectory and the least square method are used to solve the compensation table coefficients or calibrate the model parameters. The invention can be applied to various gaze estimation models including the cross-ratio method; a corresponding gaze estimation model is established according to the corresponding gaze estimation method, and parameter compensation is performed by a gaze tracking parameter compensation method based on the gaze trajectory after gazing at a given trajectory; while reducing the difficulty and workload of the error compensation or parameter calibration operation process, the calculation accuracy of the gaze mapping model is guaranteed, and the invention is applicable to various gaze tracking algorithms including the cross-ratio method, the corneal reflection method and the like.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a gaze tracking error compensation or parameter calibration method and system based on gaze trajectory. Background Art

[0002] The human eye is the main source of information for the brain. Eye tracking is a technology that uses eye movement information to calculate the direction of sight or the point of focus. It is widely used in human-computer interaction, medical assistance, advertising, driving assistance, etc., and has good application prospects in virtual reality, augmented reality and other fields. Depending on the installation position of the human eye camera, there are two types of scenarios: near-eye sight tracking and long-distance sight tracking, with different solutions such as multiple cameras and single cameras. In the problem of near-eye sight tracking, the camera is usually installed on the frame of the glasses worn, and is mainly used in medical assistance, driving assistance and other occasions; in the problem of long-distance sight tracking, it is usually installed near the screen, such as the top of a laptop screen, keeping a certain distance from the human eye, and is mainly used for human-computer interaction, advertising and so on.

[0003] Eye tracking technology is divided into two categories: model-based eye estimation method and feature learning-based eye estimation method. Both research and application are mainly based on single-camera solutions. Feature learning-based methods require a large amount of training data, and the results obtained by current machine learning models are a black box that is difficult to interpret, and their practical value is currently limited.

[0004] The model-based line of sight estimation method mainly relies on the physiological structure of the human eye, constructs a two-dimensional mapping model or a three-dimensional geometric model, and calculates the line of sight direction or gaze point of the human eye based on geometric information such as facial key points, pupils, and eye corners, which has good practical value. The two-dimensional mapping model method can be subdivided into three types: corneal reflection method, cross-ratio method, and homography normalization method. Both the corneal reflection method and the homography normalization method require parameter calibration for key parameters such as the human eye kappa angle and homography matrix; the cross-ratio method uses the cross-ratio projection invariance to estimate the line of sight, thereby avoiding the parameter calibration problem, but it is necessary to establish an error compensation table for the gaze point to compensate for the errors caused by the kappa angle.

[0005] The two-dimensional mapping model method mainly calibrates the parameters of the human eye or establishes an error compensation table through the pupil or iris center. The three-dimensional geometric model method uses statistical analysis or machine learning to calculate the parameters of the three-dimensional geometric model, which requires more basic data or data training.

[0006] Regardless of the method used, single-camera gaze tracking requires a large amount of actual measurement work based on the characteristics of the gaze tracking model, and there are problems with the large workload of system error modeling and calibration data.

[0007] Due to factors such as the kappa angle between the human eye's line of sight and the visual axis, the line of sight tracking model error, and the operational error during the experiment, there must be a deviation between the calculated line of sight landing point and the actual line of sight landing point. Line of sight tracking error modeling and parameter calibration require a necessary number of line of sight landing point measurements and estimates. The commonly used method is to record the gaze point calculation position of several screen calibration points, solve the mapping function between the gaze point calculation position and the calibration position, and perform system error compensation or model parameter calibration. Usually at least nine or more measurement points are used. The calibration process requires a long period of gaze at each calibration point, which requires high skills from the user, and long-term gaze will cause discomfort to the human eye.

[0008] Through the above analysis, the problems and defects of the prior art are as follows:

[0009] The traditional gaze tracking system requires a lot of work in gaze tracking error compensation or parameter calibration, has high requirements on user skills, and has a poor operating experience. Summary of the invention

[0010] In view of the problems existing in the prior art, the present invention provides a gaze tracking error compensation or parameter calibration method and system based on gaze trajectory.

[0011] The present invention is implemented as follows: a method for calibrating sight tracking parameters or compensating errors based on gaze trajectory, wherein the method for calibrating sight tracking errors or parameters based on gaze point motion trajectory comprises:

[0012] By tracking the moving point of a given trajectory, human eye images of different time series are selected to build a mapping model according to the selected line of sight estimation method, and the corresponding line of sight landing point is calculated; the compensation table coefficients of the system are solved or the model parameters are calibrated using the shape information of the line of sight trajectory and the least squares method.

[0013] Error compensation is represented by the cross ratio method in the two-dimensional mapping model method, and parameter calibration is represented by the corneal reflection method in the two-dimensional mapping model method. There are also feature learning-based methods for the three-dimensional geometric model method, which can also be used to calibrate model parameters based on the gaze point motion trajectory, or to establish a large sample training data set.

[0014] Among them, the compensation using trajectory shape information is to obtain the position and shape adjustment parameters of the trajectory through the position of special points, and the least squares method is used to solve the compensation table coefficients or calibrate the model parameters of the system. This is a method of tracking the gaze trajectory, which takes advantage of the fact that a large number of sight points are quickly and conveniently obtained, and the information of the sight points is brought into the least squares method to solve the compensation table coefficients or calibrate the model parameters.

[0015] Further, construct the sight line model and calculate the sight line landing point:

[0016] The point where the line of sight falls on the screen is calculated according to the line of sight estimation method used, such as the cross-ratio method. The line of sight mapping model is constructed through the cross-ratio invariance to calculate the position of the line of sight falling on the screen.

[0017] Furthermore, the gaze point is corrected using the gaze trajectory shape information, and the specific process is as follows:

[0018] By gazing at the uniformly moving point on the calibration screen, the trajectory of the moving point is a regular semicircle or circle.

[0019] Furthermore, the trajectory of the sight point on the calibration screen is calculated frame by frame. For the convenience of processing, points with special angles on the trajectory (such as 0°, 180°, etc.) are selected and compared with the original trajectory of the moving point to obtain the angle θ and the center point position (x' 0 ,y' 0 ).

[0020] By gazing at the uniformly moving points on the calibration screen, the problem of gaze point alignment in the point-by-point calibration process is avoided. At the same time, by recording the time information of the camera's video of the human eye and the movement of the moving points on the screen, the corresponding time relationship is utilized, and the line of sight mapping model is used to quickly calculate the line of sight position of each point, which greatly reduces the workload of point measurement compared to multi-point calibration.

[0021] Furthermore, according to the positions of special points, such as the two end points of the diameter, the corresponding relationship between the original gaze trajectory and the fitted gaze trajectory is calculated:

[0022] like Figure 5 As shown, the angle between the fitting trajectory and the original trajectory is denoted as θ, the radius of the original trajectory is denoted as R, and the center coordinates are The radius of the fitting trajectory is denoted as r, and the coordinates of the center of the circle are (x' 0 ,y' 0 );

[0023] The original trajectory of the moving point on the screen is recorded as a function:

[0024]

[0025] The sight point trajectory calculated based on the camera raster image and a certain sight tracking model is a closed curve f containing the trajectory error. image (x′, y′). Obviously, f image (x′, y′) must correspond to the circle of the original trajectory, which can be written as:

[0026] f image (x′, y′, model(k, d1, d2,...))=(xx′0 ) 2 +(yy′ 0 ) 2 -r 2 =0#(2)

[0027] in, (x' 1 , y' 1 ) and (x' 2 , y' 2 ) are the two endpoints of the sight point trajectory calculated according to the sight tracking model, corresponding to the starting point and end point of the original semicircular arc trajectory of the screen moving point respectively; (x 0 ,y 0 ) and R are the center and radius of the original semicircular arc trajectory of the moving point on the screen. model(k, d1, d2, ...) represents the gaze tracking model, where k, d1, d2, etc. are system parameters in the gaze tracking model.

[0028] According to the position correspondence between the two endpoints of the semicircle diameter, the projection transformation matrix between the sight point trajectory and the original trajectory is obtained, including arc radius adjustment, rotation transformation, and translation transformation, where:

[0029] First, adjust the arc radius and the proportional adjustment matrix T p Adjust the radius of the fitted arc to the radius of the original arc, that is, adjust the shape of the fitted arc to the proportional coefficient of Proportional enlargement of:

[0030]

[0031] Fit the line of sight landing point trajectory to any calculated position after scaling up:

[0032]

[0033] Then, the rotation transformation matrix T R Align the diameters of the semicircular arcs:

[0034] Angle

[0035] If y′ 1 -y′ 2 If θ>0, θ is considered to be a positive angle, otherwise, θ is considered to be a negative angle.

[0036]

[0037] Calculate the position of any landing point after rotating the fitted sight landing point trajectory:

[0038]

[0039] Translation transformation matrix T T Align the centers of the two arcs:

[0040]

[0041] Calculate the position of any landing point after the fitted sightline landing point trajectory coincides with the original semicircle trajectory:

[0042]

[0043] Furthermore, the sight point trajectory f image The positions of all points on (x′, y′, model (k, d1, d2, ...)) must satisfy the circular trajectory equation described in formula (1). Therefore, the calculated positions of each moving point in all video frames are image After being processed by formula (8), (x′, y′, model (k, d1, d2, ...)) should all meet the constraints of formula (1). However, due to the existence of the line of sight tracking model error, after the line of sight point trajectory coordinates obtained by formula (8) are substituted into formula (1), there must be a non-zero deviation.

[0044] Furthermore, when the cross ratio method without parameter calibration is used for error compensation, an error compensation item is added, such as a quadratic polynomial error Or a cubic polynomial error

[0045] Therefore, based on the principle of least squares method, the parameters in err are taken as the variables to be determined, and equation (9) is minimized:

[0046] min{∑(f image (x′, y′, model(k, d1, d2,...))·T p ·T t ·Tz+err-f(x ref , x ref )) 2}#(9)

[0047] Take the derivative of the coefficient items of the quadratic polynomial or cubic polynomial and set them to zero to obtain the error compensation coefficients. Etc. The error compensation coefficients obtained by least squares correspond to the polynomial error model compensation table.

[0048] Furthermore, since the least squares method is affected by singular values, it will cause a large deviation in the result. In the process of collecting human eye images, it is very easy for the human eye to inadvertently glance at the area around the standard point. In order to ensure the accuracy of the final result, the singular points need to be removed. Suppose a total of n fixation points are taken, and the deviation distance of each fixation point is Li ,but

[0049]

[0050] when When the ratio is greater than 1.5 or less than 0.75, the point is considered a singular point and the data of the point is removed and not included in the equation calculation.

[0051] Based on the principle of least squares method, the calculated coefficient item form is:

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058] Where n is the number of corresponding points.

[0059] The three equations about a, b, and c can be solved together to find the values ​​of a, b, and c. The three equations about d, e, and f can be solved together to find the values ​​of d, e, and f.

[0060]

[0061]

[0062] Furthermore, when the corneal reflection method or the single normalization method which requires parameter calibration is used, a similar least squares method is performed on the parameters to be calibrated (k, d1, d2, ...) in the model to obtain the minimization problem.

[0063]

[0064] make

[0065]

[0066] The coefficients obtained by the least squares method are the calibration parameter results.

[0067] Based on the principle of least squares method, the coefficient solution equation of the error polynomial is established as:

[0068]

[0069]

[0070]

[0071] …….

[0072] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the gaze tracking error compensation or parameter calibration method based on the gaze trajectory.

[0073] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the line of sight tracking error compensation or parameter calibration method based on visual trajectory.

[0074] Another object of the present invention is to provide an information data processing terminal, which is used to implement the line of sight tracking error compensation or parameter calibration system based on visual trajectory.

[0075] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0076] First, in view of the technical problems existing in the above-mentioned prior art and the difficulty of solving the problems, the technical solutions to be protected by the present invention and the results and data during the research and development process are closely combined to analyze in detail and deeply how the technical solutions of the present invention solve the technical problems, and some creative technical effects brought about after solving the problems. The specific description is as follows:

[0077] The present invention first proposes a gaze tracking error compensation or parameter calibration method based on gaze trajectory in order to solve the problems of large workload of gaze tracking error compensation or parameter calibration, high skill requirements for users, and poor operating experience in traditional gaze tracking systems. By tracking the moving point of a given trajectory, human eye images of different time series are selected to calculate the corresponding gaze landing point according to the corresponding gaze estimation method, and the shape information of the gaze gaze trajectory and the advantage of tracking the gaze trajectory method to quickly and conveniently obtain a large number of gaze landing points are used to solve the compensation table coefficients through the least squares method or perform model parameter calibration. By comparing the gaze landing point deviations processed by different methods, it is shown that the method disclosed in the present invention can simplify the process of error compensation or parameter calibration while ensuring the accuracy of gaze calculation.

[0078] Second, considering the technical solution as a whole or from the perspective of the product, the technical effects and advantages of the technical solution to be protected by the present invention are described in detail as follows:

[0079] The present invention is based on the gaze tracking system and performs simple and efficient error compensation or model parameter calibration on the gaze estimation module in the gaze tracking system by tracking the gaze trajectory, thereby reducing the difficulty of the actual operation process and the discomfort of the user's operation while ensuring the accuracy of the gaze landing point calculation.

[0080] Third, as auxiliary evidence of the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0081] The expected benefits and commercial value of the technical solution of the present invention after transformation are as follows: This patent proposes a new error compensation or parameter calibration method, which is simpler and more efficient than the traditional method and has good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 is a flow chart of a line of sight estimation method based on cross ratio invariance provided by an embodiment of the present invention;

[0083] Figure 2 is a schematic diagram of a line of sight calculation model provided by an embodiment of the present invention;

[0084] Figure 3 is a comparison diagram of the error without compensation provided by the embodiment of the present invention and the error after compensation by the method of the present invention;

[0085] Figure 4 The error comparison diagram of the error compensated by the nine-point method and the error compensated by the method of the present invention on two different calibration plates provided by the embodiment of the present invention is (a) the nine-point calibration plate and (b) the gaze track calibration plate;

[0086] Figure 5 It is a schematic diagram of the fitting trajectory and the original trajectory provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in combination with the embodiment cross-comparison method. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0088] In order to enable those skilled in the art to fully understand how to implement the present invention in detail, this section is an explanatory embodiment that expands and describes the technical solution of the claims.

[0089] like Figure 1 As shown, the gaze tracking error compensation method based on gaze trajectory provided by an embodiment of the present invention includes:

[0090] S101, by tracking the moving point of a given trajectory, selecting human eye images of different time series to construct a mapping model according to the cross ratio invariance, and calculating the corresponding sight point;

[0091] S102, performing sight point error compensation, using shape information of the gaze trajectory and the least square method to solve the compensation table coefficients to determine the final position of the sight point.

[0092] The line of sight mapping model is constructed through cross-ratio invariance, which applies the cross-ratio projective invariance principle in projective geometry, that is, the cross-ratio of corresponding points on a straight line in space remains unchanged after a geometric transformation.

[0093] Figure 2 The figure is a schematic diagram of the sight line calculation model constructed by cross-ratio invariance. It is mainly composed of three planes: the screen plane, the camera imaging plane, and the corneal reflection plane. In this model, the curved surface of the corneal reflection plane is idealized as a plane. The reflection point on the corneal surface and the light spot imaging point on the camera plane can be considered as the mapping of the light source point on the screen. Therefore, the cross-ratio invariance principle can be applied from two dimensions to three dimensions. The position of the sight line point S is calculated by fixing the cross-ratio value between each line segment of the plane figure composed of the relevant points between each plane. Among them, L1, L2, L3, and L4 are four light source points on the screen plane, G1, G2, G3, and G4 are the reflection spots formed by the light source on the human cornea, and g1, g2, g3, and g4 are the images formed by the reflection spots on the camera imaging plane. P0 is the center of the pupil, c is the center of the eyeball, and the line connecting the center of the pupil and the center of the eyeball is the optical axis of the eyeball. In this model, the direction of the optical axis is regarded as the direction of the human eye's sight line, and the intersection S of the optical axis and the screen is the sight line point obtained by the model.

[0094] Furthermore, the sources of error include the corneal curved surface and the angle between the optical axis and the visual axis; the errors of the corneal curved surface include:

[0095] When constructing the cross-ratio invariant line of sight mapping model, the reflection point and the pupil center are idealized to be in the same plane (pupil section) for the convenience of calculation. However, in reality, the human cornea is a curved surface structure, so the reflection point and the pupil center are not coplanar. When calculating the cross ratio, the length of the straight line between the two points on the ideal plane will be used instead of the length of the curve between the two real points. The length of the curve between the same two points is obviously greater than that of the straight line. This error in model construction will cause errors in the estimation of the line of sight direction.

[0096] The error of the angle between the optical axis and the visual axis is:

[0097] The optical axis is a straight line that passes through the center of the corneal surface and is perpendicular to the corneal surface. The visual axis is the line connecting the extraocular gaze point and the fovea. There is an angle kappa between the visual axis and the optical axis. The kappa angle is generally between 4 and 8 degrees. When constructing the model, the cross-ratio method directly outputs the optical axis as the visual axis in order to simplify the complexity of the model, resulting in a deviation between the final sight point position and the actual sight point position.

[0098] The errors in the above two points come from the systematic errors caused by simplifying the model for the convenience of calculation when constructing the model. The elimination of systematic errors is generally carried out through three methods: control test, blank test and correction test. Taking into account the characteristics of the gaze tracking system studied in this invention, the idea of ​​correction experiment is adopted to correct the errors.

[0099] Furthermore, the sight point error compensation includes:

[0100] (1) Using trajectory shape information to compensate for landing point error;

[0101] (2) Establish an error compensation table, use the least squares method to solve the compensation table coefficients, and perform error compensation;

[0102] Further, the use of trajectory shape information to perform landing point error compensation includes:

[0103] The human eye gazes at a uniform moving point on the calibration screen, and the trajectory of the moving point is a regular semicircle or circle. The trajectory of the sight point on the calibration screen is calculated frame by frame. According to the position of special points, such as the two end points of the diameter, the corresponding relationship between the original gaze trajectory and the fitted gaze trajectory is calculated. The angle between the fitted trajectory and the original trajectory is recorded as θ, the radius of the original trajectory is recorded as R, and the coordinates of the center of the circle are The radius of the fitting trajectory is denoted as r, and the coordinates of the center of the circle are (x' 0 , y' 0 ),like Figure 3 shown.

[0104] The original trajectory of the moving point on the screen is recorded as a function:

[0105]

[0106] The sight point trajectory calculated based on the camera raster image and a certain sight tracking model is a closed curve f containing the trajectory error. image (x′, y′). Obviously, f image (x′, y′) must correspond to the circle of the original trajectory, which can be written as:

[0107] f image (x′, y′, model(k, d1, d2,...))=(xx′0 ) 2 +(yy′ 0 ) 2 -r 2 =0#(2)

[0108] in, (x' 1 , y' 1 ) and (x' 2 , y' 2 ) are the two endpoints of the sight point trajectory calculated according to the sight tracking model, corresponding to the starting point and end point of the original semicircular arc trajectory of the screen moving point respectively; (x 0 ,y 0 ) and R are the center and radius of the original semicircular arc trajectory of the moving point on the screen. model(k, d1, d2, ...) represents the gaze tracking model, where k, d1, d2, etc. are system parameters in the gaze tracking model.

[0109] According to the position correspondence between the two endpoints of the semicircle diameter, the projection transformation matrix between the sight point trajectory and the original trajectory is obtained, including arc radius adjustment, rotation transformation, and translation transformation, where:

[0110] First, the arc radius is adjusted. The proportional adjustment matrix Tp adjusts the radius of the fitted arc to the radius of the original arc, that is, the shape of the fitted arc is proportional to Proportional enlargement of:

[0111]

[0112] Fit the line of sight landing point trajectory to any calculated position after scaling up:

[0113]

[0114] Then, the rotation transformation matrix T R Align the diameters of the semicircular arcs:

[0115] Angle

[0116] If y′ 1 -y′ 2 If θ>0, θ is considered to be a positive angle, otherwise, θ is considered to be a negative angle.

[0117]

[0118] Calculate the position of any landing point after rotating the fitted sight landing point trajectory:

[0119]

[0120] Translation transformation matrix TT Align the centers of the two arcs:

[0121]

[0122] Calculate the position of any landing point after the fitted sightline landing point trajectory coincides with the original semicircle trajectory:

[0123]

[0124] Further, the error compensation table is established, and the coefficients of the compensation table are solved by the least square method to perform error compensation, including:

[0125] Line of sight trajectoryf image The positions of all points on (x′, y′, model (k, d1, d2, ...)) must satisfy the circular trajectory equation described in formula (1). Therefore, the calculated positions of each moving point in all video frames are image After being processed by formula (8), (x′, y′, model (k, d1, d2, ...)) should all meet the constraints of formula (1). However, due to the existence of the line of sight tracking model error, after the line of sight point trajectory coordinates obtained by formula (8) are substituted into formula (1), there must be a non-zero deviation.

[0126] Furthermore, the error compensation item quadratic polynomial error is added

[0127] Therefore, based on the principle of least squares method, the parameters in model (k, d1, d2, ...) are taken as variables to be determined, and equation (9) is minimized:

[0128] min{∑(f image (x′,y′,model(k,d1,d2,...))+err-f(x ref , x ref )) 2}#(9)

[0129] Derivative the coefficient items of the quadratic polynomial respectively and set them to zero to obtain the error compensation coefficients, that is, And so on, the error compensation coefficients obtained by least squares correspond to the polynomial error model compensation table.

[0130] Furthermore, since the least squares method is affected by singular values, it will cause a large deviation in the result. In the process of collecting human eye images, it is very easy for the human eye to inadvertently glance at the area around the standard point. In order to ensure the accuracy of the final result, the singular points need to be removed. Assume that a total of n fixation points are taken, and the deviation distance of each fixation point is L i ,but

[0131]

[0132] when When the ratio is greater than 1.5 or less than 0.75, the point is considered a singular point and the data of the point is removed and not included in the equation calculation.

[0133] Based on the principle of least squares method, the calculated coefficient item form is:

[0134]

[0135]

[0136]

[0137]

[0138]

[0139]

[0140] Where n is the number of corresponding points.

[0141] The three equations about a, b, and c can be solved to find the values ​​of a, b, and c. The three equations about d, e, and f can be solved to find the values ​​of d, e, and f.

[0142]

[0143]

[0144] The embodiment of the present invention further provides a gaze tracking error compensation or parameter calibration system based on gaze trajectory, including:

[0145] A model building module is used to track the moving points of the gaze trajectory and select human eye images of different time series to build a mapping model according to the cross-ratio invariance;

[0146] The sight error compensation or parameter calibration module uses the shape information of the sight trajectory and the least squares method to solve the compensation table coefficients or calibrate the model parameters of the system.

[0147] In order to prove the creativity and technical value of the technical solution of the present invention, this section provides application examples of the claimed technical solution on specific products or related technologies.

[0148] The gaze tracking error compensation or parameter calibration method based on gaze trajectory provided by the application embodiment of the present invention is applied to a computer device, and the computer device includes a memory and a processor, and the memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the gaze tracking error compensation or parameter calibration method based on gaze trajectory.

[0149] The gaze tracking error compensation or parameter calibration method based on the gaze trajectory provided in the application embodiment of the present invention is applied to an information data processing terminal, and the information data processing terminal is used to implement the gaze tracking error compensation or parameter calibration system based on the gaze trajectory.

[0150] The embodiments of the present invention have achieved some positive effects during the development or use process and have great advantages over the prior art. The following content is described in conjunction with data, charts, etc. of the test process.

[0151] The experiment and analysis provided by the embodiment of the present invention include: an experiment preparation process and an experiment result and analysis; the experiment (selecting the cross ratio method to illustrate the parameter compensation method) preparation process includes:

[0152] Depend on Figure 2 As can be seen from the schematic diagram of the line of sight calculation model, the hardware includes a camera, 4 infrared LED lights, a display screen, and a short video or animated image of the movement speed and trajectory of a given ball.

[0153] The camera is aimed at the human eye to be tested, the infrared LED lamp is placed at the corresponding position of the display screen according to the model, the corresponding position of the human head and the display screen is adjusted, a video of the movement of a small ball is played, the human eye looks at the small ball in the video, and the human eye camera starts to record eye movement behavior (including time, pupil position, and reflected light spot position). After the calibration is completed, the solution is obtained through the line of sight gaze model of the present invention.

[0154] Further, the experimental results and analysis include:

[0155] The present invention mainly analyzes the systematic error in line of sight estimation. The cross-ratio invariance mapping model selected in this experiment mainly has two types of systematic errors. Through the cross-experiment method, it is compared with the traditional multi-point compensation method.

[0156] 1) Comparison of errors without compensation and with compensation by the method of the present invention: As shown in the figure, the error between the sight point obtained directly by the sight estimation model and the real gaze point is compared with the error between the sight point obtained after compensation by the method of the present invention and the real gaze point.

[0157] 2) Error comparison between the method of the present invention and the traditional nine-point method: As shown in the figure, the error between the sight point compensated by the traditional multi-point method and the real gaze point is compared with the error between the sight point compensated by the method of the present invention and the real gaze point.

[0158] The sight point error calculated after compensation by the method used in the present invention is significantly improved compared with the sight point error calculated after compensation by the nine-point method, indicating that the method can meet the requirements of sight line estimation in terms of accuracy.

[0159] This paper takes the parameter compensation method in the gaze tracking system as the research object. In view of the problems that the system has in the conventional multi-point calibration method for compensation, such as complex process, high skill requirements, and poor user experience, a gaze tracking error compensation or parameter calibration method based on gaze trajectory is proposed. By comparing the accuracy of the gaze point under different situations in the cross-ratio method example, it is shown that this method can reduce the complexity of compensation and the user skill requirements while ensuring accuracy, thereby improving the user experience.

[0160] It should be noted that the embodiments of the present invention can be implemented by hardware, software or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated hardware. A person of ordinary skill in the art will understand that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, for example, such codes are provided on a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware. The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A gaze tracking error compensation or parameter calibration method based on gaze trajectory, characterized in that: The gaze tracking error compensation or parameter calibration method based on the gaze trajectory includes: By tracking the moving point of a given trajectory, selecting human eye images of different time series and building a mapping model according to the selected sight line estimation method, the corresponding sight line landing point is calculated; using the shape information of the sight line trajectory and the least squares method, the system is used to solve the compensation table coefficients or calibrate the model parameters; The error compensation method includes using trajectory shape information and using the least square method to solve the correlation coefficient. The specific process is: By gazing at a uniformly moving point on the screen with human eyes, the trajectory of the moving point is a curve with a certain function form, including a regular straight line segment, a semicircle, or a full circle; according to the time information of the human eye video and the movement of the moving point on the screen, a large number of human eye image frames corresponding to any point on the screen can be quickly found, the sight point is calculated according to the sight mapping model, and the point is initially adjusted according to the information that the gaze trajectory is a regular shape; a compensation table of the sight point and the real gaze point is established, or a mapping model of the eye movement parameters and the real gaze point is established, and the least squares method is used to solve the large number of real gaze points and the corresponding sight point information or human eye parameter information after initial adjustment to obtain an error compensation coefficient or calibration parameter; The specific process of using trajectory shape information to perform sight point compensation is as follows: According to the positions of special points, including the two endpoints of the diameter, the correspondence between the original gaze trajectory and the fitted gaze trajectory is calculated: The angle between the fitted gaze trajectory and the original gaze trajectory is denoted as θ, the radius of the original gaze trajectory is denoted as R, and the center coordinates are Coordinates of the two end points of the original gaze trajectory and The radius of the fitted gaze trajectory is denoted as r, the coordinates of the center of the circle are (x'0, y'0), and the coordinates of the two end points of the fitted gaze trajectory are (x'1, y'1) and (x'2, y'2). in, According to the position correspondence between the two endpoints of the semicircle diameter, the projection transformation matrix between the sight point trajectory and the original gaze trajectory is obtained, including arc radius adjustment, rotation transformation, and translation transformation, where: First, adjust the arc radius and the proportional adjustment matrix T p Adjust the radius of the fitted arc to the radius of the original arc, that is, adjust the shape of the fitted arc to the proportional coefficient of Proportional enlargement of: Fit the line of sight landing point trajectory to any calculated position after scaling up: Then, the rotation transformation matrix T R Align the diameters of the semicircular arcs: Angle If y′1′-y′2′>0, θ is considered to be a positive angle, otherwise, θ is considered to be a negative angle; Calculate the position of any landing point after rotating the fitted sight landing point trajectory: Translation transformation matrix T T Align the centers of the two arcs: Calculate the position of any landing point after the fitted sightline landing point trajectory coincides with the original semicircle trajectory:

2. The gaze tracking error compensation or parameter calibration method based on gaze trajectory according to claim 1, characterized in that: It can be applied to various line of sight estimation models including the cross-ratio method; A corresponding gaze estimation model is established according to a corresponding gaze estimation method, and after gazing at a given trajectory, a gaze tracking parameter compensation method based on the gaze trajectory is used to perform parameter compensation; The original data required by the parameter compensation method include: human eye pupil image, track points on the screen, and corresponding time information.

3. The gaze tracking error compensation or parameter calibration method based on gaze trajectory according to claim 1, characterized in that: The specific process of using the least square method to solve and obtain the error compensation coefficient or calibration parameter is as follows: Since the least square method is affected by singular values, it will cause a large deviation in the result. It is very easy for the human eye to accidentally glance at the area around the standard point during the acquisition of human eye images. In order to ensure the accuracy of the final result, the singular points need to be removed. Assume that there are n fixation points in total, and the deviation distance of each fixation point is L i ,but when When the ratio is greater than 1.5 or less than 0.75, the point is considered a singular point and the data of the point is removed and not included in the equation calculation; When using the cross ratio method without parameter calibration and direct error compensation, add error compensation items, including quadratic polynomial error Or a cubic polynomial error Based on the least squares principle, er r The parameters in are taken as the variables to be determined, and the following formula is minimized: min{Σ(f(x′,y′)·T p ·T t ·Tz+err-f(x ref ,y ref )) 2 } Take the derivative of the coefficient items of the quadratic polynomial or cubic polynomial respectively and set them to zero to obtain the error compensation coefficients, that is, Each error compensation coefficient obtained by least squares corresponds to the polynomial error model compensation table; Compensation table for fitted gaze trajectory When using the corneal reflection method or single normalization method that requires parameter calibration, perform a similar least squares method on the parameters to be calibrated in the model (k, d1, d2, ...) to obtain the minimization problem min{Σ(f(x′,y′,map(k,d1,d2,…))-f(x ref ,y ref )) 2 } make The coefficients obtained by the least squares method are the calibration parameter results.

4. A gaze tracking error compensation or parameter calibration system based on gaze trajectory implementing the gaze tracking error compensation or parameter calibration method based on gaze trajectory according to any one of claims 1 to 3, characterized in that: The gaze tracking error compensation or parameter calibration system based on gaze trajectory includes: A model building module is used to track the moving points of a given trajectory, select human eye images of different time series and build a mapping model according to the corresponding line of sight tracking method; The sight error compensation or parameter calibration module uses the shape information of the sight trajectory and the least squares method to solve the compensation table coefficients or calibrate the model parameters of the system.

5. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the gaze tracking error compensation or parameter calibration method based on the gaze trajectory as described in any one of claims 1-3.

6. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the gaze tracking error compensation or parameter calibration method based on the gaze trajectory as described in any one of claims 1 to 3.

Images