A method of improving eye movement gaze compensation

Abstract

The embodiment of the application provides an eye movement line compensation method, in the rough screening, the pupil area of the sample eye diagram is intercepted for light spot screening, the threshold value is dynamically calculated by using the average gray mean value of the pupil area of the sample eye diagram, so that the effective light spot imaging area is obtained. By taking the pupil center as the screening area, the second light spot is roughly screened by taking the light spot area and the circularity as the parameters, the deformed light spot and the actual light spot are accurately screened by filtering the skin light spot. Finally, by normalizing the changing pupil area as a feature, the automatic suppression of the weight of the redundant feature is realized by fusing the L2 regularization characteristics of the ridge regression, and the traditional scheme of fixed order polynomial fitting is abandoned, so that the compensation scheme suitable for different crowds is adapted, and the anti-interference ability and the generalization performance of the model are significantly improved. In addition, by using the partition calibration compensation mode for different marking points, the compensation precision can be effectively improved.

Description

Technical Field

[0001] The embodiments of this application belong to the field of visual line compensation technology, and particularly relate to a method for improving eye movement visual line compensation. Background Technology

[0002] Gait estimation methods based on pupil-corneal reflexes typically calibrate eye parameters only under a single background brightness, and perform well in scenes with constant brightness. However, in actual use, changes in the brightness of the scene being viewed cause pupil dilation and contraction, and the appearance of more abnormal light spots, leading to a decrease in the accuracy of gaze estimation. Summary of the Invention

[0003] To address or mitigate the technical problems in the prior art, embodiments of this application provide a method for improving eye-tracking gaze compensation, including:

[0004] Collect sample eye images and determine the pupil center and pupil region on the sample eye images;

[0005] Identify suspected light spots in the pupil area;

[0006] The suspected light spots are screened to obtain the actual light spots;

[0007] Based on the actual light spot and the pupil center, eye parameters are calibrated and line of sight estimation points are calculated.

[0008] Calculate the compensation coefficient of the line-of-sight estimation point based on the aforementioned line-of-sight estimation point;

[0009] Acquire an actual eye diagram, calculate the compensation amount for each line of sight estimation point on the actual eye diagram based on the compensation coefficient of the line of sight estimation point, and compensate the line of sight estimation point based on the compensation amount.

[0010] As a preferred embodiment of this application, the step of determining a suspected light spot in the pupil region includes:

[0011] Calculate the average gray value of all pixels in the pupil region, and subtract the average gray value from the gray value of each pixel in the pupil region to obtain a temporary pupil region;

[0012] The temporary pupil region is binarized based on the grayscale mean to obtain a binary image, which is used to find suspected light spots.

[0013] As a preferred embodiment of this application, the step of binarizing the temporary pupil region based on the grayscale mean includes:

[0014] The binary image is then subjected to noise reduction processing.

[0015] As a preferred embodiment of this application, the step of screening the suspected light spots to obtain the actual light spots includes:

[0016] Obtain a light spot whose two-dimensional area is within a preset pixel range from the suspected light spot to obtain a first light spot;

[0017] Obtain a light spot whose outline roundness is within a preset roundness range, and obtain a second light spot;

[0018] The center coordinates of the second light spot are obtained within the aspect ratio range of the pupil region to obtain the third light spot;

[0019] Remove the extra third spot to obtain the actual spot.

[0020] As a preferred embodiment of this application, the step of removing the excess third light spot to obtain the actual light spot includes:

[0021] When the total number of the third light spots is greater than a preset value, calculate the difference between the maximum and minimum values ​​of the center point coordinates of all the third light spots in the horizontal and vertical directions;

[0022] Determine whether the difference between the maximum and minimum values ​​in the horizontal and vertical directions is greater than the horizontal and vertical thresholds, respectively.

[0023] If so, remove the third spot whose difference between the maximum and minimum values ​​in the horizontal and vertical coordinate directions is greater than the horizontal and vertical coordinate thresholds, respectively, and calculate the mean horizontal and vertical coordinates of the remaining center coordinates of the third spot.

[0024] Remove the third light spot with the largest difference from the corresponding mean values ​​of the horizontal and vertical coordinates in the horizontal and vertical directions, until the total number of the third light spots is not greater than the preset value, to obtain the actual light spot.

[0025] As a preferred embodiment of this application, the step of calculating the compensation coefficient of the line-of-sight estimation point based on the line-of-sight estimation point includes:

[0026] Obtain the true coordinates of different fixation points on the sample eye diagram;

[0027] Calculate the estimated coordinates of different fixation points based on the eye parameters of each of the sample eye diagrams, and calculate the difference between the estimated coordinates of different fixation points and the true coordinates in terms of horizontal and vertical coordinates;

[0028] Based on the difference between the horizontal and vertical coordinates, the estimated coordinates of the abnormal gaze points are removed to obtain the effective estimated coordinates of the gaze points;

[0029] The sample eye diagrams with pupil areas within a preset pixel range are normalized to obtain normalized pupil areas.

[0030] A polynomial model is constructed and optimized. The compensation coefficient of the gaze estimation point is calculated using the polynomial model based on the normalized pupil area.

[0031] As a preferred embodiment of this application, the construction and optimization of the polynomial model includes:

[0032] We construct a higher-order polynomial feature and then optimize the polynomial feature using ridge regression to obtain the optimized polynomial model.

[0033] As a preferred embodiment of this application, the step of acquiring the actual eye map, calculating the compensation amount for each gaze estimation point on the actual eye map based on the compensation coefficient of the gaze estimation point, and compensating the gaze estimation point according to the compensation amount includes:

[0034] In the actual eye diagram, identify the n calibration points closest to the line-of-sight estimation point, where n is less than the number of calibration points N for the compensation coefficient;

[0035] Based on the two-dimensional pupil area of ​​the actual eye diagram, pupil normalization is performed on the pupils of the n calibration points respectively;

[0036] Calculate the Euclidean distance between the line-of-sight estimation point and the calibration point respectively, and calculate the compensation weights of n calibration points based on the Euclidean distance;

[0037] The compensation values ​​for n calibration points are calculated based on the compensation coefficient and the compensation weight, and the line-of-sight estimation points are compensated based on the compensation amount.

[0038] Compared with existing technologies, this application provides a method to improve eye-tracking gaze compensation. In the initial screening, the pupil region of the sample eye map is extracted for spot selection. A threshold is dynamically calculated based on the average brightness change of the pupil region to adapt to brightness variations in eye-tracking scenarios, thus obtaining an effective spot imaging area. By using normalized pupil area as a feature and incorporating the L2 regularization property of ridge regression, redundant feature weights are automatically suppressed, abandoning the traditional fixed-order polynomial fitting scheme. This allows for compensation schemes tailored to different populations, significantly improving the model's anti-interference ability and generalization performance. Furthermore, this application employs a partitioned calibration compensation method, which effectively improves compensation accuracy. Attached Figure Description

[0039] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. Some specific embodiments of this application will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings designate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0040] Figure 1 This is a flowchart of a method for improving eye movement accuracy compensation provided in an embodiment of this application. Detailed Implementation

[0041] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort should fall within the scope of protection of the present application.

[0042] like Figure 1 As shown in the figure, this application provides a method for improving eye-tracking accuracy compensation, including:

[0043] Step S101: Acquire a sample eye diagram and determine the pupil center and pupil region on the sample eye diagram;

[0044] Specifically, sample eye images are collected, and a deep learning model is used to detect the pupil center and pupil outline. The aspect ratio h / w of the pupil outline is calculated, and the pupil region image is cropped with the pupil center as the center. The length of the pupil region is 1.45*h and the width is 1.25*w, where h is the height of the pupil outline and w is the width of the pupil region.

[0045] Deep learning models can use a variety of models, including but not limited to the following:

[0046] U-Net model: A convolutional neural network that is used to segment the eye region and extract the pupil region.

[0047] Mask R-CNN model: also used for eye region segmentation, extracting the pupil region.

[0048] EyeCool model: This is an improved U-Net-based model that can segment the pupil and iris, as well as the internal and external boundaries, simultaneously.

[0049] Step S102: Identify a suspected light spot in the pupil area;

[0050] Before determining the suspected light spot in step S102, the method further includes:

[0051] Calculate the average gray value of all pixels in the pupil region, and subtract the average gray value from the gray value of each pixel in the pupil region to obtain a temporary pupil region;

[0052] The temporary pupil region is binarized based on the grayscale mean to obtain a binary image, which is used to find suspected light spots.

[0053] Specifically, the mean gray value m of the pupil region obtained in step S101 is calculated. The mean gray value m is subtracted from the gray value of each pixel in the pupil region to obtain a temporary pupil region. Then, the threshold function in OpenCV is used to segment the temporary pupil region according to the threshold (threshold = m * 1.8) to obtain a binary image.

[0054] The difference between the pupil region and the temporary pupil region is the difference in brightness. By using the temporary pupil region, global illumination unevenness / brightness shift can be eliminated, local contrast can be enhanced, and the target region can be highlighted. If the pupil region is directly binarized, the threshold used in the threshold function is difficult to set due to poor consistency. Therefore, this application uses a multiple of the grayscale mean to determine the threshold.

[0055] In this embodiment, since the threshold is determined by the grayscale mean, which is determined based on the grayscale value of each pixel in the image, if the brightness of the image changes, the grayscale value of each pixel in the image will change, and the grayscale mean of images with different brightness will be different. This embodiment adopts a dynamic threshold calculation method, which is more suitable for situations where the brightness of the eye diagram changes due to brightness changes, while ensuring the real-time performance of spot detection.

[0056] OpenCV is a powerful computer vision library that provides rich image processing capabilities. In OpenCV, grayscale images are single-channel, and OpenCV offers many image processing functions. The `threshold` function is used for image thresholding, converting a grayscale image into a binary image by comparing each pixel value with a threshold and then setting the pixel value to 0 or another specified value based on the comparison result. This is a simple and effective image segmentation method, often used to separate foreground objects from the background.

[0057] A grayscale image is an image that contains only brightness information and no color information. Each pixel is represented by a numerical value, typically in the range of 0-255. 0 represents black, 255 represents white, and values ​​between 0 and 255 represent different shades of gray.

[0058] In digital images, grayscale value specifically refers to the brightness value of a pixel in a grayscale image. It is the concrete manifestation of a pixel value within the specific type of grayscale image. Grayscale value is a numerical value that represents the brightness level of a particular pixel in an image, ranging from pure black to pure white.

[0059] To obtain a grayscale image, color information is usually discarded (or brightness information is captured from the beginning), and only the brightness and darkness of each pixel are retained.

[0060] The grayscale mean of an image of the pupil region is calculated by summing and averaging the grayscale values ​​of different pixels in the entire image.

[0061] After binarizing the temporary pupil region based on the grayscale mean, the process includes: performing noise reduction processing on the binary image.

[0062] Specifically, image dilation and erosion morphological processing is performed on the binary image to remove noise.

[0063] Image dilation and erosion are fundamental operations in morphological processing. They are typically performed on binary images to enhance structural features, expand white areas (foreground) in an image, connect adjacent objects, fill small holes, and increase object size.

[0064] Dilation expands white areas (foreground), fills small holes, and connects adjacent objects, which can eliminate small noise points (especially bright noise) in an image because noise points are usually small and isolated, and erosion makes these small bright areas disappear. However, erosion can also make the target object smaller.

[0065] Erosion shrinks white areas (foreground), eliminates small objects, separates adjacent objects, and can fill small holes (dark noise) inside target objects, but it also amplifies noise points (if the noise is bright, dilation will make bright noise bigger). Therefore, simply using dilation to remove noise is usually not ideal.

[0066] Step S103: The suspected light spots are screened to obtain the actual light spots;

[0067] Step S103 specifically includes:

[0068] Step 1: Obtain the first light spot by finding a light spot with a two-dimensional area within a preset pixel range;

[0069] Specifically, the findContours function in OpenCV is used to search for possible light spot contours in the pupil region after the image has undergone dilation and erosion morphological processing.

[0070] If the two-dimensional area corresponding to the suspected light spot outline is less than 8 pixels but greater than 50 pixels, the suspected light spot is removed. This step can remove tear spots and astigmatic spots.

[0071] Step 2: Obtain the second light spot by acquiring the light spot whose outline roundness is within a preset roundness range.

[0072] Specifically, in image processing, roundness (p) measures how closely a shape approximates a circle. A common formula for calculating roundness (p) is: p = 4π * A / C 2 Let A be the area and C be the perimeter. The calculation is performed to obtain the first light spot in the first step. The roundness of the first light spot is calculated. If the roundness of the first light spot is less than 0.65, the first light spot is determined to be a deformed light spot and is removed to obtain the second light spot.

[0073] Step 3: Obtain the center coordinates of the second spot within the aspect ratio range of the pupil region image to obtain the third spot;

[0074] Specifically, if the center coordinates of the second spot are not within the aspect ratio (0.5, 0.95) of the pupil area image, the second spot is identified as a tear film flash or a spot on the skin. The second spot is then removed to obtain the third spot.

[0075] Step 4: Remove the extra third light spot to obtain the actual light spot.

[0076] The fourth step specifically includes: Since the light spots on the sample eye diagram are all obtained by infrared light source illumination, and the number of light spots is the same as the number of infrared light sources, when the total number of third light spots is greater than the total number of infrared light sources, the center coordinates of all third light spots are calculated according to the difference between the maximum and minimum values ​​of the horizontal coordinate (x_max-x_min) and the difference between the maximum and minimum values ​​of the vertical coordinate (y_max-y_min) of each third light spot in the horizontal coordinate direction (X direction). The difference between the maximum and minimum values ​​of the horizontal coordinate (x_max-x_min) and the difference between the maximum and minimum values ​​of the vertical coordinate (y_min) of each third light spot in the vertical coordinate direction (y direction) is then determined. If the value of x_max - x_min is greater than the corresponding threshold (150 pixels in the x-direction and 100 pixels in the y-direction), the third spot with the maximum difference between the maximum and minimum values ​​of the x-coordinate (x_max - x_min) and the third spot with the maximum difference between the maximum and minimum values ​​of the y-coordinate (y_max - y_min) are removed. The mean of the x-coordinate and y-coordinate of all remaining third spots is calculated. Finally, the third spot with the largest distance from the mean in both the x-coordinate and y-coordinate directions is removed until the total number of remaining third spots is less than or equal to the total number of infrared light sources, thus obtaining the actual spot.

[0077] Then, the centroid coordinates of the actual light spot obtained in the fourth step are calculated; the centroid coordinates of the actual light spot are compared with the coordinates of the standard light spot in a polling manner to sort the light spots.

[0078] Specifically, in image processing, the centroid usually refers to the center point of an object or region, which is the average value of all points. For contours in a binary image, the centroid can be obtained by calculating the average value of all points on the contour.

[0079] In this embodiment, the standard light spot coordinates can be obtained by looking straight ahead. When sorting the light spots, the centroids of all standard light spots are first connected, and then sorted clockwise according to the vector magnitude between any two standard light spots to obtain their serial numbers. Simultaneously, the centroids of the actual light spots are connected, and the vector magnitudes between any two actual light spots are sorted. Finally, the actual light spots are sorted clockwise according to their serial numbers and the vector magnitudes between all actual light spots to obtain their serial numbers.

[0080] Step S104: Based on the actual light spot and the pupil center, perform eye parameter calibration and line of sight estimation point calculation;

[0081] In this embodiment, calibration is performed in three brightness ranges (low, medium, and high). Taking a background grayscale value of 0-255 as an example, 0-50 grayscale value is the low brightness range, 50-200 is the medium brightness range, and 200-255 is the high brightness range. Eye parameter calibration is performed under the medium brightness background range. The actual light spot and the pupil center obtained in step S101 are used to calibrate the eye parameters, determining the parameters of the left and right eyes: Kappa angle, corneal radius, distance between the pupil center and the corneal center, etc.

[0082] Eye parameter calibration typically refers to determining the mapping relationship between an eye model (such as the cornea, pupil, etc.) and a camera image in eye tracking or gaze estimation tasks. The eye parameter calibration process generally includes the following steps:

[0083] Camera calibration: Determine the camera's intrinsic parameters (focal length, principal point, distortion coefficient, etc.) and extrinsic parameters (the camera's position and orientation in the world coordinate system).

[0084] Eye model creation: Select or create an eye model, for example, simplify the eyeball as a sphere and consider structures such as the cornea and pupil.

[0085] Data acquisition: The subject is instructed to look at a known calibration point (usually displayed on a screen) while an image of their eye is captured by a camera.

[0086] Parameter estimation: Based on the acquired images and known calibration point locations, estimate the parameters of the eye model (such as eye radius, corneal curvature radius, pupil center, etc.) and the position of the eye in the camera coordinate system.

[0087] Geometric model-based methods are used to calculate the gaze estimation point, with the pupil center-corneal reflex (PCCR) method serving as an example. This typically includes the following steps:

[0088] Detect the eye area;

[0089] Detect corneal reflection points and pupil center in eye images;

[0090] Calculate the vector of the pupil center relative to the corneal reflection point;

[0091] A mapping from vectors to screen gaze points is established through a calibration process;

[0092] Use the calibrated model to estimate the line-of-sight estimation point.

[0093] Step S104: Based on the actual light spot and the pupil center, perform eye parameter calibration and line of sight estimation point calculation;

[0094] Step S104 specifically includes:

[0095] Step 1: Obtain the third eye map at different brightness fixation points;

[0096] Specifically, when the background grayscale is 0, the test subject's eyes are focused on point j, and the pupils of the human eyes are dilated as much as possible. The test subject's eye images are collected evenly within 3-6 seconds, with a frame rate of no less than 60 frames.

[0097] Images of the test subject's eyes are uniformly acquired while the background grayscale increases from 50 to 235, with a frame rate of no less than 90 frames. Alternatively, the brightness is divided into three groups: low, medium, and high. Data on the human eye's gaze at the aforementioned points is acquired in each brightness group, with no less than 30 frames acquired in each brightness group.

[0098] Step 2: Calculate the difference between the estimated coordinates and the true coordinates of the gaze point;

[0099] Specifically, current gaze calibration The pupil area corresponding to all captured frame images is The fixation obtained based on the above steps The estimated coordinates are ; gaze The three-dimensional true coordinates are The difference between the estimated coordinates and the true coordinates in the XY plane is: , For frame order, the line-of-sight compensation coefficient and corresponding compensation calculation are performed on the XY plane.

[0100] Step 3: Normalize the sample eye diagrams whose pupil area is within a preset pixel range to obtain the normalized pupil area.

[0101] When the fixation point j is the largest, the two-dimensional pupil area is at its maximum. Minimum value The pupil area of ​​the third eye After normalization, the pupil area of ​​the third eye diagram is obtained. for:

[0102] ;

[0103] It should be noted that the pupil area of ​​the third eye diagram is... When the pupil area Frames with fewer than 1000 pixels are discarded. (When the pupil area...) When the number of pixels exceeds 5000, the data in that frame is discarded, and then the pupil area in the corresponding remaining frame image is normalized to obtain the normalized pupil area.

[0104] Step 4: Remove the estimated coordinates of the abnormal fixation points based on the difference between the horizontal and vertical coordinates to obtain the effective estimated coordinates of the fixation points;

[0105] Based on the difference between the estimated gaze point and the actual gaze point, when the angular error between the estimated gaze direction and the actual gaze direction is greater than 5°, the gaze estimate of that data point is considered abnormal and should be removed. Then, outliers are removed using Z-score normalization to obtain the difference between the effective estimated gaze coordinates and the actual coordinates of the gaze point. ,remember .

[0106] Z-score normalization (standardization) transforms data into a distribution with a mean of 0 and a standard deviation of 1.

[0107] Step S105: Calculate the compensation coefficient of the line of sight estimation point based on the line of sight estimation point;

[0108] Specifically, a polynomial model is constructed and optimized, and the compensation coefficient for each gaze estimation point is calculated using the polynomial model based on the normalized pupil area.

[0109] The specific process includes the following steps:

[0110] Step 1: Construct polynomial features: Construct polynomial features using a relatively high order (e.g., 4th order), and then use ridge regression to automatically suppress unimportant features (i.e., higher-order terms) to obtain a practically effective polynomial model.

[0111] For example, if the highest order is 4, then the eigenvector is: Normalized pupil area was used as a feature.

[0112] Step 2: Constructing a higher-order polynomial: Using normalized pupil area Difference between estimated coordinates and actual coordinates of the line of sight Perform polynomial fitting;

[0113] System of polynomial equations:

[0114]

[0115]

[0116] The above system of equations can be simplified as follows:

[0117]

[0118]

[0119] To compensate for the difference in the actual x-axis, To compensate for the difference in the actual ordinate, A is... B is H is ;

[0120] Step 3: Ridge Regression Solution: Add a regularization term to the loss function of the x and y coordinates of the line-of-sight estimation point, i.e. minimize it.

[0121]

[0122]

[0123] in, and These are the L2 regularization parameters. and The larger the value, the stronger the penalty. Based on the loss function, the closed-form solution for ridge regression is:

[0124]

[0125]

[0126] in, yes The identity matrix, where T is the matrix transpose symbol.

[0127] Step 4: Determine the penalty coefficient: The core objective of determining the penalty coefficient is to find the optimal coefficient that can suppress the overfitting risk of the fourth-order model while preserving its fitting ability when the preset equation is a fourth-order polynomial. and The values ​​are selected from the six numbers [0.001, 0.01, 0.1, 1, 10, 100] and iterated through them. and After solving for the fitting coefficients in step three, the mean fitting error (MAE) is calculated using regression. The mean fitting error is the line-of-sight compensation value. Compensated line of sight estimation After fixation point true value By comparison, the one with the smallest average fitting error is selected. and value.

[0128]

[0129] Where ACC is the function for calculating line-of-sight error, and n is the total number of frames involved in the calculation.

[0130] When determined and After determining the penalty coefficient, you can proceed to step three to solve the problem. .

[0131] Step 5: Eliminate redundant high-order terms: Find the optimal one and Then, check the compensation coefficients obtained from the solution. If the absolute value of the compensation coefficient of a certain 4th-order term is less than 1e-4, the term can be directly removed to further simplify the model.

[0132] Step 6: Calibration point selection: Perform regression based on the compensation coefficient and calculate the regression results. Statistical error difference :

[0133] Pre-regression line of sight estimation error for:

[0134] ,

[0135] It is the total number of frames involved in the calculation. The line-of-sight error calculation function is the angle between the estimated line-of-sight point and the actual point in the same coordinate system.

[0136] Line of sight error after regression compensation

[0137] ;

[0138] The statistical difference in line-of-sight error before and after compensation, e, is:

[0139]

[0140] when If the calibration parameters for the line-of-sight estimation point are unavailable, discard the line-of-sight estimation point; otherwise, record the set of calibration parameters.

[0141] Repeat the above steps to form a set of compensation coefficients for each line of sight estimation point until all line of sight estimation calibrations are completed.

[0142] Step S106: Acquire the actual eye diagram, calculate the compensation amount of the corresponding gaze estimation point on the actual eye diagram based on the compensation coefficient of the gaze estimation point, and compensate the gaze estimation point according to the compensation amount.

[0143] Step S106 specifically includes:

[0144] Step 1: Identify the n calibration points closest to the line-of-sight estimation point in the actual eye diagram, where n is less than the number of calibration points N for the compensation coefficient;

[0145] Specifically, the actual eye diagram has 5 compensation coefficient calibration points, which are then input as gaze estimation points. Find the three calibration points closest to the estimated line-of-sight point. ,

[0146] Step 2: Normalize the pupil area of ​​each calibration point;

[0147] Specifically, the actual two-dimensional pupil area in the eye diagram is The pupil area is normalized based on the pupil area of ​​the three calibration points closest to the estimated gaze point. This normalization process is then applied to the actual two-dimensional pupil area of ​​the eye diagram. If the pupil area is outside the range of the corresponding calibration point, this calibration point will not be used for compensation weighting, and the corresponding point will not participate in the weight calculation.

[0148] Step 3: Calculate the Euclidean distance between the line-of-sight estimation point and each calibration point, and calculate the compensation weight of the calibration point based on the Euclidean distance;

[0149] Specifically, the Euclidean distance between the estimated line-of-sight point and each calibration point is calculated. The compensation weights for the three calibration points are as follows:

[0150]

[0151]

[0152]

[0153] Step 4: Calculate the compensation amount for each calibration point according to the compensation coefficient, and compensate the line-of-sight estimation point according to the compensation amount.

[0154] Specifically, based on the compensation coefficient and Calculate the compensation value for each calibration point. Compensation amount at three calibration points for:

[0155]

[0156] During the compensation process, the depth dimension (Z value) of the gaze point remains unchanged; correspondingly, the compensated coordinates of the gaze estimation points on the horizontal (x-direction) and vertical (y-direction) axes change. for:

[0157]

[0158] In VR and AR headsets, when the background brightness at the subject's gaze point changes, especially in excessively dark or bright environments, the compensation method provided in this application improves gaze point accuracy. Specifically, during the coarse screening, the pupil region of the sample eye map is extracted for spot selection. The average gray value of the pupil region of the sample eye map is used to dynamically calculate the threshold to adapt to brightness changes in eye-movement scenarios, obtaining an effective spot imaging area. Furthermore, by using the pupil center as the screening area and using spot area and roundness as parameters, a second spot is coarsely selected, filtering out deformed spots and spots on the skin to accurately select the actual spot. Finally, by normalizing the changing pupil area as a feature and incorporating the L2 regularization property of ridge regression, redundant feature weights are automatically suppressed, abandoning the traditional fixed-order polynomial fitting scheme to adapt to different populations and significantly improve the model's anti-interference ability and generalization performance. In addition, this application also effectively improves compensation accuracy by using a partitioned calibration compensation method at different calibration points.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for improving eye movement and gaze compensation, characterized in that, include: Collect sample eye images and determine the pupil center and pupil region on the sample eye images; Identify suspected light spots in the pupil area; The suspected light spots are screened to obtain the actual light spots; Based on the actual light spot and the pupil center, eye parameters are calibrated and line of sight estimation points are calculated. Calculate the compensation coefficient of the line-of-sight estimation point based on the aforementioned line-of-sight estimation point; Acquire an actual eye diagram, calculate the compensation amount for each line of sight estimation point on the actual eye diagram based on the compensation coefficient of the line of sight estimation point, and compensate the line of sight estimation point based on the compensation amount; The calculation of the compensation coefficient for the line-of-sight estimation point based on the line-of-sight estimation point includes: Obtain the true coordinates of different fixation points on the sample eye diagram; Calculate the estimated coordinates of different fixation points based on the eye parameters of each of the sample eye diagrams, and calculate the difference between the estimated coordinates of different fixation points and the true coordinates in terms of horizontal and vertical coordinates; Based on the difference between the horizontal and vertical coordinates, the estimated coordinates of the abnormal gaze points are removed to obtain the effective estimated coordinates of the gaze points; The sample eye diagrams with pupil areas within a preset pixel range are normalized to obtain normalized pupil areas. A polynomial model is constructed and optimized. The compensation coefficient of the gaze estimation point is calculated using the polynomial model based on the normalized pupil area.

2. The method for improving eye movement and gaze compensation as described in claim 1, characterized in that, Before identifying a suspected light spot in the pupil region, the process includes: Calculate the average gray value of all pixels in the pupil region, and subtract the average gray value from the gray value of each pixel in the pupil region to obtain a temporary pupil region; The temporary pupil region is binarized based on the grayscale mean to obtain a binary image, which is used to find suspected light spots.

3. The method for improving eye movement and gaze compensation as described in claim 2, characterized in that, After binarizing the temporary pupil region based on the grayscale mean, the process includes: The binary image is then subjected to noise reduction processing.

4. The method for improving eye movement and gaze compensation as described in claim 2, characterized in that, The process of screening the suspected light spots to obtain the actual light spots includes: Obtain a light spot whose two-dimensional area is within a preset pixel range from the suspected light spot to obtain a first light spot; Obtain a light spot whose outline roundness is within a preset roundness range, and obtain a second light spot; The center coordinates of the second light spot are obtained within the aspect ratio range of the pupil region to obtain the third light spot; Remove the extra third spot to obtain the actual spot.

5. The method for improving eye movement and gaze compensation as described in claim 4, characterized in that, The process of removing excess third light spots to obtain the actual light spot includes: When the total number of the third light spots is greater than a preset value, calculate the difference between the maximum and minimum values ​​of the center point coordinates of all the third light spots in the horizontal and vertical directions; Determine whether the difference between the maximum and minimum values ​​in the horizontal and vertical directions is greater than the horizontal and vertical thresholds, respectively. If so, remove the third spot whose difference between the maximum and minimum values ​​in the horizontal and vertical coordinate directions is greater than the horizontal and vertical coordinate thresholds, respectively, and calculate the mean horizontal and vertical coordinates of the remaining center coordinates of the third spot. Remove the third light spot with the largest difference from the corresponding mean values ​​of the horizontal and vertical coordinates in the horizontal and vertical directions, until the total number of the third light spots is not greater than the preset value, to obtain the actual light spot.

6. The method for improving eye movement and gaze compensation as described in claim 1, characterized in that, The construction and optimization of the polynomial model includes: A polynomial feature is constructed using a higher-order number, and then the polynomial feature is optimized by ridge regression to obtain the optimized polynomial model.

7. The method for improving eye movement and gaze compensation as described in claim 1, characterized in that, The process of acquiring the actual eye diagram, calculating the compensation amount for each gaze estimation point on the actual eye diagram based on the compensation coefficient of the gaze estimation point, and compensating the gaze estimation points according to the compensation amount includes: In the actual eye diagram, identify the n calibration points closest to the line-of-sight estimation point, where n is less than the number of calibration points N for the compensation coefficient; Based on the two-dimensional pupil area of ​​the actual eye diagram, pupil normalization is performed on the pupils of the n calibration points respectively; Calculate the Euclidean distance between the line-of-sight estimation point and the calibration point respectively, and calculate the compensation weights of n calibration points based on the Euclidean distance; The compensation amount for each of the n calibration points is calculated based on the compensation coefficient and the compensation weight, and the line-of-sight estimation point is compensated based on the compensation amount.

Images