Eye movement tracking and positioning method and device

Abstract

The invention discloses an eye movement tracking and positioning method and device. The method comprises the following steps: acquiring to-be-processed image information; the to-be-processed image information comprises two to-be-processed images; analyzing and identifying the to-be-processed image information to obtain target eye movement coordinate information; the target eye movement coordinate information comprises two pieces of target eye movement coordinate position information; the target eye movement coordinate position information comprises first target eye movement coordinate position information and second target eye movement coordinate position information; and performing calculation processing on the target eye movement coordinate information by using the tracking and positioning model to obtain target positioning coordinate information. Therefore, the eye movement tracking and positioning accuracy and efficiency can be improved, and the use efficiency and the user experience of the eyeball tracking equipment can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an eye tracking positioning method and device. Background Art

[0002] With the rapid development of eye tracking technology, its application in the field of computer vision is increasing. A key challenge of eye tracking technology is to achieve fast and accurate eye tracking, especially in head-mounted devices. At present, most eye tracking devices need to be calibrated before use, which is usually done by requiring the user to look at multiple points on the screen. This calibration process is not only cumbersome, but also affects the user experience. Even if single-point calibration or scene saliency assisted calibration is used, the calibration problem cannot be completely solved. Therefore, an eye tracking positioning method and device are provided to improve the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of eye tracking devices. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an eye tracking positioning method and device that are conducive to improving the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of eye tracking equipment.

[0004] In order to solve the above technical problems, the first aspect of the embodiment of the present invention discloses an eye tracking positioning method, the method comprising:

[0005] Acquire image information to be processed; the image information to be processed includes 2 images to be processed;

[0006] Analyze and identify the image information to be processed to obtain target eye movement coordinate information; the target eye movement coordinate information includes two target eye movement coordinate position information; the target eye movement coordinate position information includes first target eye movement coordinate position information and second target eye movement coordinate position information;

[0007] The target eye movement coordinate information is calculated and processed using a tracking positioning model to obtain target positioning coordinate information.

[0008] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the analyzing and identifying the image information to be processed to obtain the target eye movement coordinate information includes:

[0009] Performing segmentation and recognition processing on the image information to be processed to obtain first target point information; the first target point information includes two first target point position information;

[0010] Target eye movement coordinate information is determined based on the image information to be processed and the first target point information.

[0011] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the segmentation and recognition processing of the image information to be processed to obtain the first target point information includes:

[0012] For any of the images to be processed, the image to be processed is reduced by half to obtain a first processed image corresponding to the image to be processed;

[0013] Binarize the first processed image based on a first binary threshold to obtain a second processed image corresponding to the image to be processed;

[0014] Based on the second processed image, the first target point position information corresponding to the image to be processed is determined.

[0015] As an optional implementation manner, in the first aspect of the embodiment of the present invention, determining the first target point position information corresponding to the image to be processed based on the second processed image includes:

[0016] Performing contour recognition processing on the second processed image to obtain first image point contour information; the first image point contour information includes a plurality of sequentially distributed contour information to be processed;

[0017] sequentially determining the contour information to be processed in the first image point contour information as the contour information to be screened;

[0018] Determine whether the number of contour points in the contour information to be filtered is not less than M, and obtain a first number determination result; wherein M is an odd number not less than 20;

[0019] When the first number judgment result is yes, the to-be-processed contour information corresponding to the to-be-screened contour information is deleted from the first image point contour information, and the judgment process corresponding to the first number judgment result is ended;

[0020] When the first number judgment result is no, calculating the maximum distance value of the contour of the contour information to be filtered;

[0021] Determine whether the maximum distance value of the contour is within the contour value interval, and obtain a contour interval determination result;

[0022] When the contour interval judgment result is yes, the contour information to be processed corresponding to the contour information to be filtered is deleted from the first image point contour information, and the judgment process corresponding to the first quantity judgment result is ended;

[0023] When the contour interval judgment result is no, ending the judgment process corresponding to the first quantity judgment result;

[0024] Determine whether the contour information to be processed corresponding to the contour information to be filtered is the last contour information to be processed in the first image point contour information, and obtain a first order determination result;

[0025] When the first sequence judgment result is no, triggering the step of sequentially determining the contour information to be processed in the first image point contour information as the contour information to be screened;

[0026] When the first order judgment result is yes, the first image point contour information is scored and screened to obtain the first target point position information corresponding to the image to be processed.

[0027] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the scoring and screening processing of the first image point contour information to obtain the first target point position information corresponding to the image to be processed includes:

[0028] For any of the to-be-processed contour information in the first image point contour information, use the contour scoring model to perform scoring and evaluation processing on the to-be-processed contour information to obtain a contour scoring value corresponding to the to-be-processed contour information;

[0029] Wherein, the profile scoring model is:

[0030] PF = 0.5 * A1 + 0.5 * A2;

[0031] Wherein, PF is the contour score value; A1 is the ratio of the minor axis to the major axis of the ellipse corresponding to the contour information to be processed; A2 is the ellipse integrity level corresponding to the contour information to be processed;

[0032] Selecting the processing profile information corresponding to the largest profile score value as target processing profile information;

[0033] The contour center point position information corresponding to the target processing contour information is determined as the first target point position information corresponding to the image to be processed.

[0034] As an optional implementation manner, in the first aspect of the embodiment of the present invention, determining the target eye movement coordinate information based on the image information to be processed and the first target point information includes:

[0035] For any of the images to be processed, binarization processing is performed on the image to be processed based on the second binary threshold value to obtain a third processed image corresponding to the image to be processed;

[0036] Identify the light spot area in the third processed image to obtain first processed light spot area information corresponding to the image to be processed; the first processed light spot area information includes a plurality of first processed light spot areas;

[0037] Eliminate the first processing spot region whose area in the first processing spot region is greater than an area threshold from the first processing spot region information, and obtain second processing spot region information corresponding to the image to be processed; the second processing spot region information includes a plurality of second processing spot regions;

[0038] For any second processing light spot area, calculating the area point position distance between the area center point position information corresponding to the second processing light spot area and the area point position information corresponding to the first target point position information of the image to be processed;

[0039] Determine whether the regional point position distance is greater than the regional distance threshold, and obtain a regional distance determination result;

[0040] When the area distance judgment result is yes, the second processed light spot area is removed from the second processed light spot area information, and the judgment process corresponding to the area distance judgment result is ended;

[0041] When the regional distance judgment result is no, obtaining the historical regional center point position information corresponding to the regional center point position information;

[0042] Determine whether the brightness value corresponding to each historical area center point in the historical area center point position information is greater than a brightness threshold, and obtain a brightness determination result;

[0043] When the brightness judgment result is yes, the second processed light spot area is removed from the second processed light spot area information, and the judgment process corresponding to the area distance judgment result is ended;

[0044] When the brightness judgment result is no, ending the judgment process corresponding to the area distance judgment result;

[0045] Based on the second processing light spot area information and the angle threshold, determining a target light spot area corresponding to the image to be processed;

[0046] Determine the area center point position information corresponding to the target light spot area as the target eye movement coordinate position information corresponding to the image to be processed.

[0047] As an optional implementation, in the first aspect of the embodiment of the present invention, the use of the tracking positioning model to calculate and process the target eye movement coordinate information to obtain the target positioning coordinate information includes:

[0048] Get model parameter information;

[0049] Using the tracking positioning model to calculate and process the model parameter information and the target eye movement coordinate information to obtain the target positioning coordinate information;

[0050] Wherein, the tracking and positioning model is:

[0051]

[0052] Wherein, (x1, y1) is the target positioning coordinate information; (L x ,L y ) and (R x ,R y ) are respectively the first target eye movement coordinate position information and the second target eye movement coordinate position information in the target eye movement coordinate information; a0, a1, a2, a3, a4, b0, b1, b2, b3 and b4 are respectively the model parameters in the model parameter information.

[0053] A second aspect of an embodiment of the present invention discloses an eye tracking and positioning device, the device comprising:

[0054] An acquisition module, used for acquiring image information to be processed; the image information to be processed includes 2 images to be processed;

[0055] A first processing module is used to analyze and identify the image information to be processed to obtain target eye movement coordinate information; the target eye movement coordinate information includes two target eye movement coordinate position information; the target eye movement coordinate position information includes first target eye movement coordinate position information and second target eye movement coordinate position information;

[0056] The second processing module is used to calculate and process the target eye movement coordinate information using the tracking positioning model to obtain the target positioning coordinate information.

[0057] The third aspect of the present invention discloses another eye tracking and positioning device, the device comprising:

[0058] A memory storing executable program code;

[0059] a processor coupled to the memory;

[0060] The processor calls the executable program code stored in the memory to execute part or all of the steps in the eye tracking and positioning method disclosed in the first aspect of the embodiment of the present invention.

[0061] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the eye tracking and positioning method disclosed in the first aspect of an embodiment of the present invention.

[0062] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0063] In the embodiment of the present invention, the image information to be processed is obtained; the image information to be processed includes 2 images to be processed; the image information to be processed is analyzed and identified to obtain the target eye movement coordinate information; the target eye movement coordinate information includes 2 target eye movement coordinate position information; the target eye movement coordinate position information includes the first target eye movement coordinate position information and the second target eye movement coordinate position information; the target eye movement coordinate information is calculated and processed using the tracking positioning model to obtain the target positioning coordinate information. It can be seen that the present application is conducive to improving the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of the eye tracking device. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0065] Figure 1 It is a flow chart of an eye tracking and positioning method disclosed in an embodiment of the present invention;

[0066] Figure 2 It is a structural schematic diagram of an eye tracking and positioning device disclosed in an embodiment of the present invention;

[0067] Figure 3 It is a schematic diagram of the structure of another eye tracking and positioning device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0070] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0071] The present invention discloses an eye tracking positioning method and device, which are beneficial to improving the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of eye tracking equipment. Detailed descriptions are given below.

[0072] Embodiment 1

[0073] See also Figure 1 , Figure 1 : is a flow chart of an eye tracking positioning method disclosed in an embodiment of the present invention. Figure 1 The eye tracking positioning method described is applied to an eye tracking system, such as a local server or cloud server for eye tracking positioning management, and the embodiments of the present invention do not limit this. Figure 1 As shown, the eye tracking positioning method may include the following operations:

[0074] 101. Obtain image information to be processed.

[0075] In the embodiment of the present invention, the image information to be processed includes 2 images to be processed.

[0076] 102. Analyze and identify the image information to be processed to obtain target eye movement coordinate information.

[0077] In the embodiment of the present invention, the target eye movement coordinate information includes two target eye movement coordinate position information; the target eye movement coordinate position information includes first target eye movement coordinate position information and second target eye movement coordinate position information.

[0078] 103. The target eye movement coordinate information is calculated and processed using the tracking positioning model to obtain the target positioning coordinate information.

[0079] It should be noted that the eye tracking positioning method of the present application can be applied to head-mounted AR devices. First, the user's eye image is obtained and the uncalibrated eye tracking parameters are obtained. Then, personalized calibration is performed, such as five-point calibration, nine-point calibration or single-point space calibration, to obtain the key information required for eye tracking. This information is expressed in different forms, such as linear polynomial matrix coefficients of two-dimensional line of sight estimation methods, affine transformation matrices, or fully connected layer coefficients in appearance-based deep learning methods. After calibration, the accuracy of eye tracking can be significantly improved. In the early stage of model operation, we will record the image of exactly eight spots detected as the reference frame, and record the position of each spot in sequence as a template. If the number of spots detected is exactly eight points, we will use a moving average algorithm to update the average position of the spots. If the number of spots detected in the end exceeds eight points, we will use an affine transformation method to remove interference items. Specifically, considering that the geometric positions between all Purchin spots are relatively fixed, when the eye position changes, there is a certain affine transformation relationship between the newly detected Purchin spot combination and the standard template. In this process, we first determine the number of noise points that need to be eliminated. Then, we enumerate all possible eight-point combinations from the candidate points and calculate the corresponding affine transformation matrix between them and the template points. After applying these affine transformations, we calculate the Euclidean distance error between the transformed points and the template points and record it. Finally, we select the point set with the smallest error as the final candidate point set. This method effectively removes noise points, but in some cases, some light spots may be missing, for example, two or three points may be missing. In this case, since the overall geometric position relationship has not changed significantly, we can still use the affine transformation method to predict the possible location of the missing points. We assume that the total number of template points is n and the number of points actually detected is m. We enumerate all m combinations of points from the template points and calculate the affine transformation matrix between these combinations and the actual detection point set. Then, we apply these affine transformations to project the template points and calculate the Euclidean error between these points and the actual point set. By recording and comparing these errors, we select the affine transformation matrix with the smallest error. Finally, the entire template point set is projected using the affine transformation matrix to make up for the missing items in the original point set. This method is mainly used to deal with a small number of missing points. When the number of missing points is large, for example, less than four, we consider the image quality to be poor, so the image will be removed and not included in the subsequent processing process.

[0080] It should be noted that the eye tracking positioning method of the present application realizes instant and accurate interaction with the eye movement device without the need for the user to perform a tedious calibration process. This innovative method improves the efficiency and accuracy of eye movement interaction, making the user experience more natural and smooth. Especially in virtual reality or augmented reality environments, the application of this technology will greatly enhance the user's immersion and interactive experience.

[0081] It can be seen that implementing the eye tracking and positioning method described in the embodiment of the present invention is conducive to improving the accuracy and efficiency of eye tracking and positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0082] In an optional embodiment, the above-mentioned analyzing and identifying the image information to be processed to obtain the target eye movement coordinate information includes:

[0083] Perform segmentation and recognition processing on the image information to be processed to obtain first target point information; the first target point information includes two first target point position information;

[0084] Based on the image information to be processed and the first target point information, target eye movement coordinate information is determined.

[0085] It can be seen that implementing the eye tracking and positioning method described in the embodiment of the present invention is conducive to improving the accuracy and efficiency of eye tracking and positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0086] In another optional embodiment, performing segmentation and recognition processing on the image information to be processed to obtain the first target point information includes:

[0087] For any image to be processed, the image to be processed is reduced by half to obtain a first processed image corresponding to the image to be processed;

[0088] Binarization is performed on the first processed image based on the first binary threshold value to obtain a second processed image corresponding to the image to be processed;

[0089] Based on the second processed image, the first target point position information corresponding to the image to be processed is determined.

[0090] It should be noted that the above-mentioned halving of the image to be processed is to compress the size of the image to be processed from 480×640 pixels to 240×320 pixels in proportion to improve the algorithm processing efficiency.

[0091] It should be noted that binarization of the first processed image based on the first binary threshold can effectively remove most of the interference information in the image, such as the white of the eye, iris, eyelashes, etc., while retaining the most obvious pupil features. Compared with most current pupil detection algorithms, our algorithm process is greatly simplified while retaining the key features of the pupil to the greatest extent.

[0092] It can be seen that implementing the eye tracking and positioning method described in the embodiment of the present invention is conducive to improving the accuracy and efficiency of eye tracking and positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0093] In yet another optional embodiment, determining the first target point position information corresponding to the image to be processed based on the second processed image includes:

[0094] Performing contour recognition processing on the second processed image to obtain contour information of the first image point; the contour information of the first image point includes a plurality of contour information to be processed that are distributed in sequence;

[0095] sequentially determining the contour information to be processed in the contour information of the first image point as the contour information to be screened;

[0096] Determine whether the number of contour points in the contour information to be filtered is not less than M, and obtain a first number determination result; M is an odd number not less than 20;

[0097] When the first quantity judgment result is yes, the contour information to be processed corresponding to the contour information to be screened is deleted from the first image point contour information, and the judgment process corresponding to the first quantity judgment result is ended;

[0098] When the first quantity judgment result is no, calculating the maximum distance value of the contour of the contour information to be filtered;

[0099] Determine whether the maximum distance value of the contour is within the contour value interval, and obtain the contour interval judgment result;

[0100] When the contour interval judgment result is yes, the contour information to be processed corresponding to the contour information to be screened is deleted from the contour information of the first image point, and the judgment process corresponding to the first quantity judgment result is ended;

[0101] When the contour interval judgment result is no, the judgment process corresponding to the first quantity judgment result is ended;

[0102] Determine whether the contour information to be processed corresponding to the contour information to be filtered is the last contour information to be processed in the contour information of the first image point, and obtain a first order determination result;

[0103] When the result of the first sequence judgment is no, triggering execution to sequentially determine the contour information to be processed in the contour information of the first image point as the contour information to be screened;

[0104] When the first order judgment result is yes, the first image point contour information is scored and screened to obtain the first target point position information corresponding to the image to be processed.

[0105] Preferably, the above M is 25.

[0106] It should be noted that the above contour value interval may be from the minimum pupil diameter to the maximum pupil diameter.

[0107] It should be noted that the above method removes most of the tiny noise points and the outer contours formed by lighting due to lighting reasons by judging whether the maximum distance value of the contour is within the contour value interval. Because the contours after binarization are relatively sparse, basically only the pupil contour and very few interference items are retained at this time.

[0108] It should be noted that the above maximum distance value of the contour represents the maximum value of the distance between any two points in the contour.

[0109] It can be seen that implementing the eye tracking and positioning method described in the embodiment of the present invention is conducive to improving the accuracy and efficiency of eye tracking and positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0110] In yet another optional embodiment, scoring and screening the first image point contour information to obtain the first target point position information corresponding to the image to be processed includes:

[0111] For any contour information to be processed in the contour information of the first image point, scoring and evaluating the contour information to be processed using the contour scoring model to obtain a contour scoring value corresponding to the contour information to be processed;

[0112] Among them, the contour scoring model is:

[0113] PF = 0.5 * A1 + 0.5 * A2;

[0114] Where PF is the contour score; A1 is the ratio of the minor axis to the major axis of the ellipse corresponding to the contour information to be processed; A2 is the ellipse integrity level corresponding to the contour information to be processed;

[0115] Select the processed contour information corresponding to the maximum contour score value as the target processed contour information;

[0116] The contour center point position information corresponding to the target processing contour information is determined as the first target point position information corresponding to the image to be processed.

[0117] It should be noted that the ratio of the minor axis to the major axis of the ellipse corresponding to the contour information to be processed represents the ratio of the minor axis to the major axis of the fitted ellipse, and the closer it is to 1, the closer the ellipse is to a complete circle.

[0118] It should be noted that the ellipse integrity level corresponding to the contour information to be processed can be five levels: 0, 0.25, 0.5, 0.75, and 1. Furthermore, in the original contour, if there is a point in the four directions centered on the center of the fitted ellipse, 0.25 points will be added. For example, if the A2 value is 0.25, it means that all contours are concentrated in a certain direction of the fitted contour, such as the upper left corner. At this time, the contour that may be used to fit the ellipse is only a small arc of the ellipse. When the value is 1, it means that the center of the ellipse is located inside the polygon constructed by the contour. In this case, the integrity of the ellipse contour is more ideal. If there are multiple ellipse contours that meet the conditions, we will select the ellipse with the highest score as the final pupil ellipse contour.

[0119] Since the lighting conditions in actual application environments may be changeable, although the infrared light source can provide a stable spot effect, the intensity change of natural light may still affect the detection effect. Therefore, our algorithm designs a threshold adaptive mechanism. When the user wears the headset for the first time, the system will automatically perform adaptive detection on the image. If the initial image cannot correctly detect the pupil, the algorithm will continuously adjust the binarization threshold until the most ideal contour score value is obtained. Once the number of adjustments reaches a certain number, the system will lock the current threshold as the optimal binarization threshold under the lighting environment and remain unchanged in subsequent line of sight estimation tasks. If the pupil cannot be detected normally, such as closing the eyes, the algorithm will return a null value and stop all subsequent processing processes to ensure the accuracy and reliability of the data.

[0120] It can be seen that implementing the eye tracking and positioning method described in the embodiment of the present invention is conducive to improving the accuracy and efficiency of eye tracking and positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0121] In an optional embodiment, the above-mentioned determining the target eye movement coordinate information based on the image information to be processed and the first target point information includes:

[0122] For any image to be processed, binarization processing is performed on the image to be processed based on the second binary threshold value to obtain a third processed image corresponding to the image to be processed;

[0123] Identify the light spot area in the third processed image, and obtain first processed light spot area information corresponding to the image to be processed; the first processed light spot area information includes a plurality of first processed light spot areas;

[0124] Eliminate the first processing spot region whose area is greater than the area threshold in the first processing spot region information from the first processing spot region information, and obtain the second processing spot region information corresponding to the image to be processed; the second processing spot region information includes a plurality of second processing spot regions;

[0125] For any second processing light spot area, calculating the area point position distance between the area center point position information corresponding to the second processing light spot area and the area point position information corresponding to the first target point position information of the image to be processed;

[0126] Determine whether the regional point position distance is greater than the regional distance threshold, and obtain the regional distance judgment result;

[0127] When the regional distance judgment result is yes, the second processed light spot area is removed from the second processed light spot area information, and the judgment process corresponding to the regional distance judgment result is ended;

[0128] When the regional distance judgment result is no, the historical regional center point location information corresponding to the regional center point location information is obtained;

[0129] Determine whether the brightness value corresponding to each historical area center point in the historical area center point position information is greater than a brightness threshold, and obtain a brightness determination result;

[0130] When the brightness judgment result is yes, the second processed light spot area is removed from the second processed light spot area information, and the judgment process corresponding to the area distance judgment result is ended;

[0131] When the brightness judgment result is no, the judgment process corresponding to the regional distance judgment result ends;

[0132] Based on the second processing spot area information and the angle threshold, determining the target spot area corresponding to the image to be processed;

[0133] Determine the area center point position information corresponding to the target light spot area as the target eye movement coordinate position information corresponding to the image to be processed.

[0134] It should be noted that the above-mentioned binarization processing of the image to be processed based on the second binary threshold to obtain the third processed image corresponding to the image to be processed is to establish points with pixel values ​​greater than the second binary threshold as light spots. Further, the above-mentioned second binary threshold can be set by the user or determined based on historical data, which is not limited in the embodiment of the present invention.

[0135] It should be noted that the above-mentioned identification of the light spot area in the third processed image can be implemented based on a neural network or based on a preset rule, which is not limited in the embodiment of the present invention.

[0136] It should be noted that the above-mentioned angle threshold, area distance threshold and brightness threshold may be set by a user or determined based on historical data, and the embodiment of the present invention does not limit this.

[0137] In this optional embodiment, as an optional implementation manner, the above-mentioned determining the target spot area corresponding to the image to be processed based on the second processing spot area information and the angle threshold includes:

[0138] Calculate the angle between the center point position information of the second processing spot area and the first target point position information of the image to be processed to obtain the point angle information;

[0139] Deleting the second processing light spot area corresponding to the point angle in the point angle information that is greater than the angle threshold from the second processing light spot area information;

[0140] Calculate the regional angle of the regional center point position information corresponding to any two second processing spot regions to obtain regional angle information;

[0141] Based on the area angle information, the target spot area corresponding to the image to be processed is determined.

[0142] It should be noted that the above-mentioned determination of the target spot area corresponding to the image to be processed based on the area angle information may be implemented based on a neural network or based on a preset rule, which is not limited in the embodiment of the present invention.

[0143] It can be seen that implementing the eye tracking and positioning method described in the embodiment of the present invention is conducive to improving the accuracy and efficiency of eye tracking and positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0144] In another optional embodiment, the target eye movement coordinate information is calculated and processed using the tracking positioning model to obtain the target positioning coordinate information, including:

[0145] Get model parameter information;

[0146] The tracking positioning model is used to calculate and process the model parameter information and the target eye movement coordinate information to obtain the target positioning coordinate information;

[0147] Among them, the tracking and positioning model is:

[0148]

[0149] Where (x1, y1) is the target positioning coordinate information; (L x ,L y ) and (R x ,R y) are respectively the first target eye movement coordinate position information and the second target eye movement coordinate position information in the target eye movement coordinate information; a0, a1, a2, a3, a4, b0, b1, b2, b3 and b4 are respectively the model parameters in the model parameter information.

[0150] In this optional embodiment, as an optional implementation manner, the above-mentioned obtaining of model parameter information includes:

[0151] Get model image information;

[0152] The model image information is filtered using a Gaussian blur function to obtain filtered image information;

[0153] Positioning the two-dimensional code in the filtered image information to obtain the target two-dimensional code information;

[0154] Perform binary conversion on the target two-dimensional code information to obtain binary two-dimensional code information;

[0155] Using a Reed-Solomon encoder to perform error correction processing on the binary two-dimensional code information to obtain corrected two-dimensional code information;

[0156] The QR code encoding rules are used to convert the format of the corrected two-dimensional code information to obtain the model parameter information.

[0157] It should be noted that the above QR code encoding rules are constructed based on the ISO / IEC 18004 standard.

[0158] It can be seen that implementing the eye tracking and positioning method described in the embodiment of the present invention is conducive to improving the accuracy and efficiency of eye tracking and positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0159] Embodiment 2

[0160] See also Figure 2 , Figure 2 : is a schematic diagram of the structure of an eye tracking and positioning device disclosed in an embodiment of the present invention. Figure 2 The described device can be applied to an eye tracking system, such as a local server or a cloud server for eye tracking positioning management, and the embodiments of the present invention are not limited thereto. Figure 2 As shown, the device may include:

[0161] The acquisition module 201 is used to acquire the image information to be processed; the image information to be processed includes 2 images to be processed;

[0162] The first processing module 202 is used to analyze and identify the image information to be processed to obtain target eye movement coordinate information; the target eye movement coordinate information includes two target eye movement coordinate position information; the target eye movement coordinate position information includes first target eye movement coordinate position information and second target eye movement coordinate position information;

[0163] The second processing module 203 is used to calculate and process the target eye movement coordinate information using the tracking positioning model to obtain the target positioning coordinate information.

[0164] It can be seen that the implementation Figure 2 The described eye tracking positioning device is beneficial to improving the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0165] In another optional embodiment, Figure 2 As shown, the first processing module 202 performs analysis and recognition processing on the image information to be processed to obtain target eye movement coordinate information, including:

[0166] Perform segmentation and recognition processing on the image information to be processed to obtain first target point information; the first target point information includes two first target point position information;

[0167] Based on the image information to be processed and the first target point information, target eye movement coordinate information is determined.

[0168] It can be seen that the implementation Figure 2 The described eye tracking positioning device is beneficial to improving the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0169] In yet another optional embodiment, Figure 2 As shown, the first processing module 202 performs segmentation and recognition processing on the image information to be processed to obtain first target point information, including:

[0170] For any image to be processed, the image to be processed is reduced by half to obtain a first processed image corresponding to the image to be processed;

[0171] Binarization is performed on the first processed image based on the first binary threshold value to obtain a second processed image corresponding to the image to be processed;

[0172] Based on the second processed image, the first target point position information corresponding to the image to be processed is determined.

[0173] It can be seen that the implementation Figure 2 The described eye tracking positioning device is beneficial to improving the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0174] In yet another optional embodiment, Figure 2 As shown, the first processing module 202 determines the first target point position information corresponding to the image to be processed based on the second processed image, including:

[0175] Performing contour recognition processing on the second processed image to obtain contour information of the first image point; the contour information of the first image point includes a plurality of contour information to be processed that are distributed in sequence;

[0176] sequentially determining the contour information to be processed in the contour information of the first image point as the contour information to be screened;

[0177] Determine whether the number of contour points in the contour information to be filtered is not less than M, and obtain a first number determination result; M is an odd number not less than 20;

[0178] When the first quantity judgment result is yes, the contour information to be processed corresponding to the contour information to be screened is deleted from the first image point contour information, and the judgment process corresponding to the first quantity judgment result is ended;

[0179] When the first quantity judgment result is no, calculating the maximum distance value of the contour of the contour information to be filtered;

[0180] Determine whether the maximum distance value of the contour is within the contour value interval, and obtain the contour interval judgment result;

[0181] When the contour interval judgment result is yes, the contour information to be processed corresponding to the contour information to be screened is deleted from the contour information of the first image point, and the judgment process corresponding to the first quantity judgment result is ended;

[0182] When the contour interval judgment result is no, the judgment process corresponding to the first quantity judgment result is ended;

[0183] Determine whether the contour information to be processed corresponding to the contour information to be filtered is the last contour information to be processed in the contour information of the first image point, and obtain a first order determination result;

[0184] When the result of the first sequence judgment is no, triggering execution to sequentially determine the contour information to be processed in the contour information of the first image point as the contour information to be screened;

[0185] When the first order judgment result is yes, the first image point contour information is scored and screened to obtain the first target point position information corresponding to the image to be processed.

[0186] It can be seen that implementation Figure 2 The described eye tracking positioning device is beneficial to improving the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0187] In yet another optional embodiment, Figure 2 As shown, the first processing module 202 performs scoring and screening processing on the first image point contour information to obtain the first target point position information corresponding to the image to be processed, including:

[0188] For any contour information to be processed in the contour information of the first image point, scoring and evaluating the contour information to be processed using the contour scoring model to obtain a contour scoring value corresponding to the contour information to be processed;

[0189] Among them, the contour scoring model is:

[0190] PF = 0.5 * A1 + 0.5 * A2;

[0191] Where PF is the contour score; A1 is the ratio of the minor axis to the major axis of the ellipse corresponding to the contour information to be processed; A2 is the ellipse integrity level corresponding to the contour information to be processed;

[0192] Select the processed contour information corresponding to the maximum contour score value as the target processed contour information;

[0193] The contour center point position information corresponding to the target processing contour information is determined as the first target point position information corresponding to the image to be processed.

[0194] It can be seen that the implementation Figure 2 The described eye tracking positioning device is beneficial to improving the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0195] In yet another optional embodiment, Figure 2 As shown, the first processing module 202 determines the target eye movement coordinate information based on the image information to be processed and the first target point information, including:

[0196] For any image to be processed, binarization processing is performed on the image to be processed based on the second binary threshold value to obtain a third processed image corresponding to the image to be processed;

[0197] Identify the light spot area in the third processed image, and obtain first processed light spot area information corresponding to the image to be processed; the first processed light spot area information includes a plurality of first processed light spot areas;

[0198] Eliminate the first processing spot region whose area is greater than the area threshold in the first processing spot region information from the first processing spot region information, and obtain the second processing spot region information corresponding to the image to be processed; the second processing spot region information includes a plurality of second processing spot regions;

[0199] For any second processing light spot area, calculating the area point position distance between the area center point position information corresponding to the second processing light spot area and the area point position information corresponding to the first target point position information of the image to be processed;

[0200] Determine whether the regional point position distance is greater than the regional distance threshold, and obtain the regional distance judgment result;

[0201] When the regional distance judgment result is yes, the second processed light spot area is removed from the second processed light spot area information, and the judgment process corresponding to the regional distance judgment result is ended;

[0202] When the regional distance judgment result is no, the historical regional center point location information corresponding to the regional center point location information is obtained;

[0203] Determine whether the brightness value corresponding to each historical area center point in the historical area center point position information is greater than a brightness threshold, and obtain a brightness determination result;

[0204] When the brightness judgment result is yes, the second processed light spot area is removed from the second processed light spot area information, and the judgment process corresponding to the area distance judgment result is ended;

[0205] When the brightness judgment result is no, the judgment process corresponding to the regional distance judgment result ends;

[0206] Based on the second processing spot area information and the angle threshold, determining the target spot area corresponding to the image to be processed;

[0207] Determine the area center point position information corresponding to the target light spot area as the target eye movement coordinate position information corresponding to the image to be processed.

[0208] It can be seen that implementation Figure 2 The described eye tracking positioning device is beneficial to improving the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0209] In yet another optional embodiment, Figure 2 As shown, the second processing module 203 uses the tracking positioning model to calculate and process the target eye movement coordinate information to obtain the target positioning coordinate information, including:

[0210] Get model parameter information;

[0211] The tracking positioning model is used to calculate and process the model parameter information and the target eye movement coordinate information to obtain the target positioning coordinate information;

[0212] Among them, the tracking and positioning model is:

[0213]

[0214] Where (x1, y1) is the target positioning coordinate information; (L x ,L y ) and (R x ,R y ) are respectively the first target eye movement coordinate position information and the second target eye movement coordinate position information in the target eye movement coordinate information; a0, a1, a2, a3, a4, b0, b1, b2, b3 and b4 are respectively the model parameters in the model parameter information.

[0215] It can be seen that implementation Figure 2 The described eye tracking positioning device is beneficial to improving the accuracy and efficiency of eye tracking positioning, thereby improving the use efficiency and user experience of the eye tracking device.

[0216] Embodiment 3

[0217] See also Figure 3 , Figure 3 is a schematic diagram of the structure of another eye tracking and positioning device disclosed in an embodiment of the present invention. Figure 3 The described device can be applied to an eye tracking system, such as a local server or a cloud server for eye tracking positioning management, and the embodiments of the present invention are not limited thereto. Figure 3 As shown, the device may include:

[0218] A memory 301 storing executable program codes;

[0219] a processor 302 coupled to the memory 301;

[0220] The processor 302 calls the executable program code stored in the memory 301 to execute the steps in the eye tracking positioning method described in the first embodiment.

[0221] Embodiment 4

[0222] An embodiment of the present invention discloses a computer-readable storage medium, which stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the eye tracking positioning method described in the first embodiment.

[0223] Embodiment 5

[0224] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the eye tracking positioning method described in the first embodiment.

[0225] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0226] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0227] Finally, it should be noted that the eye tracking and positioning method and device disclosed in the embodiments of the present invention disclose only the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An eye tracking positioning method, characterized in that: The method comprises: Acquire image information to be processed; the image information to be processed includes 2 images to be processed; Analyze and identify the image information to be processed to obtain target eye movement coordinate information; the target eye movement coordinate information includes two target eye movement coordinate position information; the target eye movement coordinate position information includes first target eye movement coordinate position information and second target eye movement coordinate position information; The target eye movement coordinate information is calculated and processed using a tracking positioning model to obtain target positioning coordinate information.

2. The eye tracking positioning method according to claim 1, characterized in that: The analyzing and identifying the image information to be processed to obtain target eye movement coordinate information includes: Performing segmentation and recognition processing on the image information to be processed to obtain first target point information; the first target point information includes two first target point position information; Target eye movement coordinate information is determined based on the image information to be processed and the first target point information.

3. The eye tracking positioning method according to claim 2, characterized in that: The step of performing segmentation and recognition processing on the image information to be processed to obtain first target point information includes: For any of the images to be processed, the image to be processed is reduced by half to obtain a first processed image corresponding to the image to be processed; Binarize the first processed image based on a first binary threshold to obtain a second processed image corresponding to the image to be processed; Based on the second processed image, the first target point position information corresponding to the image to be processed is determined.

4. The eye tracking positioning method according to claim 3, characterized in that: The determining, based on the second processed image, the first target point position information corresponding to the image to be processed includes: Performing contour recognition processing on the second processed image to obtain first image point contour information; the first image point contour information includes a plurality of sequentially distributed contour information to be processed; sequentially determining the contour information to be processed in the first image point contour information as the contour information to be screened; Determine whether the number of contour points in the contour information to be filtered is not less than M, and obtain a first number determination result; wherein M is an odd number not less than 20; When the first number judgment result is yes, the to-be-processed contour information corresponding to the to-be-screened contour information is deleted from the first image point contour information, and the judgment process corresponding to the first number judgment result is ended; When the first number judgment result is no, calculating the maximum distance value of the contour of the contour information to be filtered; Determine whether the maximum distance value of the contour is within the contour value interval, and obtain a contour interval determination result; When the contour interval judgment result is yes, the contour information to be processed corresponding to the contour information to be filtered is deleted from the first image point contour information, and the judgment process corresponding to the first quantity judgment result is ended; When the contour interval judgment result is no, ending the judgment process corresponding to the first quantity judgment result; Determine whether the contour information to be processed corresponding to the contour information to be filtered is the last contour information to be processed in the first image point contour information, and obtain a first order determination result; When the first sequence judgment result is no, triggering the step of sequentially determining the contour information to be processed in the first image point contour information as the contour information to be screened; When the first order judgment result is yes, the first image point contour information is scored and screened to obtain the first target point position information corresponding to the image to be processed.

5. The eye tracking positioning method according to claim 4, characterized in that: The scoring and screening process is performed on the first image point contour information to obtain the first target point position information corresponding to the image to be processed, including: For any of the to-be-processed contour information in the first image point contour information, using the contour scoring model to perform scoring and evaluation processing on the to-be-processed contour information to obtain a contour scoring value corresponding to the to-be-processed contour information; Wherein, the profile scoring model is: PF = 0.5 * A1 + 0.5 * A2; Wherein, PF is the contour score value; A1 is the ratio of the minor axis to the major axis of the ellipse corresponding to the contour information to be processed; A2 is the ellipse integrity level corresponding to the contour information to be processed; Selecting the processing profile information corresponding to the largest profile score value as target processing profile information; Determine the contour center point position information corresponding to the target processing contour information as the first target point position information corresponding to the image to be processed.

6. The eye tracking positioning method according to claim 2, characterized in that: The step of determining target eye movement coordinate information based on the image information to be processed and the first target point information includes: For any of the images to be processed, binarization processing is performed on the image to be processed based on the second binary threshold value to obtain a third processed image corresponding to the image to be processed; Identify the light spot area in the third processed image to obtain first processed light spot area information corresponding to the image to be processed; the first processed light spot area information includes a plurality of first processed light spot areas; Eliminate the first processing spot region whose area in the first processing spot region is greater than an area threshold from the first processing spot region information, and obtain second processing spot region information corresponding to the image to be processed; the second processing spot region information includes a plurality of second processing spot regions; For any second processing light spot area, calculating the area point position distance between the area center point position information corresponding to the second processing light spot area and the area point position information corresponding to the first target point position information of the image to be processed; Determine whether the regional point position distance is greater than the regional distance threshold, and obtain a regional distance determination result; When the area distance judgment result is yes, the second processed light spot area is removed from the second processed light spot area information, and the judgment process corresponding to the area distance judgment result is ended; When the regional distance judgment result is no, obtaining the historical regional center point position information corresponding to the regional center point position information; Determine whether the brightness value corresponding to each historical area center point in the historical area center point position information is greater than a brightness threshold, and obtain a brightness determination result; When the brightness judgment result is yes, the second processed light spot area is removed from the second processed light spot area information, and the judgment process corresponding to the area distance judgment result is ended; When the brightness judgment result is no, ending the judgment process corresponding to the area distance judgment result; Based on the second processing light spot area information and the angle threshold, determining a target light spot area corresponding to the image to be processed; Determine the area center point position information corresponding to the target light spot area as the target eye movement coordinate position information corresponding to the image to be processed.

7. The eye tracking positioning method according to claim 1, characterized in that: The method of calculating and processing the target eye movement coordinate information by using the tracking positioning model to obtain the target positioning coordinate information includes: Get model parameter information; Using the tracking positioning model to calculate and process the model parameter information and the target eye movement coordinate information to obtain the target positioning coordinate information; Wherein, the tracking and positioning model is: Wherein, (x1, y1) is the target positioning coordinate information; (L x ,L y ) and (R x ,R y ) are respectively the first target eye movement coordinate position information and the second target eye movement coordinate position information in the target eye movement coordinate information; a0, a1, a2, a3, a4, b0, b1, b2, b3 and b4 are respectively the model parameters in the model parameter information.

8. An eye tracking and positioning device, characterized in that: The device comprises: An acquisition module, used for acquiring image information to be processed; the image information to be processed includes 2 images to be processed; A first processing module is used to analyze and identify the image information to be processed to obtain target eye movement coordinate information; the target eye movement coordinate information includes two target eye movement coordinate position information; the target eye movement coordinate position information includes first target eye movement coordinate position information and second target eye movement coordinate position information; The second processing module is used to calculate and process the target eye movement coordinate information using the tracking positioning model to obtain the target positioning coordinate information.

9. An eye tracking and positioning device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the eye tracking and positioning method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when called, are used to execute the eye tracking and positioning method as described in any one of claims 1-7.

Images