A post-processing correction method and system for VR eye tracking data

Abstract

The present invention provides a post-processing correction method and system for VR eye tracking data, the method comprising: performing eye calibration to ensure that the eye tracker can accurately capture and record eye movements; collecting eye data after the eye calibration is successful; cleaning the eye data; performing post-processing of the eye data correction to generate new eye data; and updating and storing the new eye data in the VR system. The present invention utilizes the gaze following characteristics of moving objects in a specific picture scene in a VR head display device, and effectively improves the accuracy of eye data coordinate positioning by analyzing the correlation between the user's gaze point trajectory and the target object's motion trajectory, and using a multivariate linear regression model for secondary correction; this process not only includes cleaning, segmenting and de-spike processing of the original data, but also innovatively introduces target object tracking data based on scene content, significantly improving the performance and practicality of the eye tracking system.

Description

Technical Field

[0001] The present invention relates to the field of eye tracking technology, and in particular to a post-processing correction method and system for VR eye tracking data. Background Art

[0002] VR technology is a brand-new practical technology developed in the 20th century; virtual reality technology encompasses computers, electronic information, and simulation technology. Its basic implementation method is based on computer technology, utilizing and integrating the latest developments in three-dimensional graphics technology, multimedia technology, simulation technology, display technology, servo technology, and other high-tech technologies, and using computers and other equipment to create a realistic three-dimensional visual virtual world (in the future, it will even include multiple sensory experiences such as touch and smell), thereby giving people in the virtual world an immersive feeling; with the continuous development of social productivity and science and technology, the demand for VR technology in all walks of life is growing; VR technology has also made great progress and is gradually becoming a new field of science and technology.

[0003] As an important tool in psychology research, eye trackers are widely used in various fields, such as industrial design, advertising design, usability testing, market research, sports, traffic driving, air transportation, information dissemination, judicial detective and other disciplines. In addition, miniaturized eye trackers can be built into VR headsets. They use pupil / cornea reflection infrared imaging to capture the movement angle of the pupil to locate the wearer's gaze point in the VR headset, and use eye gaze to complete tasks in the 3D virtual scene of related applications in the VR headset.

[0004] Because the VR headset has an eye tracking coordinate calibration process before collecting eye movement data, the user needs to concentrate and look at the only bright spot in the headset. The bright spot will appear in different positions of the headset, such as Figure 1 Shown: Schematic diagram of the paradigm for eye movement coordinate correction.

[0005] Eye movement data collection starts after calibration. However, due to factors such as user inattention or hardware and software failures, eye tracking positioning may be inaccurate, thus affecting the accuracy of eye movement data, which may seriously affect the user experience and the accuracy of evaluation results given by some application software.

[0006] In the current field of virtual reality (VR), eye tracking is a key technology that enables more natural and accurate human-computer interaction by capturing and analyzing user eye movement data. However, in existing VR eye tracking technology, due to the influence of various factors, such as initial calibration errors, equipment accuracy limitations, environmental interference, etc., the collected eye movement data may have certain positioning deviations, thus affecting the accuracy and reliability of subsequent research or applications.

[0007] Deficiencies of existing technology:

[0008] 1. Imperfect eye movement calibration: Existing eye tracking systems may have limitations during the calibration phase, resulting in the inability of even a calibrated eye tracker to accurately capture eye movement trajectories during actual use.

[0009] 2. The original eye movement data has large errors: Due to factors such as noise, hardware sampling errors, and individual differences among users, the original collected eye movement data may have a certain degree of inaccuracy and instability.

[0010] 3. Lack of correction mechanism in dynamic scenes: Traditional processing methods may fail to fully utilize the dynamic information of objects in the VR environment, especially the user's gaze movement characteristics following the moving target, and fail to effectively utilize this behavior pattern to improve the accuracy of eye movement data positioning.

[0011] Therefore, the prior art has deficiencies and needs further improvement. Summary of the invention

[0012] In view of the problems existing in the prior art, the present invention provides a post-processing correction method and system for VR eye tracking data.

[0013] To achieve the above object, the specific scheme of the present invention is as follows:

[0014] The present invention provides a post-processing correction method for VR eye tracking data, the method comprising the following steps:

[0015] S1, perform eye movement calibration to ensure that the eye tracker can accurately capture and record eye movements;

[0016] S2, collect eye movement data after successful eye movement calibration;

[0017] S3, cleaning of eye movement data;

[0018] S4, performing post-processing of eye movement data correction to generate new eye movement data;

[0019] S5, updating and storing the new eye movement data in the VR system.

[0020] Furthermore, in step S1, the eye movement calibration adopts the gaze tracking of a moving target in a specific screen scene part of the VR head display device, and utilizes the overlap between the motion trajectory of the moving target and the gaze point trajectory to correct the coordinate positioning of the eye movement data.

[0021] Further, in step S2, the eye movement data includes a timestamp, horizontal and vertical coordinates of the eye gaze point, and left and right eye status information.

[0022] Furthermore, in step S3, the eye movement data is cleaned by removing invalid numbers or garbled information including NaN and inf, and deleting the data at the time when both eyes are closed.

[0023] Furthermore, in step S4, the eye movement data is corrected and processed, specifically including the following steps:

[0024] S401, segmenting the eye movement data and taking the gaze following part;

[0025] S402, calculating the correlation coefficient between the gaze tracking trajectory of the eye movement data and the movement trajectory of the target object;

[0026] S403, determining whether the correlation coefficient is greater than 0.9, if the correlation coefficient is greater than 0.9, performing a de-peaking process to remove extreme values, otherwise no processing is performed;

[0027] S404, after de-peaking, the least squares method is used to calculate the estimated values ​​of the coefficient matrices A and B of the linear regression model (Y=X*A+B) ;

[0028] S405, using the linear model ( ), predict the user’s actual gaze point at each moment and generate new corrected eye movement data.

[0029] The present invention also provides a post-processing correction system for VR eye tracking data, which is used to implement the above method. The system comprises:

[0030] Eye movement calibration module, used for initial eye movement calibration;

[0031] An eye movement data collection unit, used for collecting eye movement data;

[0032] Data preprocessing unit, used to clean eye movement data;

[0033] The core data processing module is used for post-processing of eye movement data correction;

[0034] The core data processing module includes: a data segmentation unit, a correlation analysis unit, a peak removal processing unit, and a new data generation and storage unit;

[0035] A data segmentation unit, used for identifying and extracting data of the gaze-following part;

[0036] A correlation analysis unit, used to calculate the correlation coefficient between the eye movement data and the target object trajectory;

[0037] a de-spike processing unit, which is activated to remove extreme peaks when a correlation threshold is met;

[0038] The new data generating and storing unit is used to generate and replace the original eye movement data.

[0039] The present invention also provides a readable storage medium having instructions stored thereon, which, when executed, enables a VR device to execute the post-processing correction method of the VR eye tracking data.

[0040] The present invention also provides a miniaturized eye tracker, which is built into a VR head display device and is used to implement the post-processing correction method of the VR eye tracking data.

[0041] The technical solution of the present invention has the following beneficial effects:

[0042] 1. Improve accuracy: The secondary correction technology can effectively improve the accuracy of VR eye tracking data coordinate positioning. On the basis of the existing eye movement calibration, the correlation between the target object's motion trajectory and the gaze point trajectory is further used to estimate the linear regression model coefficients, and all eye movement data are linearly transformed and corrected, thereby reducing the eye movement data errors caused by insufficient initial calibration or other factors.

[0043] 2. Dynamic scene adaptability: The present invention pays special attention to and utilizes dynamic scene information in VR environments. By analyzing and correcting the gaze tracking data of users when following moving objects, the performance of the eye tracking system in complex and dynamic scenes is enhanced. This is particularly suitable for research and application fields that rely on accurate eye movement capture.

[0044] 3. Efficient and accurate data processing flow: This method includes a complete and sophisticated data processing flow, starting from eye movement calibration, through data collection, cleaning, segmentation, correlation calculation, de-spike processing, and then to multivariate linear regression correction, and finally generating new and more accurate eye movement data. This series of steps ensures that the quality of the original eye movement data is greatly improved, providing reliable basic data support for subsequent research or practical applications.

[0045] 4. Strong practicability: It provides specific steps from obtaining raw eye movement data to generating corrected data, including the elimination of invalid data, data segmentation based on scene changes, and parameter estimation based on the least squares method. This makes the present invention not only innovative in theory, but also highly feasible and practical in actual operation.

[0046] 5. The present invention realizes effective correction of eye tracking data by adopting advanced eye movement data processing technology and combining the unique moving object information in the VR environment, which greatly improves the overall performance of the system and the credibility of data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the paradigm of eye movement coordinate correction in the prior art;

[0048] Figure 2 is a logical schematic diagram of the present invention;

[0049] Figure 3 This is a schematic diagram of the original eye movement data;

[0050] Figure 4 is a schematic diagram of eye movement data corrected by the method of the present invention;

[0051] Figure 5 is a system block diagram of the present invention;

[0052] Figure 6 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0053] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments; it can be understood that the specific embodiments described herein are only used to explain the present invention rather than to limit the present invention; it should also be noted that, for ease of description, only the parts related to the present invention rather than all are shown in the accompanying drawings.

[0054] Combination Figure 1-Figure 6 As shown, the present invention provides a post-processing correction method for VR eye tracking data, the method comprising the following steps:

[0055] S1, perform eye movement calibration to ensure that the eye tracker can accurately capture and record eye movements;

[0056] S2, collect eye movement data after successful eye movement calibration;

[0057] S3, cleaning of eye movement data;

[0058] S4, performing post-processing of eye movement data correction to generate new eye movement data;

[0059] S5, updating and storing the new eye movement data in the VR system.

[0060] In step S1, the eye movement calibration adopts the gaze tracking of a moving target in a specific screen scene part of the VR head display device, and uses the overlap between the movement trajectory of the moving target and the gaze point trajectory to correct the coordinate positioning of the eye movement data.

[0061] In step S2, the eye movement data includes a timestamp, horizontal and vertical coordinates of the eye gaze point, and left and right eye status information.

[0062] In step S3, the eye movement data is cleaned by removing invalid numbers or garbled information including NaN and inf, and deleting the data at the time when both eyes are closed.

[0063] In step S4, the eye movement data is corrected and processed, specifically including the following steps:

[0064] S401, segmenting the eye movement data and taking the gaze following part;

[0065] S402, calculating the correlation coefficient between the gaze tracking trajectory of the eye movement data and the movement trajectory of the target object;

[0066] S403, determining whether the correlation coefficient is greater than 0.9, if the correlation coefficient is greater than 0.9, performing a de-peaking process to remove extreme values, otherwise no processing is performed;

[0067] S404, after de-peaking, the least squares method is used to calculate the estimated values ​​of the coefficient matrices A and B of the linear regression model (Y=X*A+B) ;

[0068] S405, using the linear model ( ), predict the user’s actual gaze point at each moment and generate new corrected eye movement data.

[0069] The present invention also provides a post-processing correction system for VR eye tracking data, the system comprising:

[0070] Eye movement calibration module, used for initial eye movement calibration;

[0071] An eye movement data collection unit, used for collecting eye movement data;

[0072] Data preprocessing unit, used to clean eye movement data;

[0073] The core data processing module is used for post-processing of eye movement data correction;

[0074] The core data processing module includes: a data segmentation unit, a correlation analysis unit, a peak removal processing unit, and a new data generation and storage unit;

[0075] A data segmentation unit, used for identifying and extracting data of the gaze-following part;

[0076] A correlation analysis unit, used to calculate the correlation coefficient between the eye movement data and the target object trajectory;

[0077] a de-spike processing unit, which is activated to remove extreme peaks when a correlation threshold is met;

[0078] The new data generating and storing unit is used to generate and replace the original eye movement data.

[0079] The present invention also provides a readable storage medium having instructions stored thereon, which, when executed, enables a VR device to execute the post-processing correction method of the VR eye tracking data.

[0080] The present invention also provides a miniaturized eye tracker, which is built into a VR head display device and is used to implement the post-processing correction method of the VR eye tracking data.

[0081] The specific embodiments are as follows:

[0082] The eye tracking of moving objects in a specific part of the VR head display device is used to correct the coordinate positioning of the eye movement data. The target object motion trajectory (Y) and the gaze point trajectory (X) are used to estimate the coefficient matrices A and B of the linear regression model (Y=X*A+B). Then, the linear regression model constructed with the estimated coefficient matrix is ​​applied to all eye movement data to achieve the purpose of secondary correction of the eye movement data:

[0083] Figure 2 , is a flow chart of the technical solution of the present invention, the contents are as follows:

[0084] Start: The process starts from the Start node.

[0085] Eye movement calibration: First, eye movement calibration is performed to ensure that the eye tracker can accurately capture and record eye movements.

[0086] Eye tracking data collection: After calibration is completed, start collecting eye tracking data.

[0087] Eye tracking data cleaning: The collected data needs to be cleaned to remove noise and inaccurate parts.

[0088] Post-processing of eye movement data correction: The cleaned data will enter the post-processing stage of eye movement data correction, which is completed through the core data processing module:

[0089] Eye movement data segmentation (gaze following part): Split the eye movement data into gaze following parts for easier analysis.

[0090] Calculate the correlation between eye movement data and the trajectory of the observed object: Calculate the correlation between the eye movement data and the trajectory of the observed object.

[0091] Determine whether the correlation coefficient is greater than 0.9: If the correlation coefficient is greater than 0.9, proceed to the next step; otherwise skip.

[0092] Assuming that we have two trajectories X and Y corresponding to the numerical sequence x1, x2, ..., xn and y1, y2, ..., yn, then the calculation formula of the correlation coefficient r is as follows:

[0093]

[0094] in:

[0095] and are the average values ​​of series X and Y respectively;

[0096] and is the difference between each observation and its respective series mean.

[0097] Despike the eye movement data: If the correlation coefficient is greater than 0.9, the eye movement data will be despike.

[0098] Multivariate linear regression estimates the coefficient matrices A and B of the linear transformation (Y=X*A+B): Linear transformation correction is performed on the eye movement data.

[0099] Apply linear transformation correction to all eye movement data: Apply linear transformation correction to all eye movement data.

[0100] Generate new eye tracking data: Generate new eye tracking data.

[0101] Update eye movement data: Update the corrected data.

[0102] End: The process ends at the "End" node.

[0103] This flowchart focuses on the processing of eye movement data, including data collection, cleaning, correction, and generation of new data. These steps help improve the accuracy of eye movement data analysis and provide reliable basic data for subsequent research or applications.

[0104] The eye movement data can be corrected quickly and accurately to make up for the shortcomings of poor calibration before data collection. Moreover, the accuracy of eye tracking can be increased after the gaze following correction of the moving object.

[0105] Complete workflow (generating corrected eye tracking data):

[0106] (1) Obtaining eye movement data

[0107] After the initial eye movement calibration through the built-in micro eye tracker of the VR glasses, eye tracking data begins to be collected. The data is phenotypic data. The first column is the timestamp, accurate to milliseconds, and the sampling frequency is about 70Hz. The second and third columns are VR scene status information. The fourth column is the horizontal coordinate of the eye gaze point, the fifth column is the vertical coordinate, and the sixth and seventh columns are the closure status of the left and right eyes (open or closed).

[0108] (2) Data preprocessing

[0109] Data cleaning mainly includes removing invalid numbers such as NaN, inf or other garbled information, deleting the data at the time when both eyes are closed, and retaining the data when one eye is closed.

[0110] (3) Data segmentation

[0111] According to different screen scenes in VR, the data is segmented to determine the start time of the screen containing the only moving object in the scene. The eye movement data is intercepted according to the start time, and the trajectory data (time and coordinates) of the designed moving object is obtained at the same time.

[0112] (4) Parameter estimation.

[0113] The motion trajectory (Y) and gaze point trajectory (X) of the moving target object are used to calculate the estimated values ​​of the coefficient matrices A and B of the linear regression model (Y=X*A+B) using the least squares method. ;

[0114] (5) Eye tracking data correction;

[0115] The linear model fitted by multiple linear regression method ( ), predict the user’s actual gaze point at each moment, and generate new corrected eye tracking data.

[0116] The above description is only a preferred embodiment of the present invention, and does not limit the scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the protection scope of the present invention.

Claims

1. A post-processing correction method for VR eye tracking data, characterized in that: The method comprises the following steps: S1, perform eye movement calibration to ensure that the eye tracker can accurately capture and record eye movements; S2, collect eye movement data after successful eye movement calibration; S3, cleaning of eye movement data; S4, performing post-processing of eye movement data correction to generate new eye movement data; S5, updating and storing the new eye movement data in the VR system; In step S1, the eye movement calibration uses the gaze tracking of a moving target in a specific screen scene part of the VR head display device, and uses the overlap between the moving trajectory of the moving target and the gaze point trajectory to correct the coordinate positioning of the eye movement data; In step S2, the eye movement data includes a timestamp, horizontal and vertical coordinates of the eye gaze point, and left and right eye status information; In step S3, the eye movement data is cleaned by removing invalid numbers or garbled information containing NaN and inf, and deleting the data at the time when both eyes are closed; In step S4, the eye movement data is corrected and processed, specifically including the following steps: S401, segmenting the eye movement data and taking the gaze following part; S402, calculating the correlation coefficient between the gaze tracking trajectory of the eye movement data and the movement trajectory of the target object; S403, determining whether the correlation coefficient is greater than 0.9, if the correlation coefficient is greater than 0.9, performing a de-peaking process to remove extreme values, otherwise no processing is performed; S404, after de-peaking, the least squares method is used to calculate the estimated values ​​of the coefficient matrices A and B of the linear regression model (Y=X*A+B) ; S405, using the linear model ( ), predict the user’s actual gaze point at each moment and generate new corrected eye movement data.

2. A post-processing correction system for VR eye tracking data, used to implement the correction method of claim 1, characterized in that: The system includes: Eye movement calibration module, used for initial eye movement calibration; An eye movement data collection unit, used for collecting eye movement data; Data preprocessing unit, used to clean eye movement data; The core data processing module is used for post-processing of eye movement data correction; The core data processing module includes: a data segmentation unit, a correlation analysis unit, a peak removal processing unit, and a new data generation and storage unit; A data segmentation unit, used for identifying and extracting data of the gaze-following part; A correlation analysis unit, used to calculate the correlation coefficient between the eye movement data and the target object trajectory; a de-spike processing unit, which is activated to remove extreme peaks when a correlation threshold is met; The new data generating and storing unit is used to generate and replace the original eye movement data.

3. A readable storage medium having instructions stored thereon, which, when executed, causes a VR device to execute the post-processing correction method for VR eye tracking data as described in claim 1.

4. A miniaturized eye tracker, built into a VR head display device, used to implement the post-processing correction method of VR eye tracking data as described in claim 1.

Images