An eye movement precision tracking and recognition system and method based on artificial intelligence

Abstract

The application discloses an eye movement precision tracking and recognition system and method based on artificial intelligence; an intelligent interactive tracking extension module integrates and encapsulates an HTML instruction file set of a multi-source database, constructs a video eye movement tracking data collection platform, and intelligently extends eye movement tracking information and high-precision eye movement tracking; an eye movement tracking and calibration module controls eye movement tracking and triggers calibration, controls a camera access, eye movement key point recognition, gaze point prediction calibration, and precision testing; a video synchronization and eye movement analysis module constructs an immersive video playing environment, associates eye movement data and a video timestamp, and identifies and counts gaze point saccade behaviors; a data visualization and cloud storage module generates and renders a heat map, generates a composite screen snapshot, generates eye movement structured data, and stores the composite screen snapshot in the cloud, monitors a library loading state, a database connection state, eye movement data uploading results, and tracking data collection process reliability.

Description

Technical Field

[0001] This invention relates to the field of intelligent machine learning precision tracking and monitoring systems, and more specifically, to an eye-tracking precision tracking and recognition system and method based on artificial intelligence. Background Technology

[0002] Currently, with the development of computer vision and human-computer interaction technologies, eye-tracking technology has been widely applied in research, experience testing, and evaluation of promotion effects. However, current eye-tracking systems suffer from the following technical shortcomings: High deployment complexity: Most systems rely on dedicated hardware or complex server-side architectures, requiring environment configuration and service deployment, resulting in poor portability and an inability to quickly adapt to different terminals (such as mobile phones, PCs, and laptops) or experimental scenarios; Insufficient data association accuracy: Existing systems struggle to achieve precise time synchronization between eye-tracking data and video playback, failing to establish a one-to-one correspondence between video content frames and user gaze points, thus hindering in-depth correlation analysis between video content and user attention; Limited gaze point prediction accuracy: Some lightweight eye-tracking solutions lack effective calibration and optimization mechanisms, relying solely on basic computer vision algorithms to identify facial key points, making them susceptible to changes in ambient light and user posture. High noise levels in fixation prediction make data quality difficult to guarantee; data storage and visualization are disconnected: most systems only store raw eye-tracking coordinate data, lacking real-time visualization (such as heatmaps), or the visualization results are not stored in conjunction with the raw data, making it impossible to combine intuitive visual distribution with structured data for comprehensive judgment during subsequent analysis; high development and debugging barriers: for researchers or developers, existing systems lack transparent debugging mechanisms and data verification tools, making it difficult to monitor the reliability of the data acquisition process and increasing the difficulty of investigating experimental errors; to address the above problems, how to simplify deployment, improve data accuracy and synchronization, integrate visualization and storage, and facilitate debugging of eye-tracking systems to meet the needs of lightweight and highly adaptable user attention analysis remains to be solved; therefore, it is necessary to propose an artificial intelligence-based eye-tracking precision tracking and recognition system and method to at least partially solve the problems existing in the current technology. Summary of the Invention

[0003] The summary of this invention introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. The summary of this invention does not mean that it attempts to limit the key features and essential technical features of the claimed technical solution, nor does it mean that it attempts to determine the scope of protection of the claimed technical solution.

[0004] To at least partially solve the above problems, the present invention provides an artificial intelligence-based eye-tracking precision recognition system, comprising:

[0005] The intelligent interactive tracking extension module integrates and encapsulates multi-source databases into an HTML instruction file set to build a video eye-tracking data collection platform, enabling intelligent interactive extension of eye-tracking information and high-precision eye-tracking.

[0006] The eye tracking and calibration module controls eye tracking and triggers calibration. It performs eye tracking accuracy calibration tests by controlling camera access, eye key point recognition, gaze point prediction calibration, and accuracy testing.

[0007] The video synchronization and eye-tracking analysis module constructs an immersive video playback environment, controls full-screen video playback via the Video tag, associates eye-tracking data with video timestamps, and performs gaze point saccade behavior recognition and statistics.

[0008] The data visualization and cloud storage module generates and renders heatmaps, generates composite screen snapshots, generates eye-tracking structured data, and stores composite screen snapshots in the cloud. It monitors the library loading status, database connection status, and eye-tracking data upload results, and tracks the reliability of the data acquisition process.

[0009] Preferably, the intelligent interactive tracking extension module includes:

[0010] The video eye-tracking data collection platform integrates an eye-tracking JavaScript library, a heatmap visualization JavaScript library, and a cloud-based NoSQL document database into a single HTML file, forming an integrated HTML instruction file set to build the video eye-tracking data collection platform. Through the video eye-tracking data collection platform, artificial intelligence interaction is used to expand eye-tracking information, and machine learning is used to intelligently update the eye-tracking control process for high-precision eye-tracking.

[0011] The eye-tracking library fault-tolerant loading unit dynamically loads the eye-tracking JavaScript library from the CDN source by listening to the onload event, based on the integrated encapsulated HTML instruction file set.

[0012] The cloud database connection unit synchronously initializes the connection with the cloud NoSQL document database and verifies database access permissions.

[0013] The user session identifier generation unit verifies database access permission settings and generates a user information overlay; the user information overlay includes: UserId and SessionId; and constructs basic eye-tracking interface control buttons.

[0014] By utilizing a video eye-tracking data collection platform, and leveraging artificial intelligence (AI) interaction to expand eye-tracking information, and machine learning to intelligently update the eye-tracking control process, high-precision eye-tracking is achieved. This includes: collecting video information and eye-tracking patterns during video playback through the platform; statistically analyzing the location of visual attention points; expanding the attention distribution range and information through AI interaction; and intelligently updating the eye-tracking control process through machine learning. This enables high-precision eye-tracking, accurate analysis of visual attention patterns in digital products and applications, real-time monitoring of attention distribution and intensity of displayed content, optimization of information delivery strategies, evaluation of creative learning effects, and quantitative analysis of precise recommendation mechanisms. Other aspects include visual attention trajectory tracking; virtual reality experience optimization; and the development of eye-tracking accessibility technologies for attention deficit disorders and autism.

[0015] Preferably, the eye-tracking and calibration module includes:

[0016] The camera access permission request unit calls the eye-tracking JavaScript library API to request webcam access permission from the user and starts the webcam.

[0017] The camera control unit controls the webcam to acquire real-time eye-tracking video streams.

[0018] The nine-point calibration execution unit displays the preset calibration points sequentially according to the set flash duration, collects eye movement key point data and screen click coordinates when clicking, and inputs them into the ridge regression model for parameter training and optimization to improve the accuracy of fixation point prediction.

[0019] The nine preset calibration points include: four corners of the screen, the midpoints of the four sides, and the center. The quantization accuracy of the predicted gaze point is ≥95% if the deviation is ≤5px, which serves as a basis for data quality assessment.

[0020] Preferably, the video synchronization and eye-tracking analysis module includes:

[0021] The immersive video playback unit automatically invokes the full-screen playback command through the Video tag of the interface video player to enter full-screen mode, thus creating an immersive video playback environment;

[0022] The real-time eye-tracking data acquisition unit continuously receives screen gaze coordinates output by the eye-tracking JS library by registering a high-frequency callback function; it applies Kalman filtering to the screen gaze coordinate data, sets the process noise covariance and measurement noise covariance, smooths the data and reduces environmental noise interference.

[0023] The data timestamp association unit listens for video time update events, records video playback timestamps according to the set video recording cycle, and binds the video playback timestamps with the coordinates of the current gaze point to form an eye-tracking basic data set.

[0024] The dynamic statistics panel update unit performs gaze point saccade behavior recognition statistics, dynamically calculates and displays real-time eye movement data information;

[0025] Registered high-frequency callback function sampling rate ≥ 30Hz; process noise covariance Q = 0.01, measurement noise covariance R = 0.1; screen gaze point coordinates (x, y);

[0026] Real-time eye movement data includes: total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime. The criteria for determining the number of saccades are: adjacent fixation point displacement > 10px and time interval < 100ms; the criteria for determining the number of fixations are: dwell time in the same area > 200ms and displacement < 5px. The system dynamically calculates and displays the real-time eye movement data, including: based on the criteria for determining the number of saccades and fixations, when adjacent fixation point displacement > 10px and time interval < 100ms, and dwell time in the same area > 200ms and displacement < 5px, calculating the total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime; and displaying the total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime.

[0027] Preferably, the data visualization and cloud storage module includes:

[0028] The real-time heatmap generation unit calls the heatmap visualization JavaScript library based on real-time eye-tracking data to accumulate all fixation points and form a fixation point density distribution based on screen coordinates.

[0029] The composite screen snapshot capture unit automatically triggers snapshot generation, captures composite screen snapshot information of the video playback area, cumulative heatmap, user information overlay and eye movement detection box, associates it with the corresponding gaze SessionId, generates an HTTP read / write snapshot URL, and uploads it to cloud storage.

[0030] Automatically structured eye-tracking data units automatically structure real-time eye-tracking data to form structured eye-tracking data, and automatically upload the structured eye-tracking data to a cloud-based NoSQL document database;

[0031] The eye-tracking debugging and monitoring unit tracks the eye-tracking data test and calibration execution status, test data upload status, heatmap update status, monitoring library loading status, database connection status, eye-tracking data upload results, and tracking the reliability of the data acquisition process.

[0032] Based on real-time eye-tracking data, a heatmap visualization JavaScript library is invoked. Using screen coordinates as a base, all fixation points are accumulated to form a fixation point density distribution. This includes: setting a color gradient for the fixation point density distribution; setting the color gradient for the fixation point density distribution as follows: low density distribution in blue, medium density distribution in yellow, and high density distribution in red; accumulating all fixation points based on screen coordinates to form a fixation point density distribution; constructing a heatmap visualization JavaScript library based on the fixation point density distribution; and invoking the heatmap visualization JavaScript library to render the heatmap visualization JavaScript library in real-time to the underlying interface based on the color gradient of the fixation point density distribution, without obscuring the video or operation buttons, thus displaying the distribution of user attention.

[0033] This invention provides an artificial intelligence-based eye-tracking precision recognition method, comprising:

[0034] P100 integrates and encapsulates multi-source databases into HTML instruction file sets to build a video eye-tracking data collection platform, and intelligently interacts to expand eye-tracking information and high-precision eye-tracking.

[0035] The P200 controls eye tracking and triggers calibration. It performs eye tracking accuracy calibration tests by controlling camera access, eye key point recognition, gaze point prediction calibration, and accuracy testing.

[0036] P300 constructs an immersive video playback environment, controls full-screen video playback through the Video tag, associates eye-tracking data with video timestamps, and performs gaze point saccade behavior recognition and statistics.

[0037] The P400 performs heatmap generation and rendering, composite screen snapshot generation, eye-tracking structured data generation, and composite screen snapshot cloud storage. It monitors the library loading status, database connection status, and eye-tracking data upload results, and tracks the reliability of the data acquisition process.

[0038] Preferably, P100 includes:

[0039] P101 integrates an eye-tracking JavaScript library, a heatmap visualization JavaScript library, and a cloud-based NoSQL document database into a single HTML file, forming an integrated encapsulated HTML instruction file set to build a video eye-tracking data collection platform. Through the video eye-tracking data collection platform, artificial intelligence interaction is used to expand eye-tracking information, and machine learning is used to intelligently update the eye-tracking control process for high-precision eye-tracking.

[0040] P102, based on the integrated encapsulated HTML instruction file set, implements a fault tolerance mechanism by listening to the onload event, and dynamically loads the eye-tracking JavaScript library from the CDN source;

[0041] P103, synchronously initialize the connection with the cloud-based NoSQL document database and verify database access permissions;

[0042] P104, verify database access permission settings, generate user information overlay; user information overlay includes: UserId and SessionId; construct basic eye-tracking interface control buttons;

[0043] The system integrates and encapsulates an HTML instruction file set, utilizes a Video tag-based full-screen control library for video management and speed control, and encapsulates the `enterFullscreen()` function to trigger full-screen mode via `videoElement.requestFullscreen()`, supporting non-integer speed adjustment. It also synchronizes timestamps by obtaining video playback timestamps at synchronization intervals, accurate to less than the video frame switching frequency, and binds them to global variables for use by the eye-tracking data collection group. The eye-tracking data collection group collects data based on the Video tag-based full-screen control library, and uses a callback command module to obtain eye-tracking coordinates (x, y) in real-time. The video playback timestamp data field is combined with the eye-tracking data to form a complete data unit, and data read / write commands are used to read and write Gaze data storage. This completes the construction of a video eye-tracking data collection platform.

[0044] By utilizing a video eye-tracking data collection platform, and leveraging artificial intelligence (AI) interaction to expand eye-tracking information, and machine learning to intelligently update the eye-tracking control process, high-precision eye-tracking is achieved. This includes: collecting video information and eye-tracking patterns during video playback through the platform; statistically analyzing the location of visual attention points; expanding the attention distribution range and information through AI interaction; and intelligently updating the eye-tracking control process through machine learning. This enables high-precision eye-tracking, accurate analysis of visual attention patterns in digital products and applications, real-time monitoring of attention distribution and intensity of displayed content, optimization of information delivery strategies, evaluation of creative learning effects, and quantitative analysis of precise recommendation mechanisms. Other aspects include visual attention trajectory tracking; virtual reality experience optimization; and the development of eye-tracking accessibility technologies for attention deficit disorders and autism.

[0045] Preferably, P200 includes:

[0046] P201 calls the eye-tracking JavaScript library API to request webcam access from the user and starts the webcam.

[0047] P202 controls the webcam to acquire real-time eye-tracking video streams;

[0048] P203, through preset calibration points, displays sequentially according to the set flash duration, collects eye movement key point data and screen click coordinates when clicking, inputs them into the ridge regression model for parameter training and optimization, and improves the accuracy of fixation point prediction;

[0049] The nine preset calibration points include: four corners of the screen, the midpoints of the four sides, and the center. The quantization accuracy of the predicted gaze point is ≥95% if the deviation is ≤5px, which serves as a basis for data quality assessment.

[0050] Preferably, P300 includes:

[0051] The P301 automatically invokes the full-screen playback command through the Video tab of the interface video player to enter full-screen mode, creating an immersive video playback environment;

[0052] P302 continuously receives screen gaze coordinates from the eye-tracking JS library by registering a high-frequency callback function; it applies Kalman filtering to the screen gaze coordinate data, sets the process noise covariance and measurement noise covariance, smooths the data and reduces environmental noise interference.

[0053] P303 listens for video time update events, records video playback timestamps according to the set video recording cycle, and binds the video playback timestamps with the coordinates of the current gaze point to form an eye-tracking basic data set;

[0054] P304 performs gaze point saccade behavior recognition and statistics, dynamically calculates and displays real-time eye movement data information;

[0055] Real-time eye movement data includes: total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime. The criteria for determining the number of saccades are: adjacent fixation point displacement > 10px and time interval < 100ms; the criteria for determining the number of fixations are: dwell time in the same area > 200ms and displacement < 5px. The system dynamically calculates and displays the real-time eye movement data, including: based on the criteria for determining the number of saccades and fixations, when adjacent fixation point displacement > 10px and time interval < 100ms, and dwell time in the same area > 200ms and displacement < 5px, calculating the total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime; and displaying the total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime.

[0056] Preferably, P400 includes:

[0057] P401, based on real-time eye-tracking data, calls a JavaScript library for heatmap visualization, and accumulates all fixation points based on screen coordinates to form a fixation point density distribution;

[0058] P402 automatically triggers snapshot generation, capturing composite screen snapshot information of the video playback area, cumulative heatmap, user information overlay, and eye-tracking detection box, and associates it with the corresponding gaze SessionId, generates an HTTP read / write snapshot URL, and uploads it to cloud storage;

[0059] P403 automatically structures real-time eye movement data, forming structured eye movement data, and automatically uploads the structured eye movement data to a cloud-based NoSQL document database.

[0060] P404, the interface tracks the eye-tracking data test calibration execution status, test data upload status, heatmap update status, monitors the library loading status, database connection status, eye-tracking data upload results, and tracks the reliability of the data acquisition process;

[0061] Based on real-time eye-tracking data, a heatmap visualization JavaScript library is invoked. Using screen coordinates as a base, all fixation points are accumulated to form a fixation point density distribution. This includes: setting a color gradient for the fixation point density distribution; setting the color gradient for the fixation point density distribution as follows: low density distribution in blue, medium density distribution in yellow, and high density distribution in red; accumulating all fixation points based on screen coordinates to form a fixation point density distribution; constructing a heatmap visualization JavaScript library based on the fixation point density distribution; and invoking the heatmap visualization JavaScript library to render the heatmap visualization JavaScript library in real-time to the underlying interface based on the color gradient of the fixation point density distribution, without obscuring the video or operation buttons, thus displaying the distribution of user attention.

[0062] Compared with the prior art, the present invention has at least the following beneficial effects:

[0063] This invention provides an artificial intelligence-based eye-tracking precision recognition system and method, which has profound application significance and great technological value. Achieving high-precision eye tracking via web pages brings innovative solutions to multiple fields: In user experience research and interface optimization, this technology can accurately analyze users' visual attention patterns on web pages, applications, or digital products, helping designers and product managers optimize interface layouts, improve user interaction experiences, and provide scientific data support for the iterative improvement of internet products; In digital marketing and advertising effectiveness evaluation, it can monitor users' attention distribution and viewing behavior on advertising content in real time, providing quantitative basis for optimizing advertising strategies, evaluating creative effectiveness, and precision marketing. In educational technology and online learning, this technology can track students' visual attention trajectories while watching instructional videos, identify learning priorities and difficulties, and provide objective data for personalized teaching content recommendations and learning outcome assessments. In the fields of medical and health information and cognitive assessment, it can be used for early screening of neurodevelopmental disorders such as attention deficit hyperactivity disorder and autism spectrum disorder, providing auxiliary tools for clinical diagnosis and rehabilitation treatment. In addition, this technology also shows broad prospects in market research and consumer behavior analysis, film and entertainment content testing, virtual reality experience optimization, and accessibility technology development. In particular, its web-based deployment mode and real-time data processing capabilities greatly reduce the barriers to entry and costs of eye-tracking technology, enabling SMEs and research institutions to easily obtain professional-grade eye-tracking analysis capabilities, significantly promoting the popularization and industrial application of human-computer interaction technology; it has significant meaning and remarkable effects.

[0064] The present invention provides an artificial intelligence-based eye-tracking precision recognition system and method. Other advantages, objectives and features of the present invention will be partly apparent from the following description, and partly understood by those skilled in the art through research and practice of the present invention. Attached Figure Description

[0065] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0066] Figure 1 This is a diagram of an embodiment of an artificial intelligence-based eye-tracking precision recognition system architecture described in this invention.

[0067] Figure 2 This is an embodiment of an artificial intelligence-based eye-tracking precision recognition method according to the present invention. Detailed Implementation

[0068] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments, so that those skilled in the art can implement it based on the specification; for example Figure 1 As shown, the present invention provides an artificial intelligence-based eye-tracking precision recognition system, comprising:

[0069] The intelligent interactive tracking extension module integrates and encapsulates multi-source databases into an HTML instruction file set to build a video eye-tracking data collection platform, enabling intelligent interactive extension of eye-tracking information and high-precision eye-tracking.

[0070] The eye tracking and calibration module controls eye tracking and triggers calibration. It performs eye tracking accuracy calibration tests by controlling camera access, eye key point recognition, gaze point prediction calibration, and accuracy testing.

[0071] The video synchronization and eye-tracking analysis module constructs an immersive video playback environment, controls full-screen video playback via the Video tag, associates eye-tracking data with video timestamps, and performs gaze point saccade behavior recognition and statistics.

[0072] The data visualization and cloud storage module generates and renders heatmaps, generates composite screen snapshots, generates eye-tracking structured data, and stores composite screen snapshots in the cloud. It monitors the library loading status, database connection status, and eye-tracking data upload results, and tracks the reliability of the data acquisition process.

[0073] One embodiment of the intelligent interactive tracking extension module includes:

[0074] The video eye-tracking data collection platform integrates an eye-tracking JavaScript library, a heatmap visualization JavaScript library, and a cloud-based NoSQL document database into a single HTML file, forming an integrated HTML instruction file set to build the video eye-tracking data collection platform. Through the video eye-tracking data collection platform, artificial intelligence interaction is used to expand eye-tracking information, and machine learning is used to intelligently update the eye-tracking control process for high-precision eye-tracking.

[0075] The eye-tracking library fault-tolerant loading unit dynamically loads the eye-tracking JavaScript library from the CDN source by listening to the onload event, based on the integrated encapsulated HTML instruction file set.

[0076] The cloud database connection unit synchronously initializes the connection with the cloud NoSQL document database and verifies database access permissions.

[0077] The user session identifier generation unit verifies database access permission settings and generates a user information overlay; the user information overlay includes: UserId and SessionId; and constructs basic eye-tracking interface control buttons.

[0078] Based on the integrated encapsulated HTML instruction file set, a fault tolerance mechanism is implemented by listening to the onload event. The eye-tracking JavaScript library is dynamically loaded from CDN sources, including: dynamically loading the eye-tracking JavaScript library from no less than two public CDN sources based on the integrated encapsulated HTML instruction file set; and implementing a fault tolerance mechanism by listening to the onload event, automatically switching to the next CDN source if the current CDN loading fails, until loading is successful.

[0079] Generating UserId and SessionId includes: UserId is generated from a Unix millisecond-level timestamp and a 32-bit Base64 random string; and SessionId; SessionId identifies a single access session, with the same format as UserId, maintaining the traceability of subsequent data and isolating it from the user.

[0080] By leveraging a video eye-tracking data collection platform and utilizing artificial intelligence (AI) interaction to expand eye-tracking information, and machine learning to intelligently update the eye-tracking control process, high-precision eye-tracking is achieved. This includes: collecting video information and eye-tracking patterns during video playback through the platform; statistically analyzing the location of visual attention points; expanding the attention distribution range through AI interaction; further expanding eye-tracking information through AI interaction; and intelligently updating the eye-tracking control process through machine learning to achieve high-precision eye-tracking. This allows for precise analysis of the visual attention patterns of digital products in applications, real-time monitoring of the attention distribution and intensity of displayed content; real-time feedback enabling various intelligent interactions, including attention level prompts; statistical analysis of the percentage of gaze points in the video area within a preset attention period (>90% for "Focused," 60%-90% for "Neutral," and <60% for "Distracted"), displayed with different colored icons in the upper right corner of the interface; and expansion of gaze area information, including preset video content labels (e.g., "00:00:10-00:00"). (00:20 is a product close-up, 00:00:21-00:00:30 is a dialogue scene). By matching the current video timestamp, a floating window pops up near the gaze point, displaying the corresponding content label; interactive triggering tracks visual attention trajectory, and when the user's gaze calibration button area (coordinate range: x:100-200px, y:800-900px) exceeds the set interaction trigger duration, a high-precision calibration command is automatically invoked; data visualization interaction is integrated and a heatmap is generated, and the heatmap layer is set as a child element of the video player with a set transparency ratio to avoid obscuring video content; interactive query allows users to click on any area of ​​the heatmap, and a floating window pops up displaying the average gaze time and gaze count statistics for that area, expanding the interpretation dimensions of the tracking data; virtual reality experience optimization and attention deficit resolution are performed, as well as assisting autistic eye-tracking barrier-free interaction; information delivery strategy optimization, creative learning effect evaluation, and quantitative analysis of accurate recommendation mechanisms are also performed.

[0081] One embodiment of the eye-tracking and calibration module includes:

[0082] The camera access permission request unit calls the eye-tracking JavaScript library API to request webcam access permission from the user and starts the webcam.

[0083] The camera control unit controls the webcam to acquire real-time eye-tracking video streams.

[0084] The nine-point calibration execution unit displays the preset calibration points sequentially according to the set flash duration, collects eye movement key point data and screen click coordinates when clicking, and inputs them into the ridge regression model for parameter training and optimization to improve the accuracy of fixation point prediction.

[0085] By using preset calibration points, the system sequentially displays the data according to a set flash duration. It collects eye-tracking keypoint data and screen click coordinates at each click, inputting these into a ridge regression model for parameter training and optimization to increase fixation prediction accuracy. This includes: the interface sequentially displays nine preset calibration points, prompting the user to click on each point after 500ms, with multiple consecutive clicks per point; during this time, the system collects eye-tracking keypoint data and screen click coordinates at each click, inputting these into the ridge regression model for parameter training and optimization to increase fixation prediction accuracy; after calibration, the user is prompted to fixate on the screen center point for a preset fixation duration, calculating the deviation between the predicted fixation point and the screen center point, and outputting the quantified accuracy of the predicted fixation point; the nine preset calibration points include: four corner calibration points, four midpoint calibration points, and a center calibration point; if the deviation value of the predicted fixation point quantification accuracy is ≤5px, the accuracy is ≥95%, serving as a data quality assessment criterion.

[0086] In one embodiment, the video synchronization and eye-tracking analysis module includes:

[0087] The immersive video playback unit automatically invokes the full-screen playback command through the Video tag of the interface video player to enter full-screen mode, thus creating an immersive video playback environment;

[0088] The real-time eye-tracking data acquisition unit continuously receives screen gaze coordinates output by the eye-tracking JS library by registering a high-frequency callback function; it applies Kalman filtering to the screen gaze coordinate data, sets the process noise covariance and measurement noise covariance, smooths the data and reduces environmental noise interference.

[0089] The data timestamp association unit listens for video time update events, records video playback timestamps according to the set video recording cycle, and binds the video playback timestamps with the coordinates of the current gaze point to form an eye-tracking basic data set.

[0090] The dynamic statistics panel update unit performs gaze point saccade behavior recognition statistics, dynamically calculates and displays real-time eye movement data information;

[0091] Registered high-frequency callback function sampling rate ≥ 30Hz; process noise covariance Q = 0.01, measurement noise covariance R = 0.1; screen gaze point coordinates (x, y);

[0092] Listen for video time update events, record video playback timestamps according to the set video recording cycle, and bind the video playback timestamps with the coordinates of the currently acquired gaze point to form a basic data set. This includes: changing the frequency of video time update events based on system load and the average cost of processing video time update events; listening for video time update events, recording the video playback timestamp (VideoTimestamp) every 100ms, and binding it with the coordinates (x,y) of the currently acquired gaze point to form an eye-tracking basic data set of [UserId,SessionId,timestamp,gazeX,gazeY,VideoTimestamp].

[0093] Real-time eye movement data includes: total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime. The criteria for determining the number of saccades are: adjacent fixation point displacement > 10px and time interval < 100ms; the criteria for determining the number of fixations are: dwell time in the same area > 200ms and displacement < 5px. The system dynamically calculates and displays the real-time eye movement data, including: based on the criteria for determining the number of saccades and fixations, when adjacent fixation point displacement > 10px and time interval < 100ms, and dwell time in the same area > 200ms and displacement < 5px, calculating the total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime; and displaying the total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime.

[0094] One embodiment of the data visualization and cloud storage module includes:

[0095] The real-time heatmap generation unit calls the heatmap visualization JavaScript library based on real-time eye-tracking data to accumulate all fixation points and form a fixation point density distribution based on screen coordinates.

[0096] The composite screen snapshot capture unit automatically triggers snapshot generation, captures composite screen snapshot information of the video playback area, cumulative heatmap, user information overlay and eye movement detection box, associates it with the corresponding gaze SessionId, generates an HTTP read / write snapshot URL, and uploads it to cloud storage.

[0097] Automatically structured eye-tracking data units automatically structure real-time eye-tracking data to form structured eye-tracking data, and automatically upload the structured eye-tracking data to a cloud-based NoSQL document database;

[0098] The eye-tracking debugging and monitoring unit tracks the eye-tracking data test and calibration execution status, test data upload status, heatmap update status, monitoring library loading status, database connection status, eye-tracking data upload results, and tracking the reliability of the data acquisition process.

[0099] Based on real-time eye-tracking data, a heatmap visualization JavaScript library is invoked. Using screen coordinates as a base, all fixation points are accumulated to form a fixation point density distribution. This includes: setting a color gradient for the fixation point density distribution; setting the color gradient for the fixation point density distribution as follows: low density distribution in blue, medium density distribution in yellow, and high density distribution in red; accumulating all fixation points based on screen coordinates to form a fixation point density distribution; constructing a heatmap visualization JavaScript library based on the fixation point density distribution; and invoking the heatmap visualization JavaScript library to render the heatmap visualization JavaScript library in real-time to the underlying interface based on the color gradient of the fixation point density distribution, without obscuring the video or operation buttons, thus displaying the distribution of user attention.

[0100] Automatically trigger snapshot generation, capturing composite screen snapshot information including video playback area, cumulative heatmap, user information overlay, and eye-tracking detection box, and associating it with the corresponding gaze SessionId, generating an HTTP read-write snapshot URL, and uploading it to cloud storage. This includes: according to the snapshot triggering set period, such as automatically triggering snapshot generation every 2 seconds, capturing composite screen content containing "video playback area + cumulative heatmap + user information overlay (UserId / SessionId) + eye-tracking detection box" through the html2canvas library, uploading it to cloud storage services (such as FirebaseStorage), generating an HTTP-accessible snapshot URL, and associating it with the corresponding SessionId;

[0101] The system automatically structures real-time eye movement data to form structured eye movement data and automatically uploads the structured eye movement data to a cloud-based NoSQL document database. This includes uploading the structured eye movement data to the cloud-based NoSQL document database in JSON format according to an automatic format conversion cycle. The structured eye movement data includes: raw fixation point data (including VideoTimestamp), fixation behavior data (fixationId, fixation start / end time, fixation area coordinates), saccade behavior data (saccadeId, saccade start / end coordinates, saccade duration), and eye movement data calibration information including calibration point coordinates and accuracy percentage.

[0102] like Figure 2As shown; the present invention provides an artificial intelligence-based eye-tracking precision recognition method, comprising:

[0103] P100 integrates and encapsulates multi-source databases into HTML instruction file sets to build a video eye-tracking data collection platform, and intelligently interacts to expand eye-tracking information and high-precision eye-tracking.

[0104] The P200 controls eye tracking and triggers calibration. It performs eye tracking accuracy calibration tests by controlling camera access, eye key point recognition, gaze point prediction calibration, and accuracy testing.

[0105] P300 constructs an immersive video playback environment, controls full-screen video playback through the Video tag, associates eye-tracking data with video timestamps, and performs gaze point saccade behavior recognition and statistics.

[0106] The P400 performs heatmap generation and rendering, composite screen snapshot generation, eye-tracking structured data generation, and composite screen snapshot cloud storage. It monitors the library loading status, database connection status, and eye-tracking data upload results, and tracks the reliability of the data acquisition process.

[0107] In one embodiment, P100 includes:

[0108] P101 integrates an eye-tracking JavaScript library, a heatmap visualization JavaScript library, and a cloud-based NoSQL document database into a single HTML file, forming an integrated encapsulated HTML instruction file set to build a video eye-tracking data collection platform. Through the video eye-tracking data collection platform, artificial intelligence interaction is used to expand eye-tracking information, and machine learning is used to intelligently update the eye-tracking control process for high-precision eye-tracking.

[0109] P102, based on the integrated encapsulated HTML instruction file set, implements a fault tolerance mechanism by listening to the onload event, and dynamically loads the eye-tracking JavaScript library from the CDN source;

[0110] P103, synchronously initialize the connection with the cloud-based NoSQL document database and verify database access permissions;

[0111] P104, verify database access permission settings, generate user information overlay; user information overlay includes: UserId and SessionId; construct basic eye-tracking interface control buttons;

[0112] Based on the integrated encapsulated HTML instruction file set, a fault tolerance mechanism is implemented by listening to the onload event. The eye-tracking JavaScript library is dynamically loaded from CDN sources, including: dynamically loading the eye-tracking JavaScript library from no less than two public CDN sources based on the integrated encapsulated HTML instruction file set; and implementing a fault tolerance mechanism by listening to the onload event, automatically switching to the next CDN source if the current CDN loading fails, until loading is successful.

[0113] Generating UserId and SessionId includes: UserId is generated from a Unix millisecond-level timestamp and a 32-bit Base64 random string; and SessionId; SessionId identifies a single access session, with the same format as UserId, maintaining the traceability of subsequent data and isolating it from the user.

[0114] By leveraging a video eye-tracking data collection platform and utilizing artificial intelligence (AI) interaction to expand eye-tracking information, and machine learning to intelligently update the eye-tracking control process, high-precision eye-tracking is achieved. This includes: collecting video information and eye-tracking patterns during video playback through the platform; statistically analyzing the location of visual attention points; expanding the attention distribution range through AI interaction; further expanding eye-tracking information through AI interaction; and intelligently updating the eye-tracking control process through machine learning to achieve high-precision eye-tracking. This allows for precise analysis of the visual attention patterns of digital products in applications, real-time monitoring of the attention distribution and intensity of displayed content; real-time feedback enabling various intelligent interactions, including attention level prompts; statistical analysis of the percentage of gaze points in the video area within a preset attention period (>90% for "Focused," 60%-90% for "Neutral," and <60% for "Distracted"), displayed with different colored icons in the upper right corner of the interface; and expansion of gaze area information, including preset video content labels (e.g., "00:00:10-00:00"). (00:20 is a product close-up, 00:00:21-00:00:30 is a dialogue scene). By matching the current video timestamp, a floating window pops up near the gaze point, displaying the corresponding content label; interactive triggering tracks visual attention trajectory, and when the user's gaze calibration button area (coordinate range: x:100-200px, y:800-900px) exceeds the set interaction trigger duration, a high-precision calibration command is automatically invoked; data visualization interaction is integrated and a heatmap is generated, and the heatmap layer is set as a child element of the video player with a set transparency ratio to avoid obscuring video content; interactive query allows users to click on any area of ​​the heatmap, and a floating window pops up displaying the average gaze time and gaze count statistics for that area, expanding the interpretation dimensions of the tracking data; virtual reality experience optimization and attention deficit resolution are performed, as well as assisting autistic eye-tracking barrier-free interaction; information delivery strategy optimization, creative learning effect evaluation, and quantitative analysis of accurate recommendation mechanisms are also performed.

[0115] In one embodiment, P200 includes:

[0116] P201 calls the eye-tracking JavaScript library API to request webcam access from the user and starts the webcam.

[0117] P202 controls the webcam to acquire real-time eye-tracking video streams;

[0118] P203, through preset calibration points, displays sequentially according to the set flash duration, collects eye movement key point data and screen click coordinates when clicking, inputs them into the ridge regression model for parameter training and optimization, and improves the accuracy of fixation point prediction;

[0119] By using preset calibration points, the system sequentially displays the data according to a set flash duration. It collects eye-tracking keypoint data and screen click coordinates at each click, inputting these into a ridge regression model for parameter training and optimization to increase fixation prediction accuracy. This includes: the interface sequentially displays nine preset calibration points, prompting the user to click on each point after 500ms, with multiple consecutive clicks per point; during this time, the system collects eye-tracking keypoint data and screen click coordinates at each click, inputting these into the ridge regression model for parameter training and optimization to increase fixation prediction accuracy; after calibration, the user is prompted to fixate on the screen center point for a preset fixation duration, calculating the deviation between the predicted fixation point and the screen center point, and outputting the quantified accuracy of the predicted fixation point; the nine preset calibration points include: four corner calibration points, four midpoint calibration points, and a center calibration point; if the deviation value of the predicted fixation point quantification accuracy is ≤5px, the accuracy is ≥95%, serving as a data quality assessment criterion.

[0120] In one embodiment, P300 includes:

[0121] The P301 automatically invokes the full-screen playback command through the Video tab of the interface video player to enter full-screen mode, creating an immersive video playback environment;

[0122] P302 continuously receives screen gaze coordinates from the eye-tracking JS library by registering a high-frequency callback function; it applies Kalman filtering to the screen gaze coordinate data, sets the process noise covariance and measurement noise covariance, smooths the data and reduces environmental noise interference.

[0123] P303 listens for video time update events, records video playback timestamps according to the set video recording cycle, and binds the video playback timestamps with the coordinates of the current gaze point to form an eye-tracking basic data set;

[0124] P304 performs gaze point saccade behavior recognition and statistics, dynamically calculates and displays real-time eye movement data information;

[0125] Registered high-frequency callback function sampling rate ≥ 30Hz; process noise covariance Q = 0.01, measurement noise covariance R = 0.1; screen gaze point coordinates (x, y);

[0126] Listen for video time update events, record video playback timestamps according to the set video recording cycle, and bind the video playback timestamps with the coordinates of the currently acquired gaze point to form a basic data set. This includes: changing the frequency of video time update events based on system load and the average cost of processing video time update events; listening for video time update events, recording the video playback timestamp (VideoTimestamp) every 100ms, and binding it with the coordinates (x,y) of the currently acquired gaze point to form an eye-tracking basic data set of [UserId,SessionId,timestamp,gazeX,gazeY,VideoTimestamp].

[0127] Real-time eye movement data includes: total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime. The criteria for determining the number of saccades are: adjacent fixation point displacement > 10px and time interval < 100ms; the criteria for determining the number of fixations are: dwell time in the same area > 200ms and displacement < 5px. The system dynamically calculates and displays the real-time eye movement data, including: based on the criteria for determining the number of saccades and fixations, when adjacent fixation point displacement > 10px and time interval < 100ms, and dwell time in the same area > 200ms and displacement < 5px, calculating the total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime; and displaying the total number of fixations, number of saccades, number of fixations, average fixation time, and system runtime.

[0128] In one embodiment, P400 includes:

[0129] P401, based on real-time eye-tracking data, calls a JavaScript library for heatmap visualization, and accumulates all fixation points based on screen coordinates to form a fixation point density distribution;

[0130] P402 automatically triggers snapshot generation, capturing composite screen snapshot information of the video playback area, cumulative heatmap, user information overlay, and eye-tracking detection box, and associates it with the corresponding gaze SessionId, generates an HTTP read / write snapshot URL, and uploads it to cloud storage;

[0131] P403 automatically structures real-time eye movement data, forming structured eye movement data, and automatically uploads the structured eye movement data to a cloud-based NoSQL document database.

[0132] P404, the interface tracks the eye-tracking data test calibration execution status, test data upload status, heatmap update status, monitors the library loading status, database connection status, eye-tracking data upload results, and tracks the reliability of the data acquisition process;

[0133] Based on real-time eye-tracking data, a heatmap visualization JavaScript library is invoked. Using screen coordinates as a base, all fixation points are accumulated to form a fixation point density distribution. This includes: setting a color gradient for the fixation point density distribution; setting the color gradient for the fixation point density distribution as follows: low density distribution in blue, medium density distribution in yellow, and high density distribution in red; accumulating all fixation points based on screen coordinates to form a fixation point density distribution; constructing a heatmap visualization JavaScript library based on the fixation point density distribution; and invoking the heatmap visualization JavaScript library to render the heatmap visualization JavaScript library in real-time to the underlying interface based on the color gradient of the fixation point density distribution, without obscuring the video or operation buttons, thus displaying the distribution of user attention.

[0134] Automatically trigger snapshot generation, capturing composite screen snapshot information including video playback area, cumulative heatmap, user information overlay, and eye-tracking detection box, and associating it with the corresponding gaze SessionId, generating an HTTP read-write snapshot URL, and uploading it to cloud storage. This includes: according to the snapshot triggering set period, such as automatically triggering snapshot generation every 2 seconds, capturing composite screen content containing "video playback area + cumulative heatmap + user information overlay (UserId / SessionId) + eye-tracking detection box" through the html2canvas library, uploading it to cloud storage services (such as FirebaseStorage), generating an HTTP-accessible snapshot URL, and associating it with the corresponding SessionId;

[0135] The system automatically structures real-time eye movement data to form structured eye movement data and automatically uploads the structured eye movement data to a cloud-based NoSQL document database. This includes uploading the structured eye movement data to the cloud-based NoSQL document database in JSON format according to an automatic format conversion cycle. The structured eye movement data includes: raw fixation point data (including VideoTimestamp), fixation behavior data (fixationId, fixation start / end time, fixation area coordinates), saccade behavior data (saccadeId, saccade start / end coordinates, saccade duration), and eye movement data calibration information including calibration point coordinates and accuracy percentage.

[0136] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. Other modifications can be easily made by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. An artificial intelligence-based eye-tracking precision recognition system, characterized in that, include: The intelligent interactive tracking extension module integrates and encapsulates multi-source databases into an HTML instruction file set to build a video eye-tracking data collection platform, enabling intelligent interactive extension of eye-tracking information and high-precision eye-tracking. The eye tracking and calibration module controls eye tracking and triggers calibration. It performs eye tracking accuracy calibration tests by controlling camera access, eye key point recognition, gaze point prediction calibration, and accuracy testing. The video synchronization and eye-tracking analysis module includes: The immersive video playback unit automatically invokes the full-screen playback command through the Video tag of the interface video player to enter full-screen mode, thus creating an immersive video playback environment; The real-time eye-tracking data acquisition unit continuously receives screen gaze coordinates output by the eye-tracking JS library by registering a high-frequency callback function; it applies Kalman filtering to the screen gaze coordinate data, sets the process noise covariance and measurement noise covariance, smooths the data and reduces environmental noise interference. The data timestamp association unit listens for video time update events, records video playback timestamps according to the set video recording cycle, and binds the video playback timestamps with the coordinates of the current gaze point to form an eye-tracking basic data set. The dynamic statistics panel update unit performs gaze point saccade behavior recognition statistics, dynamically calculates and displays real-time eye movement data information; The data visualization and cloud storage module includes: The real-time heatmap generation unit calls the heatmap visualization JavaScript library based on real-time eye-tracking data to accumulate all fixation points and form a fixation point density distribution based on screen coordinates. The composite screen snapshot capture unit automatically triggers snapshot generation, captures composite screen snapshot information of the video playback area, cumulative heatmap, user information overlay and eye movement detection box, associates it with the corresponding gaze SessionId, generates an HTTP read / write snapshot URL, and uploads it to cloud storage. Automatically structured eye-tracking data units automatically structure real-time eye-tracking data to form structured eye-tracking data, and automatically upload the structured eye-tracking data to a cloud-based NoSQL document database; The eye-tracking debugging and monitoring unit tracks the execution status of eye-tracking data testing and calibration, the status of test data upload, the status of heatmap updates, the status of library loading, the status of database connection, and the results of eye-tracking data upload, thus tracking the reliability of the data acquisition process.

2. The eye-tracking precision recognition system based on artificial intelligence according to claim 1, characterized in that, The intelligent interactive tracking extension module includes: The video eye-tracking data collection platform integrates an eye-tracking JavaScript library, a heatmap visualization JavaScript library, and a cloud-based NoSQL document database into a single HTML file, forming an integrated HTML instruction file set to build the video eye-tracking data collection platform. Through the video eye-tracking data collection platform, artificial intelligence interaction is used to expand eye-tracking information, and machine learning is used to intelligently update the eye-tracking control process for high-precision eye-tracking. The eye-tracking library fault-tolerant loading unit dynamically loads the eye-tracking JavaScript library from the CDN source by listening to the onload event, based on the integrated encapsulated HTML instruction file set. The cloud database connection unit synchronously initializes the connection with the cloud NoSQL document database and verifies database access permissions. The user session identifier generation unit verifies database access permission settings and generates a user information overlay; the user information overlay includes: UserId and SessionId; and constructs basic eye-tracking interface control buttons.

3. The eye-tracking precision recognition system based on artificial intelligence according to claim 1, characterized in that, The eye-tracking and calibration module includes: The camera access permission request unit calls the eye-tracking JavaScript library API to request webcam access permission from the user and starts the webcam. The camera control unit controls the webcam to acquire real-time eye-tracking video streams. The nine-point calibration execution unit displays the preset calibration points sequentially according to the set flash duration, collects eye movement key point data and screen click coordinates at the time of click, and inputs them into the ridge regression model for parameter training and optimization to improve the accuracy of fixation point prediction.

4. A precise eye-tracking recognition method based on artificial intelligence, characterized in that, include: P100 integrates and encapsulates multi-source databases into HTML instruction file sets to build a video eye-tracking data collection platform, and intelligently interacts to expand eye-tracking information and high-precision eye-tracking. The P200 controls eye tracking and triggers calibration. It performs eye tracking accuracy calibration tests by controlling camera access, eye key point recognition, gaze point prediction calibration, and accuracy testing. P300 includes: The P301 automatically invokes the full-screen playback command through the Video tab of the interface video player to enter full-screen mode, creating an immersive video playback environment; P302 continuously receives screen gaze coordinates from the eye-tracking JS library by registering a high-frequency callback function; it applies Kalman filtering to the screen gaze coordinate data, sets the process noise covariance and measurement noise covariance, smooths the data and reduces environmental noise interference. P303 listens for video time update events, records video playback timestamps according to the set video recording cycle, and binds the video playback timestamps with the coordinates of the current gaze point to form an eye-tracking basic data set; P304 performs gaze point saccade behavior recognition and statistics, dynamically calculates and displays real-time eye movement data information; P400 includes: P401, based on real-time eye-tracking data, calls a JavaScript library for heatmap visualization, and accumulates all fixation points based on screen coordinates to form a fixation point density distribution; P402 automatically triggers snapshot generation, capturing composite screen snapshot information of the video playback area, cumulative heatmap, user information overlay, and eye-tracking detection box, and associates it with the corresponding gaze SessionId, generates an HTTP read / write snapshot URL, and uploads it to cloud storage; P403 automatically structures real-time eye movement data, forming structured eye movement data, and automatically uploads the structured eye movement data to a cloud-based NoSQL document database. P404, the interface tracks the eye-tracking data test calibration execution status, test data upload status, heatmap update status, monitors the library loading status, database connection status, eye-tracking data upload results, and tracks the reliability of the data acquisition process.

5. The eye-tracking precision recognition method based on artificial intelligence according to claim 4, characterized in that, P100 includes: P101 integrates an eye-tracking JavaScript library, a heatmap visualization JavaScript library, and a cloud-based NoSQL document database into a single HTML file, forming an integrated encapsulated HTML instruction file set to build a video eye-tracking data collection platform. Through the video eye-tracking data collection platform, artificial intelligence interaction is used to expand eye-tracking information, and machine learning is used to intelligently update the eye-tracking control process for high-precision eye-tracking. P102, based on the integrated encapsulated HTML instruction file set, implements a fault tolerance mechanism by listening to the onload event, and dynamically loads the eye-tracking JavaScript library from the CDN source; P103, synchronously initialize the connection with the cloud-based NoSQL document database and verify database access permissions; P104, verify database access permission settings, generate user information overlay; the user information overlay includes: UserId and SessionId; construct basic eye-tracking interface control buttons.

6. The eye-tracking precision recognition method based on artificial intelligence according to claim 4, characterized in that, P200 includes: P201 calls the eye-tracking JavaScript library API to request webcam access from the user and starts the webcam. P202 controls the webcam to acquire real-time eye-tracking video streams; P203, through preset calibration points, displays sequentially according to the set flash duration, collects eye movement key point data and screen click coordinates at the time of click, inputs them into the ridge regression model for parameter training and optimization, and improves the accuracy of fixation point prediction.

Images