Eye movement track recognition method and system based on spatio-temporal feature fusion and storage medium

Abstract

The application discloses an eye movement track recognition method and system based on space-time feature fusion and a storage medium, relates to the technical field of artificial intelligence and man-machine interaction, and comprises the following steps: collecting eye movement track data of a user in a multitask scene, extracting position coordinates and a timestamp sequence from continuous behavior records by using a preset sampling frequency, and obtaining original sequence data containing time sequence and spatial position; the eye movement track recognition method and system based on space-time feature fusion and the storage medium can further refine the jump logic in the change of the eye movement track, and improve the reliability of the eye movement track recognition result and the response accuracy of the intelligent interaction system in a complex environment.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and human-computer interaction technology, specifically to an eye-tracking trajectory recognition method, system, and storage medium based on spatiotemporal feature fusion. Background Technology

[0002] With the development of technologies such as intelligent interaction, virtual reality, assisted driving, and medical diagnosis, eye-tracking data is gradually becoming an important basis for analyzing changes in user attention and behavioral states. Eye-tracking trajectories typically consist of continuously collected position coordinates and timestamp sequences, reflecting the user's gaze movement, changes in focus, and trajectory transition patterns in different task scenarios. Therefore, accurate identification of eye-tracking trajectories helps improve the ability of intelligent interaction systems to understand and respond to changes in user behavior.

[0003] In existing technologies, eye-tracking recognition methods typically focus on analyzing single features such as gaze position, fixation point distribution, dwell time, or trajectory morphology. Some methods determine the user's area of ​​focus through rule matching or statistical calculations. However, in multi-task scenarios, user eye-tracking trajectories often exhibit significant continuity, jumps, and dynamic changes. Relying solely on static spatial position or local trajectory features makes it difficult to accurately depict the evolution of gaze patterns over time. Furthermore, eye-tracking trajectories themselves contain both temporal and spatial dimensions. The temporal sequence influences the meaning of positional changes, and spatial distribution reflects changes in the user's focus. If temporal evolution patterns and spatial distribution characteristics cannot be effectively integrated, it becomes difficult to accurately identify eye-tracking trajectory categories in complex environments. Especially in the presence of environmental noise, multi-target switching, or frequent changes in task status, existing methods are prone to insufficient trajectory feature extraction, inaccurate trajectory category determination, and insufficient stability of recognition results. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and storage medium for eye-tracking trajectory recognition based on spatiotemporal feature fusion, thereby addressing the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an eye-tracking trajectory recognition method based on spatiotemporal feature fusion, comprising the following steps: S1. By collecting eye-tracking trajectory data of users in multi-task scenarios, and using a preset sampling frequency, position coordinates and timestamp sequences are extracted from continuous behavior records to obtain raw sequence data containing time sequence and spatial location. S2. Capture the correlation between chronological order and positional changes from historical data to determine the dynamic change characteristics under the time evolution pattern; S3. If the dynamic change characteristics exceed the preset threshold, analyze the spatial dimension information, extract the distribution pattern of the eye-tracking focus from the location coordinates, and obtain the association representation between spatial location and trajectory behavior logic. S4. Integrate the dynamic change characteristics under the time evolution law and the correlation representation of spatial location and trajectory behavior logic, filter out noise interference from environmental influences, and obtain the comprehensive feature vector after time and space fusion. S5. Obtain reference trajectory patterns in multi-task scenarios, determine the user's trajectory focus at a specific moment, and determine the preliminary eye-tracking trajectory category. S6. If the initial eye movement trajectory category does not match the evolution pattern in historical data, the fusion parameters are adjusted through iterative optimization to refine the jump logic of eye movement trajectory changes from the comprehensive feature vector and obtain accurate eye movement trajectory recognition results. S7. Generate feedback signals and apply them to the intelligent interaction system to update the system response logic from the restored real trajectory changes and judge the overall recognition accuracy.

[0006] On the other hand, the present invention also provides an eye-tracking trajectory recognition system based on spatiotemporal feature fusion, used to implement the steps of the above-described eye-tracking trajectory recognition method based on spatiotemporal feature fusion, including:

[0007] The data acquisition and parsing module collects eye-tracking trajectory data of users in multi-task scenarios, and extracts position coordinates and timestamp sequences from continuous behavior records using a preset sampling frequency to obtain raw sequence data containing time sequence and spatial location.

[0008] The time dimension feature extraction module uses a long short-term memory network to process time dimension information based on the original sequence data, captures the linkage relationship between the sequence and position changes from historical data, and determines the dynamic change features under the time evolution law.

[0009] The spatial dimension feature extraction module analyzes spatial dimension information through a convolutional neural network if the dynamic change features exceed a preset threshold. It extracts the distribution pattern of the eye-tracking focus from the location coordinates to obtain the association representation between spatial location and trajectory behavior logic.

[0010] The spatiotemporal feature fusion module integrates the dynamic change features under the time evolution law and the correlation representation of spatial location and trajectory behavior logic, filters out noise interference from environmental influences, and obtains a comprehensive feature vector after spatiotemporal fusion.

[0011] The trajectory attention point judgment module obtains reference trajectory patterns in multi-task scenarios based on the comprehensive feature vector, and determines the user's trajectory attention point at a specific time by comparing similarity, thus determining the preliminary eye-tracking trajectory category.

[0012] If the initial eye movement trajectory category does not match the evolution pattern in historical data, the iterative optimization and correction module will adjust the fusion parameters through iterative optimization to refine the jump logic of eye movement trajectory changes from the comprehensive feature vector and obtain accurate eye movement trajectory recognition results.

[0013] The feedback verification module generates feedback signals based on accurate eye-tracking trajectory recognition results and applies them to the intelligent interaction system. It updates the system response logic based on the restored real trajectory changes and judges the overall recognition accuracy.

[0014] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the aforementioned eye-tracking trajectory recognition method based on spatiotemporal feature fusion.

[0015] As can be seen from the above technical solution, the present invention has the following beneficial effects:

[0016] This eye-tracking trajectory recognition method, system, and storage medium based on spatiotemporal feature fusion collects eye-tracking trajectory data containing location coordinates and timestamp sequences. It then extracts dynamic change features under temporal evolution patterns using a long short-term memory network, analyzes the distribution patterns of attention points in spatial locations using a convolutional neural network, and integrates temporal and spatial features through a fusion module. This effectively characterizes the continuous change relationship and spatial distribution patterns of eye-tracking trajectories in multi-task scenarios, reduces the interference of environmental noise on trajectory analysis, and improves the accuracy and stability of eye-tracking trajectory category judgment. Furthermore, by comparing reference trajectory patterns, iteratively optimizing fusion parameters, and updating the system response logic with feedback signals, it further refines the jump logic in eye-tracking trajectory changes, improving the reliability of eye-tracking trajectory recognition results in complex environments and the response accuracy of intelligent interaction systems. Attached Figure Description

[0017] Figure 1 This is a signal transmission diagram of the eye movement trajectory recognition method based on spatiotemporal feature fusion according to the present invention;

[0018] Figure 2 This is a structural block diagram of the local terminal of an exemplary electronic device of the present invention;

[0019] Figure 3 This is a structural block diagram of the network terminal of an exemplary electronic device of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1:

[0022] like Figure 1 As shown, the present invention provides a technical solution: an eye-tracking trajectory recognition method based on spatiotemporal feature fusion, comprising:

[0023] S1. By collecting eye-tracking trajectory data of users in multi-task scenarios, and using a preset sampling frequency, position coordinates and timestamp sequences are extracted from continuous behavior records to obtain raw sequence data containing time sequence and spatial location.

[0024] S2. Based on the original sequence data, a long short-term memory network is used to process the time dimension information, capture the linkage relationship between the sequence and position changes from historical data, and determine the dynamic change characteristics under the time evolution law.

[0025] S3. If the dynamic change features exceed the preset threshold, the spatial dimension information is analyzed through a convolutional neural network to extract the distribution pattern of the eye-tracking focus from the position coordinates and obtain the association representation between spatial position and trajectory behavior logic.

[0026] S4. The fusion module integrates the dynamic change characteristics under the time evolution law and the correlation representation of spatial location and trajectory behavior logic, filters out noise interference from environmental influences, and obtains the comprehensive feature vector after time-space fusion.

[0027] S5. Based on the comprehensive feature vector, obtain the reference trajectory pattern in multi-task scenarios, and determine the user's trajectory focus at a specific time by comparing similarity, and determine the preliminary eye-tracking trajectory category.

[0028] S6. If the initial eye movement trajectory category does not match the evolution pattern in historical data, the fusion parameters are adjusted through iterative optimization to refine the jump logic of eye movement trajectory changes from the comprehensive feature vector and obtain accurate eye movement trajectory recognition results.

[0029] S7. Based on the accurate eye-tracking trajectory recognition results, generate feedback signals and apply them to the intelligent interaction system. Update the system response logic from the restored real trajectory changes and judge the overall recognition accuracy.

[0030] In the above embodiments, when a user is engaged in a multi-task scenario involving reading, target searching, interface selection, image browsing, and virtual interaction, the eye-tracking acquisition device continuously collects the user's eye movement trajectory data at a preset sampling frequency. Specifically, the eye-tracking acquisition device employs at least one of the following: an infrared eye tracker, a camera-based eye-tracking acquisition module, a head-mounted eye-tracking sensor, and a visual acquisition component integrated into a smart terminal.

[0031] Furthermore, the acquired data includes location coordinates and timestamps. The location coordinates utilize at least one of the following: planar coordinates in the screen coordinate system, spatial coordinates, and region-of-interest coordinates mapped by the gaze. The system extracts the eye movement position and time record corresponding to each sampling moment from the continuous behavior records in chronological order and arranges them into raw sequence data according to the sampling time. The system performs time alignment, coordinate normalization, missing value imputation, abnormal jump point removal, and smoothing filtering on the raw sequence data to reduce the impact of blinking, device jitter, short-term occlusion, and acquisition errors on the recognition results.

[0032] The Long Short-Term Memory (LSTM) network is used to process the temporal dimension of eye movement trajectories. Eye movements exhibit temporal continuity and sequential dependence; fixation, saccades, retrospection, and dwelling behaviors are all correlated with trajectory changes at preceding and following moments. The system inputs preprocessed raw sequence data into the LTM network using a sliding time window. The sliding time window extracts consecutive sampling points at fixed time lengths, ensuring that each set of input data contains a continuous trajectory change process.

[0033] Furthermore, the Long Short-Term Memory (LSTM) network selectively retains, updates, and outputs historical trajectory information through input, forget, and output gates, thereby capturing the impact of historical position changes on the current trajectory state. After processing by the LTM network, the system obtains dynamic change characteristics under the time evolution pattern. These dynamic change characteristics include changes in eye movement speed, acceleration, direction change trends, dwell time, transition patterns between consecutive coordinate points, and the rhythm of trajectory changes at different task stages.

[0034] In this embodiment, the system compares the dynamic change characteristics output by the Long Short-Term Memory network with a preset threshold. This preset threshold is determined based on statistical results of training samples, historical recognition results, and the requirements of the actual application scenario. When determining the preset threshold, the system first statistically analyzes the dynamic change range corresponding to different eye-tracking behaviors in the training samples, then determines the boundary conditions based on the boundary differences between correctly and incorrectly recognized samples in the historical recognition results, and finally determines the final threshold by combining the real-time and accuracy requirements of the actual application scenario.

[0035] When the dynamic change features do not exceed the preset threshold, the system proceeds to subsequent recognition processing based on the time dimension features; when the dynamic change features exceed the preset threshold, the system determines that the user's eye movement trajectory shows a shift in focus, rapid scanning, task switching, or abnormal jumps, and analyzes the spatial dimension information through a convolutional neural network.

[0036] In this embodiment, the convolutional neural network performs spatial dimension analysis on the position coordinate sequence. The system first maps continuous position coordinates to screen regions, spatial regions, or regions of interest, assigning each trajectory point to a specific spatial unit. Then, it statistically analyzes the distribution of trajectory points within each spatial unit, the trajectory's traversal order, and the duration of dwell time. Subsequently, the spatial distribution results are converted into trajectory heatmaps, spatial grid maps, region of interest distribution maps, or trajectory point matrices. The convolutional neural network performs local scanning on the spatial representation results using convolutional kernels, extracting the gaze point clustering region, trajectory movement direction, spatial distribution density, and the boundary of the region of interest. Thus, the system obtains a logical association between spatial location and trajectory behavior.

[0037] Furthermore, the fusion module integrates the dynamic change features under the time evolution pattern and the correlation representation of spatial location and trajectory behavior logic. The fusion module generates a comprehensive feature vector after temporal and spatial fusion using at least one of the following methods: feature concatenation, weighted fusion, attention fusion, and gating fusion. Specifically, the fusion module first receives the temporal features output by the Long Short-Term Memory network and the spatial features output by the convolutional neural network, and then determines the contribution of the temporal and spatial features to the comprehensive feature vector based on the fusion parameters. When temporal continuity plays a dominant role in the current recognition result, the fusion module increases the contribution of the temporal features; when spatial distribution plays a dominant role in the current recognition result, the fusion module increases the contribution of the spatial features. The fusion module combines a normalization layer, a confidence screening mechanism, an outlier suppression mechanism, and a noise reduction autoencoder structure to suppress non-realistic trajectory changes caused by lighting changes, screen reflections, head movement, blinking occlusion, and sensor jitter, thereby improving the stability of the comprehensive feature vector.

[0038] As a preferred embodiment, the system obtains reference trajectory patterns for multi-task scenarios based on comprehensive feature vectors. The reference trajectory patterns are derived from at least one of the following: pre-labeled training samples, historical user behavior data, a task template library, and typical trajectory category centers formed through clustering. Each reference trajectory pattern corresponds to an eye-tracking behavior category and includes the corresponding gaze area distribution, trajectory transition order, dwell time range, saccade direction, and trajectory change rhythm.

[0039] Furthermore, when performing similarity comparison, the system first aligns the current integrated feature vector with each reference trajectory pattern, then compares the degree of consistency between the two in trajectory direction, dwell area, change rhythm, and spatial transfer order. Subsequently, based on the degree of consistency, the system determines the reference trajectory pattern that is closest to the current trajectory. When the current integrated feature vector and the target reference trajectory pattern meet the preset similarity conditions, the system determines the user's trajectory focus at a specific moment and outputs a preliminary eye-tracking trajectory category.

[0040] In this embodiment, the system further matches the initial eye-tracking trajectory category with the evolution patterns in historical data. These evolution patterns include trajectory category shifts for the same user under the same task, eye-tracking behavior switching patterns between different task stages, category continuity within adjacent time windows, and reasonable trajectory paths in known visual tasks. When the initial eye-tracking trajectory category does not match the evolution patterns in historical data, the system determines that the current recognition result is affected by abnormal sampling points, short-term noise, task switching misjudgments, or fusion weight deviations. The system iteratively optimizes and adjusts the fusion parameters to ensure that the contribution of temporal and spatial features to the comprehensive feature vector aligns with the current trajectory change state. For continuous trajectories, the system increases the contribution of temporal features; for cross-regional jump trajectories, the system increases the contribution of spatial features. Through this processing, the system refines the jump logic of eye-tracking trajectory changes from the comprehensive feature vector, reducing occasional misjudgments and improving the accuracy of the final eye-tracking trajectory recognition result.

[0041] Furthermore, accurate eye-tracking recognition results are converted into feedback signals and applied to the intelligent interaction system. The intelligent interaction system utilizes at least one of the following: intelligent terminals, intelligent cockpits, virtual reality devices, augmented reality devices, assisted reading systems, medical rehabilitation interaction systems, and educational training platforms. Feedback signals are used to update the system's response logic, specifically including adjusting the interface focus based on the user's actual attention points, highlighting target areas, triggering interactive object selection, optimizing information push order, and correcting the system's judgment of user intent. The system judges the overall recognition accuracy based on the consistency between the recognition results and subsequent user operations, and evaluates the recognition effect by the degree of matching between the recognized category and the actual clicked target, the area of ​​stay, the task completion result, and the user's confirmation feedback. The evaluation results are used for subsequent model parameter updates, thus forming the process of eye-tracking recognition and intelligent interaction response.

[0042] Example 2:

[0043] S1 includes parsing the eye-tracking data to obtain an initial set of sampling points; extracting position coordinates and timestamp sequences from the initial set of sampling points to obtain a two-dimensional mapping matrix; determining whether the offset of adjacent sampling points in the two-dimensional mapping matrix is ​​less than a threshold, and if the offset is less than the threshold, obtaining a set of fixation points; aggregating the set of fixation points to obtain a sequence of saccade path nodes, and aligning the saccade path node sequence with the timestamp sequence to obtain raw sequence data containing temporal order and spatial location.

[0044] In this embodiment, the eye-tracking acquisition device first outputs continuous eye-tracking trajectory data. This continuous eye-tracking trajectory data consists of multiple sampling records, each containing sampling time, lateral position, longitudinal position, sampling status identifier, and data validity identifier. After receiving the continuous eye-tracking trajectory data, the system first splits it into individual sampling records according to the boundaries of the data frames, and then reads the sampling time, lateral position, and longitudinal position from each sampling record. Records lacking sampling time are directly deleted by the system; records lacking lateral or longitudinal position are directly deleted by the system; records with invalid data validity identifiers are directly deleted by the system.

[0045] Furthermore, after deletion, the system rearranges the remaining sampling records in chronological order of sampling time. When two sampling records have the same sampling time, the system retains the record with the higher data validity indicator and deletes the other record. After the above parsing, filtering, and sorting, the remaining sampling records constitute the initial sampling point set. Thus, each sampling point in the initial sampling point set has a definite sampling time and a definite spatial location, and subsequent offset determination will not be affected by invalid records, duplicate records, or out-of-order records.

[0046] Furthermore, after obtaining the initial set of sampling points, the system extracts the position coordinates and timestamp sequence from each sampling point. The position coordinates consist of the horizontal and vertical positions, and the timestamp sequence consists of the sampling time corresponding to each sampling point. The system establishes a two-dimensional mapping matrix according to the chronological order of the sampling times: each row in the matrix corresponds to a sampling point, and the sampling time, horizontal position, and vertical position of that sampling point are written sequentially in each row. For cases where the coordinates output by the eye-tracking acquisition device are not screen coordinates, the system first calls the coordinate transformation relationship generated during the calibration of the acquisition device to convert the acquired coordinates into screen coordinates before writing them into the two-dimensional mapping matrix.

[0047] Specifically, this coordinate transformation process is performed according to the correspondence between calibration points. That is, the line-of-sight point recorded by the acquisition device is used as input, and the actual calibration position on the screen is used as output to establish a mapping relationship from acquisition coordinates to screen coordinates. During actual acquisition, each sampling point is transformed into the same screen coordinate system through this mapping relationship. The resulting two-dimensional mapping matrix simultaneously retains the sampling time, horizontal position, and vertical position, so that the spatial offset between adjacent sampling points has a unified calculation benchmark.

[0048] As a preferred embodiment, before determining the offset of adjacent sampling points, the system first determines a threshold for distinguishing between fixation points and non-fixation points. The threshold is determined jointly by the acquisition device calibration data, user static fixation data, and the task interface area boundary. Specifically, before the formal task begins, the system sequentially displays the center calibration point, upper left calibration point, upper right calibration point, lower left calibration point, and lower right calibration point on the interactive interface; the user fixates on each calibration point sequentially as prompted; the system acquires continuous valid sampling points at each calibration point. Subsequently, the system calculates the deviation between each valid sampling point and its corresponding calibration point. The calculation first determines the difference in horizontal position between the sampling point and the calibration point, then determines the difference in vertical position. The absolute value of the horizontal difference is squared, and the absolute value of the vertical difference is squared. The two squared results are then added together, and the square root of the sum is taken to obtain the spatial deviation of the sampling point relative to the calibration point.

[0049] Furthermore, the system sorts the spatial deviations obtained from all calibration points in ascending order, removes the three largest and three smallest spatial deviations, and then averages the remaining spatial deviations to obtain the gaze fluctuation amount under static gaze state. Simultaneously, the system reads the device calibration error output by the calibration program to obtain the device error amount; it then reads the minimum interval between the boundaries of adjacent interactive areas in the task interface, and uses half of this minimum interval as the area boundary limit. The system adds the gaze fluctuation amount to the device error amount to obtain the acquisition error compensation amount; then it compares the acquisition error compensation amount with the area boundary limit amount, and takes the smaller value as the offset threshold. Through this determination method, the threshold covers both the natural minor fluctuations and device calibration errors generated during normal user gaze, and does not cross the boundaries of adjacent interactive areas, thus avoiding classifying cross-area movement errors into the same gaze point set.

[0050] In this embodiment, after the threshold is determined, the system begins to determine the offset of adjacent sampling points in the two-dimensional mapping matrix. The system reads the previous and next sampling points in the order of the matrix rows, first calculating the difference between the horizontal positions of the two sampling points, and then calculating the difference between the vertical positions of the two sampling points; then, the absolute value of the horizontal difference is taken and squared, and the absolute value of the vertical difference is taken and squared, the two squared results are added together and the square root is taken to obtain the offset between adjacent sampling points. This offset represents the magnitude of spatial movement of the user's gaze between adjacent sampling moments. The system compares this offset with the aforementioned threshold: when the offset is less than the threshold, the next sampling point is marked as a fixation point; consecutively marked fixation points are written into the same fixation point set; when the offset is equal to or greater than the threshold, the system marks the position change at that point as the boundary of the fixation segment and stops adding the next sampling point to the current fixation point set. Through the above point-by-point determination, the system filters out sampling points with limited spatial movement amplitude in the continuous trajectory, forming a fixation point set that can characterize the visual fixation state.

[0051] Furthermore, after the gaze point set is formed, the system aggregates the gaze points within the set. During aggregation, the system first divides consecutive gaze points into gaze segments according to sampling time; gaze points within the same gaze segment are temporally continuous, and the offset of adjacent sampling points is less than a threshold. For each gaze segment, the system averages the lateral positions of all gaze points within that segment, and uses the average position as the lateral center of the gaze segment; the system then averages the longitudinal positions of all gaze points within that segment, and uses the average position as the longitudinal center of the gaze segment. The system determines the earliest sampling time in the gaze segment as the start time, the latest sampling time as the end time, and subtracts the start time from the end time to obtain the duration of the gaze segment. After this aggregation process, multiple discrete gaze points are merged into a single gaze node with a center position, start time, end time, and duration.

[0052] Specifically, the saccade path node sequence is generated from aggregated gaze nodes. The system arranges all gaze nodes in ascending order of their start times and uses these arranged gaze nodes as saccade path nodes. The spatial transfer relationship between two adjacent saccade path nodes is determined by the center positions of the two nodes: the system reads the center positions of the preceding and following nodes to determine the direction and distance of the gaze from the preceding attention area to the following attention area; simultaneously, it reads the end time of the preceding node and the start time of the following node to determine the time interval corresponding to the completion of this transfer. The resulting saccade path node sequence is no longer a simple stacking of scattered coordinate points, but a trajectory structure formed by sequentially connecting multiple attention nodes with time ranges and spatial centers, capable of expressing the user's gaze transfer process between different attention areas.

[0053] Finally, the system aligns the scan path node sequence with the timestamp sequence. During alignment, the system uses the timestamp sequence as a time reference, reads each sampling time one by one, and determines which scan path node's start and end times the sampling time falls between. When a sampling time falls within the time range of a scan path node, the system establishes a correspondence between the sampling time and the center position, duration, and node order of that scan path node; when a sampling time is between the end time of the previous scan path node and the start time of the next node, the system marks the sampling time as a node transition time and maps it to the scan transition relationship between the two scan path nodes. After traversing all sampling times, the system obtains raw sequence data containing temporal order and spatial location. In this raw sequence data, each time position corresponds to a specific spatial position or a specific node transition relationship.

[0054] Example 3:

[0055] S2 includes extracting spatial coordinates and time dimension information from the original sequence data to obtain historical trajectory sequences; processing the historical trajectory sequences using a long short-term memory network to obtain hidden state sequences; calculating spatial coordinate differences based on the hidden state sequences to obtain position offset sequences; if the position offset sequence is greater than a preset offset threshold, extracting the hidden state sequences corresponding to the position offset sequences to determine linkage relationship features; performing hierarchical clustering on the linkage relationship features to capture the linkage relationship between the order of events and position changes, and determining the dynamic change features under the time evolution law.

[0056] In this embodiment, the system first reads the raw sequence data output by S1. The raw sequence data already contains temporal order and spatial location. The system extracts the sampling time, lateral position, longitudinal position, and corresponding node identifier from each trajectory record. The node identifier is used to distinguish whether the trajectory record belongs to a fixation node, a saccade transfer node, or a regular sampling node. The system arranges all trajectory records in chronological order of sampling time and binds the lateral position, longitudinal position, and node identifier at the same sampling time into a single historical trajectory record. After binding, all historical trajectory records are connected chronologically to form a historical trajectory sequence. This process ensures that each record in the historical trajectory sequence has a definite temporal and spatial source, so the subsequent Long Short-Term Memory (LSTM) network processes no longer discrete coordinate points, but rather trajectory processes with a sequential relationship.

[0057] Specifically, before inputting the data into the Long Short-Term Memory (LSTM) network, the system organizes the historical trajectory sequences. During this process, the system first checks the sampling time interval between two adjacent historical trajectory records. When the sampling time interval exceeds the sampling interval set by the acquisition device for this task, the system sets a segment boundary at that point to prevent discontinuous trajectories from being mistakenly identified as continuous behavior. Subsequently, the system reads the horizontal and vertical lengths of the screen display area and converts the horizontal positions in the historical trajectory records into unified coordinates relative to the horizontal display range, and the vertical positions into unified coordinates relative to the vertical display range. The conversion process is as follows: the distance between the current horizontal position and the horizontal display starting point is divided by the overall length of the horizontal display range to obtain the unified horizontal position; the distance between the current vertical position and the vertical display starting point is divided by the overall length of the vertical display range to obtain the unified vertical position. After this organization, trajectory records generated by different screen sizes, display resolutions, and acquisition areas are brought to the same scale, ensuring a unified basis for subsequent positional difference calculations.

[0058] Furthermore, the Long Short-Term Memory (LSTM) network receives input sequentially according to the chronological order of the historical trajectory sequence. At each input moment, the system sends the lateral position, vertical position, sampling time interval, and node identifier of the current trajectory record to the LTM network. The LTM network reads the current input content, as well as the internal memory state retained from the previous moment and the hidden state output from the previous moment. The input gate determines the trajectory change information to be written at this moment based on the current trajectory record, the forget gate determines the historical information to be retained based on the continuity relationship between the current trajectory record and the trajectory state at the previous moment, and the output gate generates the hidden state for this moment based on the updated internal memory state.

[0059] In this system, for each historical trajectory record input, the Long Short-Term Memory (LSTM) network outputs a hidden state. These hidden states are arranged in the input order of the trajectory records to obtain a hidden state sequence. Each hidden state corresponds to a specific historical trajectory record, and the system retains the index relationship between the two, enabling the hidden state to trace back to the corresponding sampling time and spatial coordinates.

[0060] As a preferred embodiment, when calculating the spatial coordinate difference based on the hidden state sequence, the system first reads the spatial coordinates corresponding to two adjacent hidden states based on the index relationship between the hidden states and historical trajectory records. The system starts processing two temporally adjacent hidden states, first extracting the horizontal and vertical positions corresponding to the previous hidden state, and then extracting the horizontal and vertical positions corresponding to the next hidden state. Next, the system calculates the difference between the two horizontal positions and the difference between the two vertical positions; after converting the horizontal difference to a non-negative number, it squares it; after converting the vertical difference to a non-negative number, it squares it; then the two squared results are added together, and the square root of the sum is taken to obtain the spatial offset corresponding to the two adjacent hidden states.

[0061] Furthermore, the system repeats this calculation process for all adjacent hidden states in the hidden state sequence, arranging all spatial offsets in chronological order to form a position offset sequence. Each offset in the position offset sequence corresponds to a temporal position in the hidden state sequence, representing the intensity of spatial change between adjacent trajectory states.

[0062] Specifically, the offset threshold is determined before the position offset sequence is judged. The system extracts adjacent sampling points within the same gaze segment from the gaze point set obtained in S1 and calculates the spatial offset between these adjacent sampling points. This type of spatial offset comes from the natural eye movement fluctuations and device acquisition errors when the user maintains a gaze, and is therefore used as gaze-stabilized samples. The system sorts the gaze-stabilized samples in ascending order of value, deletes the sample with the largest and smallest value, and then averages the remaining samples to obtain the gaze-stabilized offset baseline.

[0063] Subsequently, the system extracts the center positions of adjacent saccade path nodes from the saccade path node sequence and calculates the spatial offset between adjacent center positions. This spatial offset arises from the user's gaze shifting from one region of interest to another, and is therefore used as a saccade shift sample. The system sorts the saccade shift samples in ascending order of value, deletes the sample with the smallest value, and reads the smallest value from the remaining samples to obtain the lower limit of the saccade shift offset. The system adds the fixation stability offset baseline to the lower limit of the saccade shift offset and takes half of the result to obtain a preset offset threshold. This offset threshold lies between the fixation stability offset range and the saccade shift offset range.

[0064] In this embodiment, after obtaining the position offset sequence and offset threshold, the system performs judgment item by item. The system reads each offset in the position offset sequence in chronological order and compares the offset with a preset offset threshold. When the offset is greater than the offset threshold, the system determines that there is a valid position change at that time position; when the offset is less than or equal to the offset threshold, the system determines that the time position belongs to gaze fluctuation or stable dwell and is not used as an object for linkage extraction. For the offsets determined to be valid position changes, the system records their sequential position in the position offset sequence and returns to the hidden state sequence based on the sequential position to extract the corresponding hidden state.

[0065] Furthermore, in order to express the causal relationship before and after the position change, the system also extracts the hidden state before and after the hidden state; when the effective position change is located at the starting point of the trajectory segment, the system extracts the current hidden state and the next hidden state; when the effective position change is located at the ending point of the trajectory segment, the system extracts the previous hidden state and the current hidden state.

[0066] Specifically, the linkage feature is determined by the aforementioned hidden states and corresponding offset information. The system first compares the changes between the previous and current hidden states to obtain the trajectory accumulation relationship before the position change; then it compares the changes between the current and next hidden states to obtain the trajectory continuation relationship after the position change. Simultaneously, the system reads the occurrence time, sampling time interval, lateral movement direction, longitudinal movement direction, and spatial offset magnitude corresponding to the offset. The system binds the trajectory state before the position change, the trajectory state during the position change, the trajectory state after the position change, the offset occurrence time, the offset direction, and the offset magnitude into a linkage feature. This linkage feature reflects a complete change process, i.e., how the line of sight accumulates from the previous state to the current position change, and in which direction it continues to evolve after the current position change.

[0067] Subsequently, the system performs hierarchical clustering on the linkage features. Initially, each linkage feature is treated as an independent category. The system sequentially compares the differences between any two categories, including five aspects: first, whether their positional offset directions are consistent; second, whether their positional offset magnitudes are similar; third, whether their sampling time intervals are consistent; fourth, whether their hidden state change trends before the positional change are consistent; and fifth, whether their hidden state continuation trends after the positional change are consistent. The system converts these five differences into a difference result at the same scale and synthesizes them in the order of positional offset direction, positional offset magnitude, time interval, previous hidden state change trend, and subsequent hidden state continuation trend. The synthesized difference result is used to represent the closeness between two linkage features.

[0068] In addition, during each clustering process, the system finds the two categories with the smallest overall difference and merges them into a new category. After merging, the system recalculates the overall difference between the new category and the remaining categories. This process is repeated until the clustering stopping condition is met. The clustering stopping condition is controlled by a clustering stopping threshold, which is also determined before clustering.

[0069] Specifically, the system retrieves historical trajectory samples that have already been labeled. For samples labeled with the same eye-tracking behavior, the system calculates the comprehensive difference between their linkage features and takes the largest stable value among these comprehensive differences as the upper limit of intra-class difference. For samples labeled with different eye-tracking behaviors, the system calculates the comprehensive difference between their linkage features and takes the smallest stable value among these comprehensive differences as the lower limit of inter-class difference. The system adds the upper limit of intra-class difference and the lower limit of inter-class difference and takes half of the sum to obtain the clustering stopping threshold. When the minimum comprehensive difference between any two categories is greater than the clustering stopping threshold, the system stops merging categories; when the minimum comprehensive difference is less than or equal to the clustering stopping threshold, the system continues merging categories.

[0070] Furthermore, after hierarchical clustering, the system reads all linkage features within each cluster and extracts common change patterns within that cluster. For linkage features within the same cluster, the system statistically analyzes their common preceding trajectory states, common positional offset directions, common offset magnitude ranges, common time interval features, and common subsequent continuation directions; these common elements are identified as the dynamic change features corresponding to that cluster. For differences between clusters, the system retains the boundary content that distinguishes different dynamic change features. The resulting dynamic change features clearly represent the evolution of the eye-tracking trajectory in chronological order, specifically including how the trajectory state of the previous stage affects the positional change of the next stage, the time interval in which the positional change occurs, and the direction in which the trajectory continues after the positional change.

[0071] Example 4:

[0072] S3 includes acquiring a first position coordinate sequence and calculating a first dynamic change feature based on the first position coordinate sequence; if the first dynamic change feature exceeds a preset threshold, extracting the first spatial dimension information of the first position coordinate sequence and determining the two-dimensional matrix representation of the first spatial dimension information; extracting features from the two-dimensional matrix representation through a convolutional neural network to obtain a first distribution pattern; and obtaining the association representation of spatial position and trajectory behavior logic based on the first distribution pattern.

[0073] In this embodiment, the system first extracts the trajectory segment within the current time window to be analyzed from the processing result of S2, and uses this trajectory segment as the first position coordinate sequence. This time window is determined by the continuous sampling time, and each trajectory point within the window includes the sampling time, lateral position, and longitudinal position. After reading all trajectory points within the window, the system first deletes trajectory points lacking sampling time, then deletes trajectory points lacking lateral or longitudinal positions, and then rearranges them according to the sampling time from earliest to latest. After the rearrangement is completed, the system obtains the first position coordinate sequence. Adjacent trajectory points in this sequence are temporally adjacent and have a calculable positional change relationship in space.

[0074] Specifically, the first dynamic change characteristic is calculated point-by-point from the first position coordinate sequence. Starting from the first trajectory point in the first position coordinate sequence, the system sequentially reads the preceding and following trajectory points that are adjacent in time. The system first calculates the difference in the horizontal position between the two trajectory points, then calculates the difference in the vertical position. Subsequently, the horizontal difference is converted to a non-negative number and squared, and the vertical difference is also converted to a non-negative number and squared. The two squared results are then added together, and the square root of the sum is taken to obtain the spatial movement between adjacent trajectory points. The system continues to read the sampling time difference between two trajectory points, and divides the spatial movement by the sampling time difference to obtain the movement intensity between the adjacent trajectory points.

[0075] Next, the system compares the movement directions of two adjacent trajectories: the first trajectory moves from the previous trajectory point to the current trajectory point, and the second trajectory moves from the current trajectory point to the next trajectory point. The system calculates the turning angle between the two movement directions and uses this turning angle as the change in direction. Spatial movement, movement intensity, and change in direction together form the basis for calculating the first dynamic change characteristic.

[0076] Furthermore, to ensure that data of different dimensions can be processed in the same way, the system standardizes the scales of spatial movement, movement intensity, and directional change. Spatial movement is calculated based on the maximum trajectory movement distance allowed by the current interactive interface; movement intensity is calculated based on the maximum movement distance per unit time allowed at the current sampling frequency; and directional change is calculated based on the angle corresponding to a complete reverse turn. After the conversion, the system adds the conversion results for spatial movement, movement intensity, and directional change, and divides the sum by 3 to obtain the first dynamic change feature. This first dynamic change feature simultaneously reflects changes in trajectory movement distance, movement speed, and directional change, indicating whether there is significant spatial change in the current trajectory segment.

[0077] Specifically, the preset threshold is determined before the formal trajectory recognition is performed. The system first enters the static gaze calibration stage, displaying the center calibration point, upper left calibration point, upper right calibration point, lower left calibration point, and lower right calibration point sequentially on the interactive interface. When the user gazes at each calibration point, the system collects 30 valid trajectory points at that calibration point and calculates the dynamic change value under static gaze state according to the calculation method of the first dynamic change feature. After the collection of 5 calibration points is completed, the system arranges all static gaze dynamic change values ​​in ascending order of value, deletes the two largest and two smallest dynamic change values, and then averages the remaining dynamic change values ​​to obtain the static gaze dynamic reference.

[0078] Further, after completing the static gaze calibration, the system enters the cross-regional transfer calibration stage. The system selects two clearly spaced regions of interest in the interactive interface and prompts the user to move their gaze from the previous region of interest to the next. The system continuously collects 10 cross-regional transfer trajectories and obtains 10 sets of cross-regional transfer change values ​​according to the same calculation process. The system arranges the 10 sets of cross-regional transfer change values ​​in ascending order, deletes the set with the smallest value, and then reads the smallest value among the remaining transfer change values ​​as the lower limit of cross-regional transfer. The system adds the static gaze dynamic baseline to the lower limit of cross-regional transfer and takes half of the result to obtain a preset threshold. This threshold lies between static gaze fluctuations and cross-regional gaze transfers, and is used to determine whether the current first position coordinate sequence is necessary for spatial dimension analysis.

[0079] As a preferred embodiment, the system compares a first dynamic change feature with a preset threshold. When the first dynamic change feature is less than the preset threshold, the system determines that the current trajectory segment belongs to a stable gaze segment and does not extract the first spatial dimension information. When the first dynamic change feature is equal to the preset threshold, the system processes it as a stable gaze segment. When the first dynamic change feature is greater than the preset threshold, the system determines that the current trajectory segment has a shift in attention, saccade extension, or path jump, and initiates the extraction of the first spatial dimension information.

[0080] Furthermore, when extracting the first spatial dimension information, the system reads all horizontal and vertical positions in the first position coordinate sequence, and reads the horizontal and vertical boundaries of the current interactive interface, as well as the interactive area division results. The system divides the interactive interface into multiple spatial units, each with a defined horizontal and vertical range. For each trajectory point in the first position coordinate sequence, the system first determines which horizontal range the trajectory point falls into, and then determines which vertical range the trajectory point falls into, thereby determining the spatial unit corresponding to the trajectory point. After a trajectory point falls into a spatial unit, the system records the trajectory point's entry time, exit time, number of occurrences, and sequence number within that spatial unit. After all trajectory points are assigned, the first spatial dimension information consists of the number of trajectory points in each spatial unit, dwell time, entry order, exit order, and direction of travel.

[0081] In this embodiment, the two-dimensional matrix representation is formed by writing the first spatial dimension information into a matrix. The system arranges spatial units into a two-dimensional matrix from top to bottom and left to right according to the actual spatial layout of the interactive interface. Each element in the matrix corresponds to a spatial unit in the interactive interface. For spatial units with trajectory points, the system writes the number of trajectory points, the cumulative dwell time, and the path order identifier within that spatial unit; for spatial units without trajectory points, the system writes a blank trajectory value. When a spatial unit is traversed multiple times by a trajectory, the system accumulates the number of trajectory points and the dwell time of that spatial unit, and retains the earliest entry order and the latest exit order. Thus, the first spatial dimension information is transformed into a two-dimensional matrix representation. This two-dimensional matrix representation retains the spatial landing point, dwell intensity, and path order of the trajectory in the interface, enabling the convolutional neural network to extract features based on spatial adjacency relationships.

[0082] In this process, when processing two-dimensional matrix representations, the convolutional neural network first uses a convolutional kernel to read local spatial regions within the matrix. After the convolutional kernel covers a local matrix region, the system reads the number of trajectory points, cumulative dwell time, and path order identifiers for each matrix element within that region. The system multiplies the weight of each position in the convolutional kernel with the trajectory value in the corresponding matrix element, and then adds the multiplication results within the same local region to obtain the convolutional response for that local region. A large convolutional response indicates the presence of trajectory clustering, concentrated dwell time, or continuous paths within that local region; a small convolutional response indicates sparse trajectories or a lack of continuous behavioral relationships within that local region. After activation processing, the system retains valid responses and suppresses blank trajectory regions and isolated noise points. Subsequently, the system uses pooling to retain the main response values ​​in adjacent response regions, reducing the interference of repeated location points on distribution judgment.

[0083] Furthermore, after multi-layer convolution processing, the system obtains a first distribution pattern. This first distribution pattern includes the region of interest clustering, the trajectory extension direction, the spatial distribution boundary, and path connectivity. The region of interest clustering is determined by the matrix regions within the convolutional response set; the trajectory extension direction is determined by the change in path sequence identifiers within adjacent matrix regions; the spatial distribution boundary is determined by the boundary between the effective response region and the blank trajectory region; and the path connectivity is determined by the order in which trajectory points enter one matrix region from another. The system combines these elements to form a first distribution pattern that describes the spatial shape of the current trajectory segment.

[0084] Furthermore, when generating the association representation based on the first distribution pattern, the system first determines the attention-gathering areas in the first distribution pattern, and then reads the path connectivity between the attention-gathering areas. When the trajectory continuously stays within the same attention-gathering area, the system establishes a correspondence between that area and gaze-holding behavior; when the trajectory directly enters another attention-gathering area from one attention-gathering area, the system establishes a correspondence between the connection between the two areas and spatial jump behavior; when the trajectory moves sequentially along multiple adjacent spatial units, the system establishes a correspondence between the continuous movement path and saccade extension behavior; when the trajectory returns to an attention-gathering area that it has already passed, the system establishes a correspondence between the return path and retrospective behavior.

[0085] After completing the above correspondence, the system binds spatial location, entry order, dwell time, path direction, and behavior type into the same representation record, obtaining a logical association between spatial location and trajectory behavior. This association representation describes which spatial locations the eye-tracking trajectory falls in, in what order it moves, in which areas it stays, and the eye-tracking behavior corresponding to the spatial changes, providing spatial dimension input for the spatiotemporal feature fusion of S4.

[0086] Example 5:

[0087] S4 includes acquiring time-series data and spatial coordinate data, extracting the temporal evolution patterns of the time-series data through a long short-term memory network to obtain dynamic change features; using a graph convolutional neural network based on the dynamic change features and spatial coordinate data to obtain the correlation representation between spatial location and trajectory behavior logic; determining whether there are abnormal fluctuation values ​​in the correlation representation, and if so, smoothing the correlation representation using a Kalman filter algorithm to obtain a denoised correlation representation; and using a fusion module to concatenate the dynamic change features and the denoised correlation representation to obtain a comprehensive feature vector after temporal and spatial fusion.

[0088] In this embodiment, the system first reads the generated raw sequence data, dynamic change features, and spatial location records. The time series data consists of sampling time, time intervals between adjacent sampling points, gaze duration, scan path node sequence, and node switching time; the spatial coordinate data consists of the lateral position of the sampling point, the longitudinal position of the sampling point, the center position of the gaze node, and the center position of the scan path node. When reading the data, the system uses the sampling time or node sequence as a benchmark to bind the temporal and spatial information at the same moment into a single trajectory record. After binding, each trajectory record contains a definite temporal and spatial source, preventing misalignment of temporal and spatial features during subsequent fusion.

[0089] As a preferred implementation, the system first performs sequence organization before the time-series data enters the Long Short-Term Memory (LSTM) network. The system reads the sampling times of two adjacent trajectory records and calculates the time interval between them; the time interval is obtained by subtracting the sampling time of the previous trajectory record from the sampling time of the later one. The system then reads the start and end times of each gaze node, subtracting the start time from the end time to obtain the duration of that gaze node. For saccade path nodes, the system writes the node number according to the order of their appearance and records the switching duration between the end time of the previous node and the start time of the next node. The aforementioned time intervals, durations, switching durations, and node order together constitute the input content of the LTM network.

[0090] The Long Short-Term Memory (LSTM) network receives time-series data sequentially. For the current input, the LTM network reads the current time interval, current gaze duration, current node order, and current switch duration, while also reading the internal memory state retained from the previous moment. The input gate writes new time-related information generated at the current moment, the forget gate deletes old time information irrelevant to the current trajectory evolution, and the output gate outputs the current time state. For each time record input, the LTM network outputs a time state; all time states are arranged in the input order to form a dynamic change feature. This dynamic change feature expresses the trajectory's pauses, switches, continuations, and dependencies over time.

[0091] Furthermore, spatial processing involves first establishing a trajectory graph structure. The system sets each gaze node, scan path node, and effective sampling point as a node in the graph. Each graph node includes its lateral position, vertical position, entry time, exit time, dwell time, and corresponding dynamic change characteristics. The connections between nodes are determined by both temporal order and spatial proximity: two nodes that appear consecutively in time are connected temporally; two nodes whose spatial positions fall within the same interaction area are connected regionally; and two nodes whose spatial positions fall within adjacent interaction areas and appear consecutively in the scan path are connected by a transition. This graph structure simultaneously preserves both the relationships of "which position is visited first" and "how to transition between adjacent positions."

[0092] Specifically, node connection strength is calculated based on spatial distance, time interval, and path continuity. The system first reads the center positions of two nodes, calculates the difference between their lateral and longitudinal positions, converts the lateral difference to a non-negative number and squares it, and the longitudinal difference is also converted to a non-negative number and squared. The two squared results are then added together and the square root is taken to obtain the spatial distance between the two nodes. The system then reads the time interval between the two nodes, which is obtained by subtracting the departure time of the preceding node from the arrival time of the following node.

[0093] The system then determines whether two nodes are consecutive in the scanned path node sequence; consecutive nodes are assigned a high path continuity value, while non-consecutive nodes are assigned a low path continuity value. The system converts spatial distance, time interval, and path continuity value to the same scale before merging them to obtain the node connection strength. The smaller the spatial distance, the shorter the time interval, and the stronger the path continuity, the higher the node connection strength.

[0094] Furthermore, the graph convolutional neural network performs feature propagation on the aforementioned trajectory graph structure. When processing a graph node, the graph convolutional neural network first reads the node's own horizontal position, vertical position, dwell time, and dynamic change features, and then reads the information of the neighboring nodes connected to it. The system weights and aggregates the neighboring node information according to the node connection strength: neighboring nodes with high connection strength contribute more to the current node's spatial position and trajectory state; neighboring nodes with low connection strength contribute less.

[0095] Specifically, after completing the adjacency information aggregation, the system merges the current node's own information with the aggregated adjacency information, and then performs a nonlinear transformation using a graph convolutional neural network to obtain an updated node representation. After continuous graph convolutional propagation, each node representation simultaneously contains its own spatial location, neighboring node relationships, trajectory sequence relationships, and temporal dynamic change information. The system arranges all updated node representations according to sampling time and path order to obtain a logical association between spatial location and trajectory behavior.

[0096] In this embodiment, after the associated representation is generated, the system needs to determine whether there are any abnormal fluctuation values. The threshold for abnormal fluctuation is determined before identification. The system first collects stable gaze calibration data. During calibration, the user sequentially gazes at fixed calibration points on the interface, and 30 valid node representations are collected for each calibration point. The system calculates the variation amplitude between adjacent node representations within the same calibration point. The variation amplitude is obtained by combining the changes in spatial position, the changes in dwell time, and the changes in connection strength. The system sorts the variation amplitudes obtained from stable gaze in ascending order of value, deletes the two largest and two smallest values, and then averages the remaining values ​​to obtain the stable fluctuation benchmark.

[0097] Subsequently, the system collects normal saccade calibration data. Following prompts, the user moves their gaze from one area of ​​focus to another. The system collects 10 sets of normal saccade trajectories and calculates the variation amplitude between adjacent node representations using the same method. The system arranges the 10 sets of normal saccade variation amplitudes in ascending order of value, removes the smallest value, and reads the smallest value among the remaining values ​​to obtain the lower limit of normal transition. The system adds the stable fluctuation baseline to the lower limit of normal transition, and then takes half of the sum to obtain the abnormal fluctuation judgment threshold. This threshold lies between stable gaze jitter and normal saccade transition, and is used to identify non-genuine jumps.

[0098] Furthermore, during trajectory recognition, the system calculates the variation amplitude between adjacent node representations sequentially according to the temporal order of their associated representations. The calculation process is as follows: first, the spatial position changes of adjacent nodes are compared; then, the dwell time changes of adjacent nodes are compared; next, the connection strength changes of adjacent nodes are compared. After scaling these three changes, they are merged into a single variation amplitude. If this variation amplitude exceeds the abnormal fluctuation threshold, and the node connection relationship corresponding to this change does not exist in the scan path node sequence, the system determines that there is an abnormal fluctuation value at that location. If the variation amplitude is less than or equal to the abnormal fluctuation threshold, the system determines that there is no abnormal fluctuation value at that location. This processing can distinguish between genuine scan shifts and coordinate spikes caused by sudden changes in illumination, short-term occlusion, and equipment jitter.

[0099] When abnormal fluctuations exist, the system employs a Kalman filter algorithm to smooth the correlation representation. Before smoothing begins, the system uses the correlation representation preceding the abnormal fluctuation position as the previous state estimate. Based on the previous state estimate and the historical change directions of the preceding and following nodes, the system predicts the correlation representation that should appear at the current time position, obtaining the current state prediction. Simultaneously, the system reads the correlation representation at the current abnormal fluctuation position and uses it as the observation. The prediction value comes from the continuous evolution of the preceding trajectory, and the observation value comes from the current acquisition results; both are used together to correct the representation result of the abnormal fluctuation position.

[0100] In addition, the prediction error in Kalman filtering is determined by stable gaze calibration data. The system reads the average fluctuation of adjacent associations during the stable gaze calibration phase. The larger the average fluctuation, the stronger the trajectory jitter under the acquisition environment, and the larger the prediction error; the smaller the average fluctuation, the smaller the prediction error. The observation error is jointly determined by the acquisition equipment calibration error and the validity of the current trajectory point. The larger the acquisition equipment calibration error, the larger the observation error; the lower the validity indicator of the current trajectory point, the larger the observation error. The system determines the smoothing weight based on the prediction error and the observation error: when the prediction error is less than the observation error, the weight of the prediction value increases; when the prediction error is greater than the observation error, the weight of the observation value increases.

[0101] Specifically, after determining the smoothing weights, the system weighted and merged the current state prediction value with the current observation value to obtain a smoothed correlation representation of the abnormal fluctuation location. Subsequently, the system replaced the original abnormal fluctuation value with this smoothed correlation representation and used it as the new state estimate to continue processing subsequent time locations. After all abnormal fluctuation locations have been replaced, the system obtains a denoised correlation representation. The denoised correlation representation preserves the continuous changes formed by the true path transition while reducing the influence of non-true spikes on the spatial trajectory logic.

[0102] In addition, before the dynamic change features and denoised correlation representations enter the fusion module, the system performs alignment. The system uses sampling time and node order as dual benchmarks to check whether each dynamic change feature has a corresponding denoised correlation representation. When their time positions are consistent, they are directly included in the same fusion record; when the dynamic change feature exists but the denoised correlation representation is missing, the system reads valid denoised correlation representations from adjacent time positions to fill in the gaps; when the denoised correlation representation exists but the dynamic change feature is missing, the system reads valid dynamic change features from adjacent time positions to fill in the gaps. After completion, the fusion module obtains a set of data with consistent time positions to be stitched together.

[0103] Furthermore, during the splicing process, the fusion module first places the dynamically changing features at the beginning of the fused record, and then places the denoised correlation representation at the end, maintaining a fixed order of feature items. Subsequently, the system performs scale unification on the spliced ​​fused record. Temporal features are converted according to the maximum duration in the time series data, spatial features are converted according to the maximum spatial distance of the interactive interface, and node connection features are converted according to the range of connection strength values. After scale unification, the fusion module outputs a comprehensive feature vector after temporal and spatial fusion. This comprehensive feature vector simultaneously contains the temporal evolution pattern, spatial positional relationships, trajectory behavior logic, and a stable, denoised representation.

[0104] Example 6:

[0105] S5 includes extracting features from the comprehensive feature vector to generate a first gaze point sequence; matching the first gaze point sequence with a reference trajectory to obtain a first saccade path; calculating the similarity between the first saccade path and the reference trajectory to obtain a first similarity value; if the first similarity value is greater than a threshold, the focus is determined to be a first target region; extracting spatial cluster centers based on the first target region; obtaining a reference trajectory for multi-task scenarios based on the comprehensive feature vector; determining the user's trajectory focus by comparing similarity; and determining the preliminary eye movement trajectory category.

[0106] In this embodiment, the system reads the output comprehensive feature vector. This comprehensive feature vector contains temporal evolution information, spatial coordinate information, dwell information, path direction information, and denoised trajectory behavior information. The system splits the comprehensive feature vector according to the sampling time and trajectory node order, extracting the spatial location, dwell duration, distance between preceding and following nodes, trajectory movement direction, and behavior status identifier corresponding to each sampling time. After splitting, the system rebinds the above information at the same sampling time into candidate gaze records. Each candidate gaze record has a clear sampling time, spatial location, and trajectory status, serving as the basic data for generating the first gaze point sequence.

[0107] Specifically, the first fixation point sequence is formed by filtering candidate fixation records that meet the fixation criteria. During the filtering process, the system reads the horizontal and vertical positions of the current candidate fixation record and the previous candidate fixation record. First, it calculates the difference between the two in the horizontal position, and then calculates the difference between the two in the vertical position. Subsequently, the horizontal difference is converted to a non-negative number and squared, and the vertical difference is converted to a non-negative number and squared. Then, the two squared results are added together and the square root is taken to obtain the spatial distance between the two candidate fixation records.

[0108] Furthermore, the system continues to read the sampling time difference between the two, and divides the spatial distance by the sampling time difference to obtain the movement intensity of the trajectory segment. When the dwell time of a candidate gaze record reaches the gaze determination time threshold, and the spatial distance between it and an adjacent candidate gaze record is less than the gaze determination distance threshold, the system determines the candidate gaze record as a gaze point record. Gazette records that continuously meet the above conditions are arranged in order of sampling time from earliest to latest to obtain the first gaze point sequence.

[0109] In this embodiment, the gaze determination time threshold and gaze determination distance threshold are calibrated before formal recognition. The system sets several fixed target areas in the interactive interface, and the user gazes at each fixed target area sequentially. Thirty valid sampling points are collected for each fixed target area, and the system records the sampling time, lateral position, and longitudinal position of each sampling point. For the time threshold, the system calculates the dwell time formed by consecutive valid sampling points within the same fixed target area, arranges all dwell times in ascending order, deletes the shortest and longest set of dwell times, and then averages the remaining dwell times to obtain the gaze determination time threshold.

[0110] For the distance threshold, the system calculates the spatial distance between adjacent sampling points within the same fixed target area. All spatial distances are arranged in ascending order of value, the maximum and minimum distances are removed, and the remaining spatial distances are averaged to obtain the static gaze fluctuation distance. Simultaneously, the minimum interval between the boundaries of adjacent target areas is read, and half of this minimum interval is used as the area boundary limit distance. The system compares the static gaze fluctuation distance and the area boundary limit distance, taking the smaller value as the gaze determination distance threshold. This determination method ensures that the threshold covers natural jitter in stable gaze without crossing the boundaries of adjacent target areas.

[0111] Furthermore, after obtaining the first gaze point sequence, the system reads reference trajectories from the reference trajectory library for matching. The reference trajectory library is established during the training or calibration phase, and each reference trajectory stores the task scene identifier, target region order, reference gaze point center position, reference dwell time, reference saccade direction, and trajectory category identifier. The system first reads the current task scene identifier and retains only reference trajectories consistent with the current task scene; then it reads the number of gaze points, gaze point center positions, dwell time, and transition directions between adjacent gaze points from the first gaze point sequence, and compares them one by one with the retained reference trajectories. The comparison includes whether the target region traversal order is consistent, whether the gaze point center position falls within the corresponding reference region, whether the dwell time falls within the reference dwell range, and whether the saccade direction is consistent with the reference saccade direction. After the comparison is completed, the system selects the reference trajectory with the smallest difference value as the matching result and generates the first saccade path according to the target region connection order in that reference trajectory.

[0112] Specifically, the generation process of the first saccade path is as follows: The system reads the first gaze point in the first gaze point sequence and uses it as the path starting point; it continues to read the next gaze point, connecting the center position of the previous gaze point to the center position of the next gaze point to form a saccade path; then it reads the end time of the previous gaze point and the start time of the next gaze point, subtracting the end time of the previous gaze point from the start time of the next gaze point to obtain the transition time corresponding to this saccade path. The system processes all adjacent gaze points in the first gaze point sequence sequentially until the last gaze point is connected to the path. The resulting first saccade path includes the path starting point, the path ending point, the intermediate areas traversed, the direction of each path segment, and the transition time of each segment, which can express the actual saccade process of the user's gaze in the current task.

[0113] The similarity between the first saccade path and the reference trajectory is calculated using multiple trajectory parameters. First, the system calculates path order similarity: it compares the order of target regions in the first saccade path with the order of target regions in the reference trajectory; the more nodes with the same order, the higher the path order similarity. Next, the system calculates spatial location similarity: it reads the spatial distance between the center position of each gaze point in the first saccade path and the corresponding reference center position in the reference trajectory; the smaller the distance, the higher the spatial location similarity.

[0114] Subsequently, the system calculates dwell time similarity: comparing the dwell time of each fixation point in the first saccade path with the corresponding reference dwell time in the reference trajectory; the smaller the difference, the higher the dwell time similarity. Finally, the system calculates transfer direction similarity: comparing the directional difference between each saccade direction in the first saccade path and the corresponding reference direction in the reference trajectory; the smaller the directional difference, the higher the transfer direction similarity. After scaling the four similarity results, the system adds the four results together and divides the sum by 4 to obtain the first similarity value.

[0115] Specifically, the similarity threshold is determined during the reference trajectory database establishment phase. The system first selects historical trajectory samples that have been manually labeled. Samples belonging to the same trajectory behavior as a certain reference trajectory are designated as matching samples, while samples not belonging to the reference trajectory behavior are designated as non-matching samples. The system calculates the similarity between matching samples and their corresponding reference trajectories, as well as the similarity between non-matching samples and the reference trajectory, according to the similarity calculation method between the first scanning path and the reference trajectory. For matching samples, the system sorts the similarity values ​​from smallest to largest, deletes the lowest value, and reads the lowest value among the remaining values ​​as the lower limit of matching similarity.

[0116] Specifically, for non-matching samples, the system sorts the similarity values ​​from smallest to largest, removes the highest value, and then reads the highest value among the remaining values ​​as the upper limit of non-matching similarity. The system adds the lower limit of matching similarity and the upper limit of non-matching similarity, and then takes half of the sum to obtain the similarity threshold. This threshold lies between the matching trajectory and the non-matching trajectory, and is used to determine whether the current scanning path belongs to the same trajectory behavior as the reference trajectory.

[0117] As a preferred embodiment, the system compares a first similarity value with a similarity threshold. When the first similarity value is greater than the similarity threshold, the system determines that the first scanning path matches the current reference trajectory, and reads the target region identifier corresponding to the current time in the reference trajectory, identifying that target region as the first target region. When the first similarity value is less than or equal to the similarity threshold, the system does not use the target region corresponding to that reference trajectory, and continues to read the next reference trajectory in the reference trajectory library for similarity comparison. If the similarity values ​​of multiple reference trajectories are all greater than the similarity threshold, the system selects the reference trajectory with the highest similarity value and uses the target region corresponding to that reference trajectory as the first target region.

[0118] Once the first target region is determined, the system extracts the spatial cluster centers within that region. The system reads all gaze point records falling within the first target region, including lateral position, longitudinal position, dwell time, and entry order. At the start of clustering, each gaze point record is treated as an independent cluster point. The system calculates the spatial distance between any two cluster points, following the same process as described above: processing the lateral and longitudinal differences separately, then squaring, summing, and taking the square root to obtain the distance result. The system finds the two cluster points with the smallest spatial distance and merges them if this distance is less than the target region's clustering threshold. The center position of the merged cluster is obtained by averaging the lateral and longitudinal positions of all gaze points within the merged group. The clustering process continues until the minimum spatial distance between any two cluster points is greater than or equal to the target region's clustering threshold. The system then selects the cluster point with the most gaze points and the longest cumulative dwell time, and determines its center position as the spatial cluster center.

[0119] Furthermore, the target region clustering threshold is jointly determined by the size of the first target region and the historical gaze dispersion. The system reads the horizontal and vertical boundaries of the first target region, calculates the horizontal width and vertical height of the region, and takes the smaller of the two values ​​as the region constraint distance. The system then reads the gaze points that fall into the same target region in historical tasks, calculates the spatial distance between these gaze points and the manually calibrated center of the target region, sorts the obtained spatial distances in ascending order of value, deletes the maximum and minimum distances, and then averages the remaining distances to obtain the historical gaze dispersion distance. The system compares half of the region constraint distance with the historical gaze dispersion distance, and takes the smaller value as the target region clustering threshold. This threshold restricts the clustering process to be completed within the target region and ensures that the spatial cluster center reflects the user's actual attention position within the target region.

[0120] In addition, after obtaining the first target region and spatial cluster center, the system again acquires reference trajectories for the multi-task scenario. The system reads the current task scenario identifier, the first target region identifier, the location of the spatial cluster center, and the comprehensive feature vector, and filters reference trajectories from the reference trajectory library that pass through the first target region and belong to the current task scenario. Then, the system compares the temporal evolution information, spatial distribution information, and path behavior information in the comprehensive feature vector with the filtered reference trajectories one by one for similarity. The similarity comparison still uses four aspects: path order, spatial location, dwell time, and transfer direction, and combines the distance between the spatial cluster center and the reference target center to correct the spatial location similarity result.

[0121] Finally, the system selects the reference trajectory with the highest similarity score that is greater than the similarity threshold as the target reference trajectory, reads the trajectory category identifier corresponding to the target reference trajectory, and determines the preliminary eye movement trajectory category. This category is used to represent the type of eye movement behavior corresponding to the current user's focus point, including fixation, saccade shift, retrospective confirmation, target search, and task switching.

[0122] Example 7:

[0123] S6 includes obtaining the initial eye movement trajectory category and the matching deviation degree of historical evolution pattern; if the matching deviation degree is greater than the preset threshold, the updated fusion weight value is calculated, and the updated fusion weight value is used to process the comprehensive feature vector to obtain the refined feature matrix; the jump logic node is determined according to the refined feature matrix, and the jump logic node is used to extract trajectory change features; the initial eye movement trajectory category is corrected according to the trajectory change features to obtain accurate eye movement trajectory recognition results.

[0124] In this embodiment, the system reads the initial eye-tracking trajectory category, and simultaneously reads the corresponding current task scene, current time window, trajectory category of the previous time window, current target region, current saccade path, and comprehensive feature vector. The system also calls upon a historical evolution pattern library. This library is generated from labeled eye-tracking trajectory samples and records the order of appearance, category transition relationships, target region switching order, dwell time center value, saccade direction, actual jump nodes, and trajectory mutation categories for each eye-tracking trajectory category within the same task scene. The system places the current initial eye-tracking trajectory category into this historical evolution pattern library and compares it with the historical category evolution relationships within the same task scene to obtain the matching deviation degree of the current initial eye-tracking trajectory category relative to the historical evolution patterns.

[0125] Specifically, the matching deviation is a composite of category transfer deviation, time duration deviation, spatial region deviation, and path direction deviation. The calculation process for category transfer deviation is as follows: The system first reads the trajectory category of the previous time window, then retrieves the historical frequency of the current initial category appearing after that previous category from the historical evolution pattern database; simultaneously, the system retrieves the total historical frequency of all subsequent categories appearing after that previous category. The system divides the current category transfer frequency by the total number of subsequent category transfers to obtain the consistency level of the current category transfer; then, it subtracts this consistency level from the complete consistency value to obtain the category transfer deviation. When the category transfer relationship does not exist in the historical evolution pattern database, the system directly records the category transfer deviation as the highest deviation value.

[0126] Furthermore, the time persistence deviation is calculated by comparing the actual duration of the current category with the center value of historical durations. The system reads the start and end times of the current preliminary eye-tracking trajectory category within the current time window, subtracts the start time from the end time to obtain the actual duration. Subsequently, the system reads the center values ​​of historical durations for the same task scenario and trajectory category from the historical evolution pattern database. The system subtracts the center value of historical durations from the actual duration, converts the result to a non-negative number, and then divides this non-negative number by the center value of historical durations to obtain the time persistence deviation. The further the actual duration deviates from the center value of historical durations, the greater the time persistence deviation.

[0127] Specifically, spatial region deviation is used to represent the degree of deviation between the current area of ​​interest and the area that should appear according to historical patterns. The system reads the center position of the current target area and then reads the center position of the reference target area corresponding to the current preliminary eye-tracking trajectory category from the historical evolution pattern database. The system first calculates the difference between the two center positions in the horizontal direction, and then calculates the difference between the two center positions in the vertical direction; then, it converts the horizontal difference to a non-negative number and squares it, converts the vertical difference to a non-negative number and squares it, and then adds the two squared results and takes the square root to obtain the spatial distance between the center of the current target area and the center of the reference target area. The system then reads the maximum distance between any two target area centers in the current interactive interface and divides the above spatial distance by this maximum distance to obtain the spatial region deviation.

[0128] Furthermore, the path direction deviation is obtained from the directional difference between the current scan path and the historical reference path. The system reads the path direction of each segment in the current scan path and the reference direction of each corresponding reference path in the historical evolution pattern database. For sequentially corresponding path segments in two paths, the system calculates the angular difference between the current path direction and the reference path direction; when a cross-regional connection occurs in the current scan path where the reference path does not exist, the system records the path direction deviation of that segment as the highest deviation value. After all path segments have been processed, the system averages the direction deviations of each segment to obtain the path direction deviation.

[0129] After obtaining the four types of biases, the system performs scale unification. During scale unification, category transfer bias, temporal duration bias, spatial region bias, and path direction bias are all converted to the same numerical range, allowing for comparison of the four types of biases. The system reads the discriminative power of each type of bias against misclassified samples from historical verification samples. This discriminative power is determined by the number of times that bias alone distinguishes between the correct and incorrect categories. The higher the number of hits, the greater the contribution of that type of bias. The system multiplies each type of bias by its corresponding contribution value and then sums the results to obtain the matching deviation degree between the current preliminary eye-tracking trajectory category and the historical evolution pattern. The resulting matching deviation degree simultaneously reflects the degree of inconsistency in four aspects: category order, duration, spatial region, and path direction.

[0130] As a preferred implementation, the matching deviation threshold is determined during the historical evolution pattern database establishment phase. The system first reads labeled historical trajectory samples with correct categories and calculates the correct category sample deviation using the aforementioned matching deviation calculation process. Then, it reads the same batch of historical trajectory samples, replaces their true categories with incorrect categories, and calculates the incorrect category sample deviation using the same process. The system sorts the correct category sample deviations from smallest to largest, deletes the largest deviation, and reads the largest remaining deviation as the upper limit of the correct category deviation. Similarly, the system sorts the incorrect category sample deviations from smallest to largest, deletes the smallest deviation, and reads the smallest remaining deviation as the lower limit of the incorrect category deviation. The system adds the upper limit of the correct category deviation to the lower limit of the incorrect category deviation, and then halves the sum to obtain the preset matching deviation threshold. This threshold lies between the normal category evolution deviation and the incorrect category evolution deviation, and is used to determine whether the initial eye-tracking trajectory category needs to enter the correction process.

[0131] Furthermore, the system compares the current matching deviation with a matching deviation threshold. When the matching deviation is less than or equal to the matching deviation threshold, the system determines that the current preliminary eye movement trajectory category conforms to the historical evolution pattern and retains that preliminary eye movement trajectory category. When the matching deviation is greater than the matching deviation threshold, the system determines that the current preliminary eye movement trajectory category does not match the historical evolution pattern and begins to calculate the updated fusion weight value.

[0132] Specifically, the update of the fusion weight value is determined by the source of the bias. The system first separates temporal and spatial features from the comprehensive feature vector. Temporal features include temporal evolution patterns, dwell time, node switching time, and category transfer relationships; spatial features include spatial coordinates, target region, path direction, and denoised association representation. The system merges category transfer bias and temporal persistence bias into temporal bias, and merges spatial region bias and path direction bias into spatial bias. The system then adds the temporal and spatial biases to obtain the total bias; divides the temporal bias by the total bias to obtain the temporal bias occupancy; and divides the spatial bias by the total bias to obtain the spatial bias occupancy.

[0133] The base adjustment step size is determined by historical validation samples. The system sets several candidate adjustment step sizes in the historical validation samples, updates the fusion weights sequentially using each candidate adjustment step size, and re-executes class correction. The system counts the number of times the correct class is maintained and the number of times the incorrect class is corrected under each candidate adjustment step size, and selects the candidate adjustment step size that simultaneously satisfies the highest number of times the correct class is maintained and the highest number of times the incorrect class is corrected as the base adjustment step size. When two or more candidate adjustment step sizes satisfy the above conditions, the system selects the candidate adjustment step size with the smallest value as the base adjustment step size.

[0134] Specifically, after obtaining the basic adjustment step size, the system calculates the updated fusion weights based on the temporal and spatial bias occupancy. If the temporal bias occupancy is greater than the spatial bias occupancy, it indicates that the initial classification error mainly stems from inconsistent temporal evolution; the system increases the fusion weight of temporal features and decreases the fusion weight of spatial features. If the spatial bias occupancy is greater than the temporal bias occupancy, it indicates that the initial classification error mainly stems from inconsistent spatial trajectories; the system increases the fusion weight of spatial features and decreases the fusion weight of temporal features. When the two are equal, the system retains the original fusion ratio of both temporal and spatial features, making only a minor balancing correction according to the basic adjustment step size. After the weight update, the system adds the temporal and spatial fusion weights to obtain a normalized benchmark, and then divides the updated temporal and spatial fusion weights by this normalized benchmark respectively, ensuring that the sum of the two updated fusion weights remains a single complete weight unit.

[0135] Furthermore, when processing the comprehensive feature vector using the updated fusion weights, the system first expands the comprehensive feature vector according to the sampling time and trajectory node order. Each sampling time or trajectory node corresponds to a row of data to be processed. The system writes dynamic change features, dwell time, and category transition information into the time feature column; spatial coordinates, target area, path direction, and denoised association representation into the spatial feature column; and behavioral status identifiers and recognition confidence information into the auxiliary feature column. Subsequently, the system processes the time feature column with updated time fusion weights and processes the spatial feature column with updated spatial fusion weights. The auxiliary feature column maintains its original order and is arranged together with the processed time and spatial feature columns. After all rows are processed, the system obtains a refined feature matrix. Each row in the refined feature matrix corresponds to a time position or trajectory node, and each column corresponds to a type of trajectory feature processed with explicit weights, thus enabling its use in subsequent node jump judgments.

[0136] Furthermore, the jump logic node is determined row by row by row of the refined feature matrix. Starting from the second row of the refined feature matrix, the system reads the current row and the previous row sequentially. The system first calculates the spatial position change between the current and previous rows, processing the lateral and longitudinal position differences separately, and then squaring, summing, and taking the square root to obtain the spatial jump distance. The system continues to calculate the path direction change by reading the direction of the previous and current path segments and calculating the angle difference between them. Subsequently, the system calculates the dwell time change by subtracting the dwell time of the previous node from the dwell time of the current node and converting the difference to a non-negative number. The system also reads the behavioral state identifiers of the current and previous rows; if they match, the state change value is recorded as the lowest value; if they do not match, the state change value is recorded as the highest value. After the spatial jump distance, direction change, dwell time change, and state change value are scaled uniformly, the system merges the four results into the node jump intensity.

[0137] The threshold for determining jump nodes is determined by historical real jump samples and historical continuous samples. The system extracts real saccade jump nodes, retrospective jump nodes, and task switching jump nodes from the labeled historical trajectory, obtaining real jump intensity samples according to the node jump intensity calculation process. The system then extracts ordinary continuous nodes from stable gaze trajectories and continuous saccade trajectories, obtaining continuous node intensity samples according to the same calculation process. The system sorts the continuous node intensity samples in ascending order of value, deletes the sample with the largest value, and reads the maximum value from the remaining samples as the upper limit of continuous node intensity. The system sorts the real jump intensity samples in ascending order of value, deletes the sample with the smallest value, and reads the minimum value from the remaining samples as the lower limit of real jump intensity. The system adds the upper limit of continuous node intensity and the lower limit of real jump intensity, then halves the sum to obtain the jump node determination threshold.

[0138] The system compares the node jump intensity with a jump node determination threshold. When the node jump intensity is greater than the threshold, the system identifies the current node as a jump logical node; when the node jump intensity is less than or equal to the threshold, the system retains the current node as a normal trajectory node. Jump logical nodes are used to represent locations in the eye-tracking trajectory where there is a sudden change. This change is not a single coordinate jump, but rather simultaneously satisfies one or more of the following characteristics: a sudden change in spatial location, a change in path direction, a change in dwell state, or a switch in behavioral state.

[0139] Specifically, after identifying the jump logic node, the system extracts trajectory mutation features. The system reads the previous node, current node, and next node of the jump logic node. The system first calculates the spatial jump distance from the previous node to the current node, and then calculates the spatial continuity distance from the current node to the next node; subsequently, it reads the dwell time before the jump, the dwell time after the jump, the jump time, the jump direction, the change in target area, and the change in behavioral state. The system binds the above information into trajectory mutation features. These trajectory mutation features describe from which area the line of sight mutates to which area, when the mutation occurs, how the direction of the mutation changes, whether a dwell time occurs before or after the mutation, and whether the mutation is accompanied by a change in behavioral category.

[0140] The category correction relies on historical mutation templates. These templates are generated from real mutation segments in the labeled trajectories, and include scan-to-transfer, retrospective confirmation, target search, and task switching templates. Each mutation template records the jump distance range, direction change range, target area change order, dwell time before jump, dwell time after jump, and category transfer relationship for the corresponding category. The system compares the current trajectory mutation features with each mutation template item by item: if the jump distance falls within the template's distance range, the distance item is marked as consistent; if the direction change falls within the template's direction range, the direction item is marked as consistent; if the target area change order matches the template, the area item is marked as consistent; if the dwell time before and after jump falls within the template's time range, the dwell item is marked as consistent; if the category transfer relationship matches the template, the category item is marked as consistent. The system merges all consistent results to obtain the template matching value between the current trajectory mutation features and each mutation template.

[0141] Specifically, the mutation template matching threshold is also determined by historical labeled samples. The system reads historical samples belonging to the same mutation category as a given mutation template, calculates the template matching value between these samples and the mutation template, and obtains similar matching samples. Then, it reads historical samples that do not belong to the same mutation category, calculates the template matching value between these samples and the mutation template, and obtains dissimilar matching samples. The system sorts the similar matching samples in ascending order of value, deletes the lowest value, and reads the lowest value among the remaining values ​​as the lower limit of similar matching. The system sorts the dissimilar matching samples in ascending order of value, deletes the highest value, and reads the highest value among the remaining values ​​as the upper limit of dissimilar matching. The system adds the similar matching lower limit and the dissimilar matching upper limit, and then takes half of the sum to obtain the mutation template matching threshold.

[0142] In addition, the system corrects the initial eye-tracking trajectory category based on trajectory mutation features. When the template matching value between the current trajectory mutation feature and a certain mutation template is greater than the mutation template matching threshold, the system reads the eye-tracking trajectory category corresponding to that mutation template and corrects the initial eye-tracking trajectory category to that category. When the template matching values ​​of two or more mutation templates are all greater than the mutation template matching threshold, the system selects the category corresponding to the mutation template with the highest template matching value as the correction category. When the template matching values ​​of all mutation templates are less than or equal to the mutation template matching threshold, the system does not correct the initial eye-tracking trajectory category and outputs the initial eye-tracking trajectory category as the current recognition result.

[0143] After the correction is completed, the system outputs accurate eye-tracking trajectory recognition results. These results include the final eye-tracking trajectory category, the current target region, the location of the jump logic node, trajectory abrupt change features, matching deviation, updated fusion weight values, and the basis for category correction.

[0144] Example 8:

[0145] S7 includes acquiring eye-tracking trajectory data to obtain accurate eye-tracking trajectory recognition results, generating feedback signals based on accurate eye-tracking trajectory recognition results, obtaining the restored real trajectory changes based on the execution record of the feedback signals, updating the system response logic based on the restored real trajectory changes, calculating the verification deviation value of the recognition results through the updated system response logic, and judging the overall recognition accuracy if the verification deviation value is less than a preset threshold.

[0146] In this embodiment, the system first reads the eye-tracking trajectory data output by the eye-tracking acquisition device. This eye-tracking trajectory data includes sampling time, lateral position, longitudinal position, gaze nodes, saccade path nodes, and trajectory behavior state. The system parses the eye-tracking trajectory data, extracts temporal features, extracts spatial features, performs spatiotemporal fusion, compares with reference trajectories, and corrects the category to obtain accurate eye-tracking trajectory recognition results. The accurate eye-tracking trajectory recognition results include the final eye-tracking trajectory category, target area, trajectory focus point, jump logic nodes, trajectory abrupt change features, recognition time, recognition confidence information, and the corresponding comprehensive feature vector.

[0147] Specifically, after obtaining accurate eye-tracking trajectory recognition results, the system generates a feedback signal. During generation, the system reads the final eye-tracking trajectory category and the current interface state of the intelligent interaction system. The interface state includes the current page level, the target area within the page, the interaction permissions for the target area, the current task stage, and the system-allowed response actions. The system maps the eye-tracking trajectory category to the interface state: when the recognition result is gaze retention, the feedback signal points to the current gaze area and generates a focus-hold instruction or a target highlight instruction; when the recognition result is saccade shift, the feedback signal points to the end area of ​​the saccade path and generates a focus-switching instruction; when the recognition result is retrospective confirmation, the feedback signal points to the user's retrospective area and generates a candidate target confirmation instruction; when the recognition result is target search, the feedback signal points to the current key area in the search path and generates a prompt enhancement instruction; when the recognition result is task switching, the feedback signal points to the area corresponding to the new task and generates a response logic switching instruction. After the feedback signal is generated, the system writes the generation time, target area, action type, effective execution period, and corresponding eye-tracking trajectory category to the feedback signal, forming a pending feedback record.

[0148] Furthermore, after the feedback signal enters the intelligent interaction system, the system records its execution process. The execution record includes the feedback signal generation time, actual execution time, feedback target area, actual system response area, type of action, interface state changes, continuous eye movement focus after feedback, user operation result after feedback, and whether the feedback was canceled by the user. The system subtracts the generation time from the actual execution time to obtain the response delay; it reads the feedback target area and the actual system response area to determine if the response areas are consistent; it reads the user's continuous eye movement focus after feedback to determine if the user's gaze remains on the feedback target area; and it reads the user's operation result to determine if the user performed a confirmation, click, pause, cancel, or switch operation on the feedback target area.

[0149] As a preferred implementation, when reconstructing the actual trajectory changes, the system relies on the feedback execution record and continuous eye-tracking sampling points after the feedback. The system first reads the eye-tracking sampling points after the feedback signal is executed and rearranges them according to the sampling time from earliest to latest. Then, it reads the user's subsequent operation results, including clicks on targets, confirmation actions, cancellation actions, interface transitions, and areas of focus. When the user continues to stay within the feedback target area and performs a confirmation action within that area, the system records the feedback target area as the true area of ​​focus. When the user leaves the feedback target area after the feedback signal is executed and completes a click or confirmation in another target area, the system records the latter target area as the true area of ​​focus. When the user cancels the system response and returns to the previous area of ​​focus, the system records the previous area of ​​focus as the true area of ​​focus.

[0150] The system then connects the actual area of ​​focus, the continuous eye-tracking points after feedback, the user's operation results, and the corresponding time sequence to obtain the reconstructed true trajectory changes. This reconstruction includes the actual area of ​​focus, the actual movement path, the actual dwell time, the actual operation action, and the actual trajectory category.

[0151] Furthermore, after restoring the actual trajectory changes, the system updates its response logic. During the update, the system reads the target region from the accurate eye-tracking trajectory recognition result and the actual attention region from the restored actual trajectory changes. When they match, the system strengthens the correspondence between the eye-tracking trajectory category and the current response action, and writes this feedback process into the reliable response sample; when they do not match, the system weakens the correspondence between the eye-tracking trajectory category and the current response action, and writes the inconsistent trajectory segments into the sample to be corrected. The system further compares the trajectory category in the recognition result with the actual behavior type in the actual trajectory changes. When the recognition is gaze maintenance but the actual trajectory shows task switching, the system increases the role of cross-region jump features in the subsequent response logic; when the recognition is saccade shift but the actual trajectory shows retrospective confirmation, the system increases the role of retrospective path and return to target region in the subsequent response logic; when the recognition is target search but the actual trajectory shows stable gaze, the system increases the role of dwell time and target region stability in the subsequent response logic. After the update, the target region selection rules, action triggering rules, response waiting time, and category priority in the system response logic are all corrected based on the actual trajectory changes.

[0152] Subsequently, the system calculates the verification deviation value of the recognition results using the updated system response logic. Before the calculation, the system re-inputs the accurate eye-tracking trajectory recognition results into the updated system response logic to obtain the target area that the updated system should trigger, the type of action that should be executed, the response time that should be triggered, and the corresponding trajectory category. The system then compares these results with the reconstructed actual trajectory changes. The comparison includes target area deviation, action deviation, response time deviation, category deviation, and trajectory path deviation.

[0153] Specifically, the target area deviation is calculated using the distance between the center of the target area. The system reads the target area center position output by the updated system response logic and the actual center position of the area of ​​interest. First, it calculates the difference between the two center positions in the horizontal direction, then calculates the difference in the vertical direction. The horizontal difference is then converted to a non-negative number and squared, and the vertical difference is also converted to a non-negative number and squared. The two squared results are then added together and the square root is taken to obtain the distance between the center of the target area. The system reads the maximum distance between any two target area centers on the current screen and compares this maximum distance with the current center distance to obtain the target area deviation. The larger the distance between the center of the target area, the larger the target area deviation.

[0154] Furthermore, the action deviation is determined based on the system action type and the user's actual operation result. The system reads the action type output by the updated system response logic and then reads the user's actual operation result. When the two match, the action deviation is recorded as the minimum deviation value; when they do not match, the action deviation is recorded as the maximum deviation value. The response time deviation is obtained by subtracting the time the system should respond from the actual time the change in attention occurred. The system subtracts the two and converts the difference to a non-negative number, then calculates the response time deviation based on the maximum allowable response delay for the current task. The category deviation is obtained by comparing the eye-track category in the recognition result with the actual trajectory category. When the two match, the minimum deviation value is recorded; when they do not match, the maximum deviation value is recorded. The trajectory path deviation is obtained by comparing the node order of the recognized path and the actual path. The system compares the order of target areas in the two paths item by item. The more nodes with the same order, the smaller the trajectory path deviation; the more nodes with different orders, the larger the trajectory path deviation.

[0155] Specifically, after obtaining the aforementioned deviations, the system performs scale unification. Target area deviation, response time deviation, and trajectory path deviation are converted to the same scale according to their respective maximum allowable deviation ranges; action deviation and category deviation are directly written to the same scale based on the consistency / inconsistency judgment results. The system reads the degree of influence of each type of deviation on the final recognition accuracy from historical verification samples and assigns contribution values ​​to target area deviation, action deviation, response time deviation, category deviation, and trajectory path deviation. The contribution value is determined as follows: the system counts the number of hits when each type of deviation is used alone to distinguish between correctly and incorrectly identified samples; the higher the number of hits, the greater the contribution value of that type of deviation. The system combines each type of deviation with its corresponding contribution value, and then adds the combined results to obtain the verification deviation value of the recognition result.

[0156] Specifically, the preset threshold is determined during the system's sample verification phase. The system first reads labeled historical interaction samples that have been manually verified as correctly identified, and calculates the verification deviation value for each correctly identified sample according to the aforementioned verification deviation value calculation process. The system sorts the verification deviation values ​​of the correctly identified samples in ascending order, deletes the largest value, and then reads the largest value among the remaining verification deviation values ​​as the upper limit of the correct identification deviation. The system then reads labeled historical interaction samples that contain incorrect identification results, and calculates the verification deviation value for each incorrectly identified sample according to the same process. The system sorts the verification deviation values ​​of the incorrectly identified samples in ascending order, deletes the smallest value, and then reads the smallest value among the remaining verification deviation values ​​as the lower limit of the incorrect identification deviation. The system adds the upper limit of the correct identification deviation to the lower limit of the incorrect identification deviation, and then halves the sum to obtain the preset threshold. This threshold lies between the deviation boundaries of correctly identified samples and incorrectly identified samples, and is used to determine whether the current identification result passes the overall accuracy verification.

[0157] Furthermore, the system compares the verification deviation value of the current recognition result with a preset threshold. When the verification deviation value is less than the preset threshold, the system determines that the current eye-tracking trajectory recognition result meets the overall recognition accuracy requirements and writes the recognition result, feedback signal, execution record, and restored real trajectory changes into the trusted historical data. When the verification deviation value is greater than or equal to the preset threshold, the system does not write the recognition result into the trusted historical data, but instead writes the corresponding eye-tracking trajectory segment, feedback execution record, real trajectory changes, and deviation source into the data to be corrected.

[0158] Example 9:

[0159] A spatiotemporal feature fusion-based eye-tracking trajectory recognition system is also provided, used to implement the steps of the spatiotemporal feature fusion-based eye-tracking trajectory recognition method described in the above embodiments, including:

[0160] The data acquisition and parsing module collects eye-tracking trajectory data of users in multi-task scenarios, and extracts position coordinates and timestamp sequences from continuous behavior records using a preset sampling frequency to obtain raw sequence data containing time sequence and spatial location.

[0161] The time dimension feature extraction module uses a long short-term memory network to process time dimension information based on the original sequence data, captures the linkage relationship between the sequence and position changes from historical data, and determines the dynamic change features under the time evolution law.

[0162] The spatial dimension feature extraction module analyzes spatial dimension information through a convolutional neural network if the dynamic change features exceed a preset threshold. It extracts the distribution pattern of the eye-tracking focus from the location coordinates to obtain the association representation between spatial location and trajectory behavior logic.

[0163] The spatiotemporal feature fusion module integrates the dynamic change features under the time evolution law and the correlation representation of spatial location and trajectory behavior logic, filters out noise interference from environmental influences, and obtains a comprehensive feature vector after spatiotemporal fusion.

[0164] The trajectory attention point judgment module obtains reference trajectory patterns in multi-task scenarios based on the comprehensive feature vector, and determines the user's trajectory attention point at a specific time by comparing similarity, thus determining the preliminary eye-tracking trajectory category.

[0165] If the initial eye movement trajectory category does not match the evolution pattern in historical data, the iterative optimization and correction module will adjust the fusion parameters through iterative optimization to refine the jump logic of eye movement trajectory changes from the comprehensive feature vector and obtain accurate eye movement trajectory recognition results.

[0166] The feedback verification module generates feedback signals based on accurate eye-tracking trajectory recognition results and applies them to the intelligent interaction system. It updates the system response logic based on the restored real trajectory changes and judges the overall recognition accuracy.

[0167] In the above implementation, the data acquisition and parsing module first collects the user's eye-tracking trajectory data in a multi-task scenario, and extracts the location coordinates and timestamp sequence from the continuous behavior records to form raw sequence data with temporal order and spatial location.

[0168] After reading the original sequence data, the time dimension feature extraction module uses a long short-term memory network to process the time dimension information, identify the linkage relationship between the changes in the trajectory position before and after, and obtain dynamic change features.

[0169] The spatial dimension feature extraction module compares the dynamically changing features with a preset threshold. When the dynamically changing features exceed the preset threshold, a convolutional neural network is used to perform spatial analysis on the location coordinates, extract the distribution pattern of the focus, and obtain the association representation between spatial location and trajectory behavior logic.

[0170] The spatiotemporal feature fusion module fuses dynamically changing features and associated representations, and filters out noise generated by environmental interference to obtain a comprehensive feature vector.

[0171] The trajectory attention point judgment module calls the reference trajectory pattern in the multi-task scenario based on the comprehensive feature vector, and determines the user's trajectory attention point and preliminary eye-tracking trajectory category at a specific time through similarity comparison.

[0172] When the initial eye movement trajectory category does not match the historical evolution pattern, the iterative optimization and correction module adjusts the fusion parameters and refines the trajectory jump logic from the comprehensive feature vector to obtain accurate eye movement trajectory recognition results.

[0173] The feedback verification module generates a feedback signal based on the accurate eye-tracking trajectory recognition results and applies the feedback signal to the intelligent interaction system. The system restores the real trajectory changes based on the feedback execution record, updates the system response logic, and judges the overall recognition accuracy.

[0174] Example 10:

[0175] Figure 2 This is a structural block diagram of the local terminal of an exemplary electronic device (machine) of the present invention; as shown... Figure 2 As shown, the electronic device of the present invention includes a processor 11, a memory 12, a storage space 13 for storing program code, and program code 14 for executing the method steps according to the present invention. The program code 14 for executing the method steps according to the present invention is used to execute the above-described control logic.

[0176] Figure 3This is a structural block diagram of the network end of an exemplary electronic device of the present invention; as shown below. Figure 3 As shown, the present invention also provides an electronic device (machine), which may include at least one processor 210, at least one memory 230 communicatively connected to the processor, and a communication bus 240 and a communication interface 220 connecting different system components (including the memory 230 and the processor 210). The processor 210, the memory 230 and the communication interface 220 are connected through the communication bus 240 and communicate with each other. The communication interface 220 is used for data interaction with external devices. The memory 230 stores a machine-executable program that can be executed by the processor, and the processor 210 can execute the above-mentioned control logic by calling the machine-executable program.

[0177] Communication bus 240 represents one or more of several bus architectures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnection (PCI) bus.

[0178] Electronic devices typically include a variety of computer system readable media, which can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, and removable and non-removable media.

[0179] Memory 230 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 230 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the control logic described above.

[0180] A program / utility having a set (at least one) of program modules can be stored in memory 230. Such program modules include, but are not limited to, an operating system, one or more applications, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0181] Machine-executable programs for performing this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, C++, and Python, and may also include specialized engineering languages ​​such as R. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0182] The present invention also discloses a storage medium storing a computer program, namely a machine-executable program as described above, which, when executed, implements the eye-tracking trajectory recognition method based on spatiotemporal feature fusion.

[0183] The aforementioned storage medium may be any combination of one or more computer-readable media. Computer-readable media may be, for example, computer-readable signal media or computer-readable storage media. Computer-readable storage media include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium may be, for example, any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0184] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0185] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0186] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An eye-tracking trajectory recognition method based on spatiotemporal feature fusion, characterized in that, include: S1. By collecting eye-tracking trajectory data of users in multi-task scenarios, and using a preset sampling frequency, position coordinates and timestamp sequences are extracted from continuous behavior records to obtain raw sequence data containing time sequence and spatial location. S2. Capture the correlation between chronological order and positional changes from historical data to determine the dynamic change characteristics under the time evolution pattern; S3. If the dynamic change characteristics exceed the preset threshold, analyze the spatial dimension information, extract the distribution pattern of the eye-tracking focus from the location coordinates, and obtain the association representation between spatial location and trajectory behavior logic. S4. Integrate the dynamic change characteristics under the time evolution law and the correlation representation of spatial location and trajectory behavior logic, filter out noise interference from environmental influences, and obtain the comprehensive feature vector after time and space fusion. S5. Obtain reference trajectory patterns in multi-task scenarios, determine the user's trajectory focus at a specific moment, and identify the initial eye-tracking trajectory category; specifically: Extract features from the comprehensive feature vector to generate the first gaze point sequence; The first saccade path is obtained by matching the reference trajectory based on the first gaze point sequence; Calculate the similarity between the first scanning path and the reference trajectory to obtain the first similarity value; If the first similarity score is greater than the threshold, then the point of interest is determined to be the first target region; Based on the first target region, spatial cluster centers are extracted, and reference trajectories in multi-task scenarios are obtained based on the comprehensive feature vector. By comparing similarity, the user's trajectory focus is determined, and the preliminary eye-tracking trajectory category is determined. S6. If the initial eye movement trajectory category does not match the evolution pattern in historical data, the fusion parameters are adjusted through iterative optimization to refine the jump logic of eye movement trajectory changes from the comprehensive feature vector and obtain accurate eye movement trajectory recognition results. S7. Generate feedback signals and apply them to the intelligent interaction system to update the system response logic from the restored real trajectory changes and judge the overall recognition accuracy.

2. The eye-tracking trajectory recognition method based on spatiotemporal feature fusion according to claim 1, characterized in that: S1 includes: The eye-tracking data is analyzed to obtain an initial set of sampling points; A two-dimensional mapping matrix is ​​obtained by extracting the location coordinates and timestamp sequence from the initial set of sampling points; Determine whether the offset of adjacent sampling points in the two-dimensional mapping matrix is ​​less than a threshold. If the offset is less than the threshold, then obtain the set of fixation points. The gaze path node sequence is obtained by aggregating the gaze point set. The gaze path node sequence is then aligned with the timestamp sequence to obtain the original sequence data containing temporal order and spatial location.

3. The eye-tracking trajectory recognition method based on spatiotemporal feature fusion according to claim 1, characterized in that: S2 includes: Historical trajectory sequences are obtained by extracting spatial coordinates and temporal dimension information from the original sequence data. A long short-term memory network is used to process historical trajectory sequences to obtain hidden state sequences; The position offset sequence is obtained by calculating the spatial coordinate differences based on the hidden state sequence; If the position offset sequence is greater than the preset offset threshold, the hidden state sequence corresponding to the position offset sequence is extracted to determine the linkage relationship features. Hierarchical clustering is performed on the linkage characteristics to capture the linkage relationship between the sequence and positional changes, and to determine the dynamic change characteristics under the time evolution law.

4. The eye-tracking trajectory recognition method based on spatiotemporal feature fusion according to claim 1, characterized in that: S3 includes: Obtain the first position coordinate sequence and calculate the first dynamic change feature; If the first dynamic change feature exceeds the preset threshold, then the first spatial dimension information of the first position coordinate sequence is extracted and the two-dimensional matrix representation of the first spatial dimension information is determined. The first distribution shape is obtained by extracting features from the two-dimensional matrix representation using a convolutional neural network. Based on the first distribution pattern, the association representation between spatial location and trajectory behavior logic is obtained.

5. The eye-tracking trajectory recognition method based on spatiotemporal feature fusion according to claim 1, characterized in that: S4 includes: We acquire time series data and spatial coordinate data, and extract the temporal evolution patterns of the time series data through a long short-term memory network to obtain dynamic change characteristics. Based on dynamic change characteristics and spatial coordinate data, a graph convolutional neural network is used to obtain a logical association between spatial location and trajectory behavior. Determine whether there are abnormal fluctuations in the correlation representation. If there are abnormal fluctuations, smooth the correlation representation using the Kalman filter algorithm to obtain a denoised correlation representation. A fusion module is used to concatenate the dynamically changing features and the denoised correlation representation to obtain a comprehensive feature vector after temporal and spatial fusion.

6. The eye-tracking trajectory recognition method based on spatiotemporal feature fusion according to claim 1, characterized in that: S6 includes: To obtain the initial matching deviation between eye movement trajectory categories and historical evolution patterns; If the matching deviation is greater than the preset threshold, the updated fusion weight value is calculated, and the updated fusion weight value is used to process the comprehensive feature vector to obtain the refined feature matrix. Jump logic nodes are determined based on the refined feature matrix, and these jump logic nodes are used to extract trajectory change features. The initial eye movement trajectory category is corrected based on the trajectory mutation characteristics to obtain accurate eye movement trajectory recognition results.

7. The eye-tracking trajectory recognition method based on spatiotemporal feature fusion according to claim 1, characterized in that: The S7 includes: Acquire eye-tracking data to obtain accurate eye-tracking recognition results, and generate feedback signals based on the accurate eye-tracking recognition results; The actual trajectory changes are obtained from the execution records of the feedback signals, and the system response logic is updated based on the actual trajectory changes. The updated system response logic calculates the verification deviation value of the recognition result. If the verification deviation value is less than the preset threshold, the overall recognition accuracy is judged.

8. An eye-tracking trajectory recognition system based on spatiotemporal feature fusion, used to implement the steps of the eye-tracking trajectory recognition method based on spatiotemporal feature fusion as described in any one of claims 1-7, characterized in that, include: The data acquisition and parsing module collects eye-tracking trajectory data of users in multi-task scenarios, and extracts position coordinates and timestamp sequences from continuous behavior records using a preset sampling frequency to obtain raw sequence data containing time sequence and spatial location. The time dimension feature extraction module uses a long short-term memory network to process time dimension information based on the original sequence data, captures the linkage relationship between the sequence and position changes from historical data, and determines the dynamic change features under the time evolution law. The spatial dimension feature extraction module analyzes spatial dimension information through a convolutional neural network if the dynamic change features exceed a preset threshold. It extracts the distribution pattern of the eye-tracking focus from the location coordinates to obtain the association representation between spatial location and trajectory behavior logic. The spatiotemporal feature fusion module integrates the dynamic change features under the time evolution law and the correlation representation of spatial location and trajectory behavior logic, filters out noise interference from environmental influences, and obtains a comprehensive feature vector after spatiotemporal fusion. The trajectory attention point judgment module obtains reference trajectory patterns in multi-task scenarios based on the comprehensive feature vector, and determines the user's trajectory attention point at a specific time by comparing similarity, thus determining the preliminary eye-tracking trajectory category. If the initial eye movement trajectory category does not match the evolution pattern in historical data, the iterative optimization and correction module will adjust the fusion parameters through iterative optimization to refine the jump logic of eye movement trajectory changes from the comprehensive feature vector and obtain accurate eye movement trajectory recognition results. The feedback verification module generates feedback signals based on accurate eye-tracking trajectory recognition results and applies them to the intelligent interaction system. It updates the system response logic based on the restored real trajectory changes and judges the overall recognition accuracy.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed, it implements the eye-tracking trajectory recognition method based on spatiotemporal feature fusion as described in claims 1-7.

Images