Saccadic eye movement abnormality evaluation and classification system and method based on cross-stimulus spatiotemporal evolution characteristics
By constructing an eye movement anomaly assessment and classification system based on cross-stimulus spatiotemporal evolution characteristics, we decouple intrinsic attention patterns from emotion-specific responses. By utilizing adaptive shared basis decomposition and local-global encoding methods, we improve the accuracy and robustness of eye movement anomaly assessment and solve the problem of limited model performance in existing technologies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2026-07-07
- Publication Date
- 2026-08-04
AI Technical Summary
Existing technologies ignore the fact that eye movement patterns consist of intrinsic attentional patterns and specific response patterns triggered by specific emotional stimuli, resulting in coupling of the discriminative information learned by the model and limiting the classification performance of eye movement anomaly assessment.
An eye movement anomaly assessment and classification system based on cross-stimulus spatiotemporal evolution features is adopted. The spatial encoder captures the spatial information of eye movement trajectory, and the adaptive shared basis decomposition mechanism is used to learn a stable low-rank subspace. Combined with stimulus invariance representation learning and specific response extraction branches, a classifier is constructed to output the anomaly assessment results.
It improves the accuracy and robustness of eye movement abnormality assessment and classification, accurately captures the spatiotemporal evolution characteristics of eye movement trajectories, decouples intrinsic attention patterns from emotion-specific responses, and enhances the model's generalization ability and interpretability.
Smart Images
Figure CN122498847A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of physiological signal analysis technology, specifically relating to an eye movement abnormality assessment and classification system and method based on cross-stimulus spatiotemporal evolution characteristics. Background Technology
[0002] Eye movements directly reflect an individual's attentional orientation, attentional maintenance, and attentional shifting characteristics, serving as a direct external manifestation of the brain's cognitive processing. Analyzing eye movement data can reveal an individual's psychological and physiological state at multiple levels, helping to identify potential mental health conditions early and significantly improving the timeliness of intervention and treatment response rates, which is crucial for early intervention and disease control. In recent years, eye-tracking technology has demonstrated potential assessment and screening value in clinical applications due to its low cost and non-invasive nature. In eye movement abnormality assessment and classification tasks, researchers typically employ a free-viewing paradigm, presenting participants with different emotional stimuli (such as neutral and negative facial images) and collecting their eye movement data to reveal systematic biases in an individual's visual attention allocation.
[0003] However, existing research, when extracting eye movement features, often attempts to learn the overall eye movement pattern under all stimuli. This approach ignores the fact that eye movement patterns are actually composed of intrinsic attentional patterns and specific response patterns triggered by specific emotional stimuli. Intrinsic attentional patterns refer to stable preferences independent of the specific stimulus presented, such as central preference and salience drive in the human visual system. Attentional avoidance or maintenance in response to emotional stimuli, on the other hand, strongly depends on the type of emotional stimulus, reflecting specific attentional biases under different emotional stimuli. This coupling between stimulus-invariant intrinsic attentional patterns and stimulus-specific response patterns leads to coupling of the discriminative representation information learned by the model, limiting the classification performance of eye movement anomaly assessment. Summary of the Invention
[0004] The purpose of this invention is to address the problem that existing technologies neglect the fact that eye movement patterns are actually composed of intrinsic attention patterns and specific response patterns triggered by specific emotional stimuli, leading to coupling of the representational discriminative information learned by the model and limiting the performance of eye movement abnormality assessment and classification. This invention proposes an eye movement abnormality assessment and classification system and method based on cross-stimulus spatiotemporal evolution characteristics.
[0005] The technical solution of the present invention is as follows: Firstly, an eye movement abnormality assessment and classification system based on cross-stimulus spatiotemporal evolution characteristics, comprising: A spatial encoder is used to receive eye movement trajectories under negative and neutral stimuli, capture the spatial information correlation of each eye movement trajectory, and obtain negative eye movement trajectory features and neutral eye movement trajectory features. The stimulus invariance representation learning branch is used to receive negative and neutral eye movement trajectory features. An adaptive shared basis decomposition mechanism is used to learn and reconstruct a stable low-rank subspace, and to capture stable stimulus invariant eye movement features in the common subspace. The stimulus-specific response extraction branch is used to receive negative and neutral eye movement trajectory features, and to mine the dynamic details of eye movement trajectory features through local-global encoding to obtain the stimulus-specific response features triggered by each emotional stimulus. A classifier is used to receive stimulus-invariant eye movement features and stimulus-specific response features, and outputs the results of the abnormal assessment of eye movement representation.
[0006] As a preferred option, the adaptive shared basis decomposition mechanism is as follows: Stable universal latent subpatterns are obtained by learning basis vectors shared across stimuli in the common representation space using a set of learnable shared bases; where the maximum number of universal latent subpatterns is [number missing]. That is, the total number of basis vectors is preset to be Learnable basis vector matrix sum coefficient matrix They are respectively recorded as: , ;in, Represents the basis vector matrix Feature dimensions, Represents the coefficient matrix Feature dimensions, Represents the set of real numbers; Based on negative eye movement trajectory features Adaptive rank weights are generated using an adaptive rank predictor based on the Sigmoid activation function. :
[0007]
[0008] in, , , Indicates a linear layer. Representation layer normalization, express Activation function Indicates negative eye movement trajectory features through linear layers Layer normalization and Features after activation function processing Indicates the length of the eye-track feature. This indicates the feature in the length dimension. Features obtained by global average pooling. express Activation function; Based on neutral eye-tracking features Adaptive rank weights are generated using an adaptive rank predictor based on the Sigmoid activation function. :
[0009]
[0010] in, Indicating neutral eye movement trajectory features via linear layers Layer normalization and Features after activation function processing This indicates the feature in the length dimension. Features obtained by global average pooling; Based on rank weight and Adaptive selection of general latent sub-patterns to generate stimulus-invariant eye-tracking features and :
[0011]
[0012] in, and They are respectively and transpose, This indicates element-wise multiplication.
[0013] As a preferred approach, the stimulus invariance representation learning branch incorporates rank structure regularization constraints during training. Specifically, this includes sparse rank constraints based on L1 regularization terms. Rank order constraints based on rank index awareness and considering the rank sum constraint of normalization The corresponding formula is:
[0014]
[0015]
[0016]
[0017] in, Represents rank weights, including and , Represents the identity matrix. Indicates the first Each weighting coefficient Indicates a sequential index.
[0018] As a preferred embodiment, the stimulus-specific response extraction branch includes a local temporal coding module and a global attention aggregation module; The local temporal coding module is a bidirectional long short-term memory network used to mine the dynamic details within negative and neutral eye-tracking feature sequences, generating local dynamic representations of negative eye-tracking trajectories. Local dynamic representation of neutral eye movement trajectory ; The global attention aggregation module is a multi-head self-attention network used to receive local dynamic representations of negative eye-tracking trajectories. Local dynamic representation of neutral eye movement trajectory It captures the global dependencies between any fixation points in the eye movement trajectory and generates stimulus-specific response features.
[0019] As a preferred approach, dynamic details within the sequences of negative and neutral eye movement trajectory features are mined to generate local dynamic representations of negative eye movement trajectories. Local dynamic representation of neutral eye movement trajectory The specific method is as follows: Negative eye movement trajectory features Input positive and reverse In the middle, calculate the first Hidden coding at each gaze point :
[0020] in, The first negative eye movement trajectory Each gaze feature This indicates that negative eye movement trajectories are in the positive direction. No. The hidden state of the step, This indicates that the negative eye movement trajectory is in the reverse direction. No. The hidden state of the step, This indicates that negative eye movement trajectories are in the positive direction. No. The hidden state of the step, This indicates that the negative eye movement trajectory is in the reverse direction. No. The hidden state of the step; neutral eye movement trajectory features Input positive and reverse In the middle, calculate the first Hidden coding at each gaze point :
[0021] in, The first neutral eye movement trajectory Each gaze feature This indicates that the neutral eye movement trajectory is in the positive direction. No. The hidden state of the step, Indicates that the neutral eye movement trajectory is in the reverse direction No. The hidden state of the step, This indicates that the neutral eye movement trajectory is in the positive direction. No. The hidden state of the step, Indicates that the neutral eye movement trajectory is in the reverse direction No. The hidden state of the step; With the last fixation point, i.e. the The latent encoding of each fixation point serves as the local dynamic representation of the final negative eye movement trajectory. Local dynamic representation of neutral eye movement trajectory .
[0022] As a preferred method, the specific approach for capturing the global dependencies between any fixation points in the eye movement trajectory and generating stimulus-specific response features is as follows: Local dynamic representation of negative eye movement trajectories Input to a multi-head self-attention network, the first Specific response characteristics of negative eye-movement stimulation to individual attention heads The calculation formula is:
[0023] in, , , These are learnable parameters for the self-attention mechanism. Represents the key vector dimensionality For activation functions, superscript Indicates transpose; Local dynamic representation of neutral eye movement trajectory Input to a multi-head self-attention network, the first Neutral eye-movement stimulus-specific response characteristics of individual attention heads The calculation formula is:
[0024] in, Represents the key vector The dimension; ultimately The specific response characteristics of each attentional head to negative eye movement stimulation are as follows: Neutral ocular stimulation-specific response characteristics are .
[0025] As a preferred option, the formula for calculating the assessment results of abnormal eye movement characteristics is:
[0026] in, This represents the probability of classifying eye-tracking representations. For activation function, Indicates a fully connected layer. and Indicates learnable weights, This indicates the specific response characteristics to negative eye movement stimulation. This indicates the specific response characteristics to neutral eye movement stimulation. This indicates an invariant eye movement characteristic to negative eye movement stimulation. This indicates the invariance of eye movement characteristics to neutral eye movement stimuli. , Indicates the probability of eye movement abnormalities. This indicates the probability of normal eye movements.
[0027] As a preferred option, the loss function of the eye movement anomaly assessment and classification model during training includes rank structure regularization constraints. and classification loss The formula for classification loss is:
[0028] in, Indicates the true label, This represents a logarithmic function with the natural constant as its base.
[0029] The beneficial effects of this invention are: 1. The eye movement abnormality assessment and classification method based on cross-stimulus spatiotemporal evolution characteristics provided by this invention focuses for the first time on mining eye movement abnormality patterns under different emotional stimuli. It constructs a stimulus invariance representation learning branch to model the stable internal attention patterns that individuals maintain under different emotional stimuli, and constructs a stimulus-specific response extraction branch to mine the specific attentional biases of individuals under specific emotional stimuli. It simultaneously learns the stable representation of cross-stimuli and the differential response of emotion-specificity, realizing the effective learning of cross-stimulus spatiotemporal evolution representation, thereby improving the accuracy of eye movement abnormality assessment and classification.
[0030] 2. The eye movement anomaly assessment and classification model can utilize adaptive shared basis decomposition to learn the spatiotemporal evolution characteristics of shared intrinsic attention patterns across stimuli and sample-level adaptive rank weights. It ensures the rationality of rank weights through rank structure regularization, performs low-rank reconstruction in the common representation space to generate stimulus-invariant spatiotemporal evolution feature representations, and adopts a local and global co-encoding structure to capture dynamically changing eye movement trajectory details. From both micro-gaze and macro-scanning levels, it accurately characterizes the unique attention patterns induced by each type of emotional stimulus, extracts stimulus-specific responses, and thus achieves effective modeling of the spatiotemporal evolution characteristics of eye movement trajectories, generating reliable classification results.
[0031] 3. By designing a spatial encoder that includes a linear layer, layer normalization, activation function, and random deactivation, we can effectively extract spatial distribution features from eye movement trajectories under different stimulus conditions, including key spatial patterns such as fixation point coordinates and saccade paths. This spatial encoder processes eye movement data under negative and neutral stimuli respectively, generating corresponding spatial feature representations. This provides spatially rich input for subsequent spatiotemporal evolution analysis, thereby enhancing the model's ability to perceive abnormal eye movement spatial patterns and further improving the accuracy and robustness of objective screening.
[0032] 4. By using a stimulus-invariant representation learning branch, the intrinsic attentional patterns across stimulus scenarios are decoupled. Stable and universal latent sub-patterns such as center preference and saliency-driven patterns are extracted using the shared basis decomposition method, thereby accurately capturing the invariant features of eye movement trajectories in spatiotemporal evolution. At the same time, rank weight prediction and rank structure regularization are introduced to adaptively learn a set of sparse rank weights for each sample, achieving efficient compression and reconstruction of the common representation space across stimuli, which significantly improves the model's generalization ability and interpretability for abnormal eye movement patterns.
[0033] 5. Through the adaptive shared basis decomposition module, a set of learnable shared basis vectors is constructed, effectively decoupling and modeling a general latent sub-pattern across stimulus scenarios, providing a stable feature space foundation for stimulus-invariant representation learning. This introduces a feature space based on... The adaptive rank predictor can dynamically generate sparse rank weights for each sample, achieving adaptive compression and reconstruction of the common representation space. Finally, by fusing shared basis and sample-level rank weights through low-rank reconstruction, it accurately generates stimulus invariance representations, significantly enhancing the ability to extract cross-stimulus common features in eye-tracking patterns.
[0034] 6. By introducing sparse rank constraints based on L1 regularization, the rank weights are concentrated on a few key dimensions, achieving automated rank selection. Combined with rank index-aware order constraints, the priority of important rank components is strengthened, guiding the model to organize features in a decreasing dimensionality manner, enhancing the structure and interpretability of the representation space. Simultaneously, a normalized rank sum constraint is introduced to prevent weight imbalance during training. Ultimately, through the synergistic effect of multiple constraints, the learning quality of stimulus-invariant representations is significantly improved.
[0035] 7. By extracting stimulus-specific responses from a dual perspective of micro-gazing and macro-scanning, and employing a local-global collaborative coding method, we accurately captured the spatiotemporal evolution dynamics of eye movement trajectories under different emotional stimuli. This fully characterized the unique attentional patterns induced by each type of stimulus (negative and neutral), effectively extracted stimulus-specific responses with high discriminative power, thereby enhancing the perceptual sensitivity to abnormal eye movement patterns and providing a more distinctive feature basis for subsequent objective screening tasks.
[0036] 8. The local temporal coding module adopts a bidirectional long short-term memory network to accurately model the local dynamic dependence of eye movement trajectory in the time dimension. It simulates the integration mechanism of prior experience and real-time prediction information in the human attention process, fully captures the temporal correlation between past and future moments, and generates a local dynamic representation of eye movement trajectory.
[0037] 9. The global attention aggregation module simulates the parallel cognitive process of the human brain through a multi-head self-attention mechanism, establishes a direct dependency between any two fixation points, thereby capturing the global temporal dependency in the entire trajectory, and extracting diverse global relationships including spatial and semantic ones in parallel to generate stimulus-specific response features.
[0038] 10. By introducing learnable weights to adaptively weight and sum the stimulus invariance features and stimulus-specific response features, dynamic fusion of these two complementary information types is achieved, comprehensively integrating cross-stimulus common patterns and dynamic spatiotemporal evolution features induced by emotion in eye-tracking. This is then followed by a fully connected layer and... The function maps to the probability output of eye movement abnormalities to support accurate classification.
[0039] 11. By introducing binary cross-entropy classification loss, the difference between the model's predicted probability and the actual eye movement abnormality label is accurately measured, driving the entire network to optimize learning end-to-end, ensuring that the spatiotemporal evolution features in the eye movement trajectory effectively converge toward the direction of eye movement abnormality discrimination.
[0040] Secondly, a method for assessing and classifying eye movement abnormalities based on cross-stimulus spatiotemporal evolution characteristics includes the following steps: Collect eye movement data of the target subject under negative and neutral stimuli; The eye movement trajectories under negative and neutral stimuli are input into the spatial encoder to capture the spatial information association of the eye movement trajectories and obtain the negative eye movement trajectory features and the neutral eye movement trajectory features. Negative and neutral eye movement trajectory features are input into the stimulus invariance representation learning branch. An adaptive shared basis decomposition mechanism is used to learn and reconstruct a stable low-rank subspace, and stable stimulus invariance eye movement features are captured in the common subspace. Negative and neutral eye movement trajectory features are input into the stimulus-specific response extraction branch. The dynamic details of the eye movement trajectory features are mined through local-global encoding to obtain the stimulus-specific response features triggered by each emotional stimulus. The stimulus-invariant eye movement features and stimulus-specific response features are input into the classifier, and the resulting eye movement characterization abnormality assessment results are output. Attached Figure Description
[0041] Figure 1 The diagram shows the structure of an eye movement abnormality assessment and classification system based on cross-stimulus spatiotemporal evolution characteristics.
[0042] Figure 2 The diagram shows a flowchart of an eye movement abnormality assessment and classification method based on cross-stimulus spatiotemporal evolution characteristics. Detailed Implementation
[0043] Exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the drawings are merely exemplary and are intended to illustrate the principles and spirit of the invention, and are not intended to limit the scope of the invention.
[0044] Example 1: This invention first generates a deep model for the assessment and classification of eye movement abnormalities (hereinafter referred to as the eye movement abnormality assessment and classification model). This model utilizes eye movement trajectories under different emotional stimuli, introduces a spatial encoder to extract spatial distribution features from the eye movement trajectories, employs a stimulus-invariant representation learning branch to mine intrinsic attentional patterns across stimulus scenarios, accurately captures the invariant features of eye movement trajectories in spatiotemporal evolution, and utilizes a stimulus-specific response extraction branch to capture the dynamic features of the spatiotemporal evolution of eye movement trajectories under different emotional stimuli, thus achieving accurate assessment and classification of eye movement abnormalities. In one embodiment of this invention, the eye movement abnormality assessment and classification model includes four parts: a spatial encoder, a stimulus-invariant representation learning branch, a stimulus-specific response extraction branch, and a classifier. Figure 1 As shown, an eye movement abnormality assessment and classification system based on cross-stimulus spatiotemporal evolution features includes: A spatial encoder is used to receive eye movement trajectories under negative and neutral stimuli. and By capturing the spatial information correlation of each eye movement trajectory, negative eye movement trajectory features are obtained. and neutral eye movement trajectory features ; The stimulus-invariant representation learning branch receives negative and neutral eye-tracking features and employs an adaptive shared basis decomposition mechanism to learn and reconstruct a stable low-rank subspace, capturing stable stimulus-invariant eye-tracking features within the commonality subspace. and Specifically: A1. Construct an adaptive shared basis decomposition module; wherein, the adaptive shared basis decomposition module includes: A set of learnable shared bases is used to learn basis vectors shared across stimuli in the common representation space, serving as stable universal latent subpatterns; wherein the maximum number of universal latent subpatterns constructing the common representation space is [value missing]. That is, the total number of basis vectors is preset to be Then the learnable basis vector matrix sum coefficient matrix They are respectively recorded as: , .in, Represents the basis vector matrix Feature dimensions, Represents the coefficient matrix Feature dimensions, Representing the set of real numbers, that is... It is A real matrix, It is A real matrix.
[0045] An adaptive rank predictor based on the Sigmoid activation function is used to generate adaptive rank weights. The adaptive rank predictor consists of a linear layer, layer normalization, activation function, global average pooling, a linear layer, another activation function, another linear layer, and so on, connected in sequence. The activation function represents the negative eye movement trajectory features as: Adaptive rank weights The following was generated:
[0046]
[0047] in, , , Represents a linear layer. Representative level normalization, express Activation function This represents the features before global average pooling. The length of the eye-tracking trajectory, i.e., the number of fixation points, is obtained by global average pooling along the length dimension to obtain the feature. Subsequently through Activation function, generating rank weights Similarly, the neutral eye-tracking trajectory features are represented as follows: Adaptive rank weights The following was generated:
[0048]
[0049] in, The features represent those before global average pooling. These features are obtained by performing global average pooling along the length dimension. Subsequently through Activation function, generating rank weights Ultimately, each sample predicts a set of rank weights. and , , Each value is between 0 and 1 to control the contribution of each basis vector. If some weights are close to zero, the actual effective rank will be lower than [a certain value]. ; A low-rank reconstruction module integrating shared basis and rank weights is used to adaptively select a universal latent sub-pattern based on the effective rank, generating the final stimulus-invariant representation; wherein, the input basis vector matrix... sum coefficient matrix and rank weight and Generate stimulus invariance representations and The formula is as follows:
[0050]
[0051] in, and They are respectively and The transpose of .
[0052] A2. Implement rank structure regularization constraints, specifically: Based on the sparse rank constraint of the L1 regularization term, the rank weight distribution is encouraged to be concentrated in a small number of dimensions, forcing the weights of unimportant factors to approach 0, thus achieving automatic rank selection; the rank weights... and Collectively referred to as , Given a preset total number of basis vectors, sparse rank constraint. The formula is:
[0053] Based on rank index-aware rank order constraints, the order consistency of important ranks is strengthened, and encouragement is provided. The value decreases along the dimension. For example, later rank penalties are stronger, thus guiding the model to prioritize the use of lower-rank components; the rank weights are... and Collectively referred to as , The total number of basis vectors is the preset number. For sequential indexes, rank order constraint The formula is:
[0054] Consider the rank-sum constraint of normalization, constraint The sum of the values is 1 to prevent the weights from getting out of control; the rank weights are set to 1. and Collectively referred to as , The total number of basis vectors is the preset number. For identity matrix, rank sum constraint The formula is:
[0055] Then the global constraints of rank structure regularity The calculation is as follows: .
[0056] The stimulus-specific response extraction branch receives negative and neutral eye movement trajectory features and mines the dynamic details of the eye movement trajectory features through local-global encoding to obtain the stimulus-specific response features evoked by each emotional stimulus. and Specifically: B1. Construct a local temporal coding module; the local temporal coding module is a bidirectional long short-term memory network used to fully mine the dynamic details within the eye movement trajectory sequence to represent the local dynamic dependencies of the eye movement trajectory; among them, negative eye movement trajectory features... Input forward model and inverse model In the middle, in the Hidden coding at each gaze point The calculation formula is as follows:
[0057] in The first negative eye movement trajectory Each fixation point feature. Similarly, neutral eye movement trajectory features. Input forward model and inverse model In the middle, in the Hidden coding at each gaze point The calculation formula is as follows:
[0058] in The first neutral eye movement trajectory The first fixation feature. The last fixation, i.e., the first... The latent coding of each fixation point is used to represent the final local dynamic representation of the eye movement trajectory, denoted as... .
[0059] B2. Construct a global attention aggregation module; the global attention aggregation module is a multi-head self-attention network used to capture more complex global dependencies and obtain context-rich global modeling; input negative eye-tracking trajectory local dynamic representation. , No. The formula for calculating the number of attention heads is:
[0060] in , , These are learnable parameters for the self-attention mechanism. Represents the key vector The dimension, through Activation function, to obtain the first Specific response characteristics of negative eye-movement stimulation to individual attention heads Similarly, local dynamic representation of input neutral eye-tracking trajectory. ,pass Activation function, to obtain the first Neutral eye-movement stimulus-specific response characteristics of individual attention heads The calculation formula is:
[0061] final The stimulus-specific response characteristics of each attention head are as follows: and .
[0062] A classifier receives stimulus-invariant eye movement features and stimulus-specific response features, and outputs an assessment result of abnormal eye movement representations. The classifier consists of a weighted summation operation and a fully connected layer, with stimulus-invariant eye movement features as input. and and stimulus-specific response characteristics and The final classification probability The generating formula is:
[0063] in, and For learnable weights, It is a fully connected layer. (Through...) Activation function to obtain classification probability ,in This represents the probability of eye movement abnormalities. This represents the probability of normal eye movements.
[0064] In this embodiment, the parameters of the eye movement abnormality assessment and classification model are updated using a loss function until the model converges. During the training process, the eye movement abnormality assessment and classification model is subjected to global constraints through rank structure regularization. With classification loss function The network parameters are updated by summing the results until the eye movement anomaly assessment and classification model finally converges completely. In one embodiment of the invention, Adam is used as the optimizer during training, and a scheduling strategy with warm start and cosine annealing is employed. Classification loss function The formula for expressing this is:
[0065] in This is a real label.
[0066] Example 2: Building upon Example 1, this invention was applied to real-world eye movement data from patients with depression and healthy subjects to verify the practical effectiveness of the eye movement abnormality assessment and classification method provided by this invention. Specifically, eye movement data from 65 patients with severe depression from a hospital and 27 healthy control group members from a hospital and a university were selected. All eye movement data were collected under a free viewing task, with each subject viewing 60 single-emotion facial stimulus images, including 30 neutral expression facial images as neutral stimuli and 30 sad expression facial images as negative stimuli. 60 raw eye movement trajectory data were collected from each subject, and corresponding fixation point data were extracted. After preprocessing including trajectory length standardization and outlier cleaning, each eye movement trajectory included 8 fixation points, including fixation point order, coordinates, duration, and pupil size. To ensure fair comparison with other methods, this invention used five-fold cross-validation, employing evaluation metrics including accuracy (ACC), precision (Precision), area under the curve (AUC), and F1 score for comprehensive evaluation, and calculating the average accuracy of the five-fold cross-validation. The results are shown in Table 1.
[0067] Table 1. Results of five-fold cross-validation
[0068] As can be seen from Table 1, the present invention can achieve excellent performance in real eye movement abnormality assessment and classification tasks, and has good practical value.
[0069] Example 3: Based on Example 1, this embodiment of the invention provides a method for assessing and classifying eye movement abnormalities based on cross-stimulus spatiotemporal evolution characteristics, such as... Figure 2 As shown, it includes the following steps: S1. Collect eye movement data of the target subject under negative and neutral stimuli; S2. Input the eye movement trajectories under negative and neutral stimuli into the spatial encoder, capture the spatial information association of the eye movement trajectories, and obtain the negative eye movement trajectory features and the neutral eye movement trajectory features; S3. Input the negative eye movement trajectory features and the neutral eye movement trajectory features into the stimulus invariant representation learning branch, and use the adaptive shared basis decomposition mechanism to learn and reconstruct a stable low-rank subspace, and capture stable stimulus invariant eye movement features in the common subspace. S4. Input the negative and neutral eye movement trajectory features into the stimulus-specific response extraction branch, and mine the dynamic details of the eye movement trajectory features through local-global encoding to obtain the stimulus-specific response features triggered by each emotional stimulus; S5. Input the stimulus-invariant eye movement features and stimulus-specific response features into the classifier, and output the results of the abnormal eye movement representation assessment.
[0070] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.
Claims
1. A classification system for assessing eye movement abnormalities based on cross-stimulus spatiotemporal evolution characteristics, characterized in that, include: A spatial encoder is used to receive eye movement trajectories under negative and neutral stimuli, capture the spatial information correlation of each eye movement trajectory, and obtain negative eye movement trajectory features and neutral eye movement trajectory features. The stimulus invariance representation learning branch is used to receive negative and neutral eye movement trajectory features. An adaptive shared basis decomposition mechanism is used to learn and reconstruct a stable low-rank subspace, and to capture stable stimulus invariant eye movement features in the common subspace. The stimulus-specific response extraction branch is used to receive negative and neutral eye movement trajectory features, and to mine the dynamic details of eye movement trajectory features through local-global encoding to obtain the stimulus-specific response features triggered by each emotional stimulus. A classifier is used to receive stimulus-invariant eye movement features and stimulus-specific response features, and outputs the results of the abnormal assessment of eye movement representation.
2. The eye movement abnormality assessment and classification system based on cross-stimulus spatiotemporal evolution characteristics according to claim 1, characterized in that, The adaptive shared basis decomposition mechanism is as follows: Stable universal latent subpatterns are obtained by learning basis vectors shared across stimuli in the common representation space using a set of learnable shared bases; where the maximum number of universal latent subpatterns is [number missing]. That is, the total number of basis vectors is preset to be Learnable basis vector matrix sum coefficient matrix They are respectively recorded as: , ,in, Represents the basis vector matrix Feature dimensions, Represents the coefficient matrix Feature dimensions, Represents the set of real numbers; Based on negative eye movement trajectory features Adaptive rank weights are generated using an adaptive rank predictor based on the Sigmoid activation function. : in, , , Indicates a linear layer. Representation layer normalization, express Activation function Indicates negative eye movement trajectory features through linear layers Layer normalization and Features after activation function processing Indicates the length of the eye-track feature. This indicates the feature in the length dimension. Features obtained by global average pooling. express Activation function; Based on neutral eye-tracking features Adaptive rank weights are generated using an adaptive rank predictor based on the Sigmoid activation function. : in, Indicating neutral eye movement trajectory features via linear layers Layer normalization and Features after activation function processing This indicates the feature in the length dimension. Features obtained by global average pooling; Based on rank weight and Adaptive selection of general latent sub-patterns to generate stimulus-invariant eye-tracking features and : in, and They are respectively and transpose, This indicates element-wise multiplication.
3. The eye movement abnormality assessment and classification system based on cross-stimulus spatiotemporal evolution characteristics according to claim 2, characterized in that, Stimulus invariance means that the learning branch introduces rank structure regularization constraints during training. Specifically, this includes sparse rank constraints based on L1 regularization terms. Rank order constraints based on rank index awareness and considering the rank sum constraint of normalization The corresponding formula is: in, Represents rank weights, including and , Represents the identity matrix. Indicates the first Each weighting coefficient Indicates a sequential index.
4. The eye movement abnormality assessment and classification system based on cross-stimulus spatiotemporal evolution characteristics according to claim 1, characterized in that, The stimulus-specific response extraction branch includes a local temporal encoding module and a global attention aggregation module; The local temporal coding module is a bidirectional long short-term memory network used to mine the dynamic details within negative and neutral eye-tracking feature sequences, generating local dynamic representations of negative eye-tracking trajectories. Local dynamic representation of neutral eye movement trajectory ; The global attention aggregation module is a multi-head self-attention network used to receive local dynamic representations of negative eye-tracking trajectories. Local dynamic representation of neutral eye movement trajectory It captures the global dependencies between any fixation points in the eye movement trajectory and generates stimulus-specific response features.
5. The eye movement abnormality assessment and classification system based on cross-stimulus spatiotemporal evolution characteristics according to claim 4, characterized in that, By mining the dynamic details within negative and neutral eye movement trajectory feature sequences, a local dynamic representation of the negative eye movement trajectory is generated. Local dynamic representation of neutral eye movement trajectory The specific method is as follows: Negative eye movement trajectory features Input positive and reverse In the middle, calculate the first Hidden coding at each gaze point : in, The first negative eye movement trajectory Each gaze feature This indicates that negative eye movement trajectories are in the positive direction. No. The hidden state of the step, This indicates that the negative eye movement trajectory is in the reverse direction. No. The hidden state of the step, This indicates that negative eye movement trajectories are in the positive direction. No. The hidden state of the step, This indicates that the negative eye movement trajectory is in the reverse direction. No. The hidden state of the step; neutral eye movement trajectory features Input positive and reverse In the middle, calculate the first Hidden coding at each gaze point : in, The first neutral eye movement trajectory Each gaze feature This indicates that the neutral eye movement trajectory is in the positive direction. No. The hidden state of the step, Indicates that the neutral eye movement trajectory is in the reverse direction No. The hidden state of the step, This indicates that the neutral eye movement trajectory is in the positive direction. No. The hidden state of the step, Indicates that the neutral eye movement trajectory is in the reverse direction No. The hidden state of the step; With the last fixation point, i.e. the The latent encoding of each fixation point serves as the local dynamic representation of the final negative eye movement trajectory. Local dynamic representation of neutral eye movement trajectory .
6. The eye movement abnormality assessment and classification system based on cross-stimulus spatiotemporal evolution characteristics according to claim 4, characterized in that, The method for capturing global dependencies between arbitrary fixation points in eye-tracking trajectories and generating stimulus-specific response features is as follows: Local dynamic representation of negative eye movement trajectories Input to a multi-head self-attention network, the first Specific response characteristics of negative eye-movement stimulation to individual attention heads The calculation formula is: in, , , These are learnable parameters for the self-attention mechanism. Represents the key vector dimensionality For activation functions, superscript Indicates transpose; Local dynamic representation of neutral eye movement trajectory Input to a multi-head self-attention network, the first Neutral eye-movement stimulus-specific response characteristics of individual attention heads The calculation formula is: in, Represents the key vector The dimension; ultimately The specific response characteristics of each attentional head to negative eye movement stimulation are as follows: Neutral ocular stimulation-specific response characteristics are .
7. The eye movement abnormality assessment and classification system based on cross-stimulus spatiotemporal evolution characteristics according to claim 1, characterized in that, The formula for calculating the assessment results of abnormal eye movement representations is as follows: in, This represents the probability of classifying eye-tracking representations. For activation function, Indicates a fully connected layer. and Indicates learnable weights, This indicates the specific response characteristics to negative eye movement stimulation. This indicates the specific response characteristics to neutral eye movement stimulation. This indicates an invariant eye movement characteristic to negative eye movement stimulation. This indicates the invariance of eye movement characteristics to neutral eye movement stimuli. , Indicates the probability of eye movement abnormalities. This indicates the probability of normal eye movements.
8. The eye movement abnormality assessment and classification system based on cross-stimulus spatiotemporal evolution characteristics according to claim 7, characterized in that, The loss function used in the training process of the eye movement anomaly assessment and classification model includes rank structure regularization constraints. and classification loss The formula for classification loss is: in, Indicates the true label, This represents a logarithmic function with the natural constant as its base.
9. An eye movement abnormality assessment and classification method based on the cross-stimulus spatiotemporal evolution characteristics of an eye movement abnormality assessment and classification system according to any one of claims 1-8, characterized in that, Includes the following steps: Collect eye movement data of the target subject under negative and neutral stimuli; The eye movement trajectories under negative and neutral stimuli are input into the spatial encoder to capture the spatial information association of the eye movement trajectories and obtain the negative eye movement trajectory features and the neutral eye movement trajectory features. Negative and neutral eye movement trajectory features are input into the stimulus invariance representation learning branch. An adaptive shared basis decomposition mechanism is used to learn and reconstruct a stable low-rank subspace, and stable stimulus invariance eye movement features are captured in the common subspace. Negative and neutral eye movement trajectory features are input into the stimulus-specific response extraction branch. The dynamic details of the eye movement trajectory features are mined through local-global encoding to obtain the stimulus-specific response features triggered by each emotional stimulus. The stimulus-invariant eye movement features and stimulus-specific response features are input into the classifier, and the resulting eye movement characterization abnormality assessment results are output.