Intelligent depression assessment method and system based on ERP and eye movement data fusion model

Abstract

The invention relates to an intelligent depression assessment method and system based on an ERP and eye movement data fusion model. The method comprises the steps that a data acquisition module obtains psychological scale data, multiple ERP indexes and multiple eye movement indexes; the data analysis module is used for screening ERP (Enterprise Resource Planning) and eye movement indexes significantly related to depressive symptoms; the feature fusion module is used for fusing the screened ERP data and eye movement data and splicing the data into a fusion feature vector, and the model training module is used for inputting the fusion feature vector into a depression dichotomy recognition model constructed by combining Transform and an artificial neural network (ANN) algorithm and outputting a depression risk assessment result; and the prediction output module is used for predicting the data acquired by the new user by adopting the trained model and outputting a depression risk assessment result through a visual interface. According to the system, the technical bottleneck problems of subjective defect of traditional psychological scale evaluation, insufficient sensitivity and specificity of single physiological index detection and multi-modal data fusion are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent medical treatment and mental health assessment, and specifically relates to an intelligent depression assessment method and system based on an ERP and eye movement data fusion model. Background Art

[0002] Depression is the most common mental health problem. Currently, the screening and identification of depression mainly relies on interviews with school teachers and psychological screening scales. These scales are subjective and time-sensitive, and are easily affected by short-term emotions. Multiple measurements require certain professionals to organize and implement multiple times, which requires a lot of time and energy investment, and it is difficult to meet the large-scale, rapid and objective screening needs in school scenarios.

[0003] Existing technologies, single physiological indicator detection methods suffer from low sensitivity and insufficient specificity, while existing fusion solutions face technical bottlenecks such as inadequate handling of heterogeneous multimodal data and poor model interpretability. Furthermore, existing depression assessment models typically only provide a binary classification of depression / non-depression and lack the ability to quantitatively categorize depression severity. In clinical practice, mild, moderate, and severe depression require differentiated intervention strategies. Therefore, a technical solution that can further refine severity classification beyond this binary classification is urgently needed. Summary of the Invention

[0004] (1) Technical issues to be resolved

[0005] In view of the above-mentioned shortcomings and deficiencies of the existing technology, the present invention provides an intelligent depression assessment method and system based on an ERP and eye movement data fusion model, which solves the subjective defects of traditional psychological scale assessment, the insufficient sensitivity and specificity of single physiological indicator detection, and the technical bottleneck problems of multimodal data fusion.

[0006] (2) Technical solution

[0007] In order to achieve the above object, the main technical solutions adopted by the present invention are as follows:

[0008] In a first aspect, an embodiment of the present invention provides an intelligent depression assessment system based on an ERP and eye movement data fusion model, comprising:

[0009] The data acquisition module is used to collect the user's psychological scale data, obtain multiple ERP indicators through the evoked potential meter, and obtain multiple eye movement indicators through the eye tracker;

[0010] Data analysis module, used to screen ERP and eye movement indicators that are significantly related to depressive symptoms;

[0011] The feature fusion module is used to fuse the filtered ERP data and eye movement data into a fused feature vector as the input feature of the model;

[0012] The model training module is used to input the fused feature vector of each trainee into the depression binary classification recognition model constructed by combining the Transformer and ANN algorithms during the training phase, train the parameters of the depression binary classification recognition model, and obtain the trained depression binary classification recognition model;

[0013] For example, the input features (ERP + eye movement) in the feature fusion module are preliminarily processed by the ANN to obtain feature data processed by the ANN. This processed feature data is then converted into a data format that can be recognized by the Transformer. The Transformer encoder then enhances the feature expression to obtain new feature data processed by the Transformer. This new feature data and the feature data processed only by the ANN are then input into the ANN hidden layer for model training, and finally the depression risk assessment results are output;

[0014] The prediction output module uses the trained model to predict the fused feature vectors of new users and outputs the depression risk assessment results through a visual interface. Furthermore, the system uses a depression membership function to map the depression probability values output by the model into a continuous risk scale, including mild, moderate, and severe depression, providing a quantitative basis for clinical intervention.

[0015] In other words, the prediction output module uses the trained model to predict data collected from new users based on binary classification. Using the depression membership function, the probability values are further divided into three risk levels: mild, moderate, and severe. Finally, the graded assessment results are output through a visual interface. This system addresses the subjective limitations of traditional psychological scale assessments, the insufficient sensitivity and specificity of single physiological indicators, and the technical bottlenecks of multimodal data fusion. Furthermore, by quantifying the severity of depression and grading it, it provides an objective basis for differentiated interventions.

[0016] Specifically, obtain the depression probability value output by the depression binary classification recognition model in

[0017] The depression membership is calculated by the following membership function: D :

[0018]

[0019] μ D is depression membership; is the depression probability value (between 0 and 1) output by the depression identification model; k is the adjustment parameter that controls the steepness of membership change, and its value range is 2≤k≤10; θ is the risk critical threshold, defined as the depression risk cutoff point, and its default value is 0.5.

[0020] According to μ D Values are divided into risk levels:

[0021] If 0 < μ D ≤0.3, judged as mild depression;

[0022] If 0.3<μ D ≤0.7, judged as moderate depression;

[0023] If 0.7<μ D ≤1, judged as severe depression.

[0024] Optionally, the data acquisition module is specifically used to:

[0025] The psychological scale data of the depression assessment scale PHQ-9 were collected through the WeChat mini program;

[0026] And / or, obtain multiple ERP indicators through evoked potential monitoring, including:

[0027] Auditory evoked potential (AEP) data obtained by sound stimulation using the oddball paradigm, including: the latency and amplitude of the N100, P200, N200, and P300 components recorded at the parietal electrode Cz; the latency and amplitude of the N100, P200, N200, and P300 components recorded at the forehead midline electrode Fpz; Visual evoked potential (VEP) data obtained using the checkerboard flip pattern, including: the latency and amplitude of the N75, P100, and N145 components of the right eye; the latency and amplitude of the N75, P100, and N145 components of the left eye;

[0028] And / or, use an eye tracker to obtain multiple eye movement metrics, including:

[0029] Responsiveness Exploration Score RSS: RSS1, RSS2 and RSS1+RSS2;

[0030] Eye fixation parameters: number of viewpoints NEF;

[0031] Discriminant score D, D = 10.265-0.065×NEF-0.871×RSS;

[0032] Total fixation time TESL and mean fixation time MESL.

[0033] Optionally, the data analysis module is specifically used to:

[0034] Analysis of differences between the depression group and the healthy control group:

[0035] X represents ERP or eye movement index, corresponding to the group variable G:

[0036] G = {1, depression group (PHQ-9 ≥ 5); 0, healthy control group (PHQ-9 < 5)}

[0037] Using the Mann-Whitney U test:

[0038] H0: Median(X|G=1) = Median(X|G=0)

[0039] H1: Median(X|G=1)≠Median(X|G=0)

[0040] Statistical significance (p value) was calculated using the Wilcoxon rank sum test, and the screening criterion was set at p < 0.05;

[0041] Output: Significant difference indicator set F_diff={X1,X2,...,X_k}

[0042] And / or, correlation analysis:

[0043] In the depression group, a Pearson correlation analysis was performed between the indicator X_i in F_diff and the PHQ-9 total score Y:

[0044] Calculate the correlation coefficient: r_{X_iY}=Pearson(X_i,Y);

[0045] Test hypothesis: H0: ρ_{X_iY}=0;

[0046] H1: ρ_{X_iY}≠0;

[0047] Statistical significance (p 1 value) was calculated using the Spearman correlation test, and the screening criterion was set at p 1 < 0.05;

[0048] Output: Significant correlation index set F_corr={X_{i1},X_{i2},...,X_{im}}

[0049] Feature screening results: The features that are finally fused in the next step are the indicators in F_corr.

[0050] Optionally, the feature fusion module is specifically used to: first perform feature normalization processing on the filtered ERP data and eye movement data, and then perform multimodal feature fusion to obtain a fused feature vector;

[0051] Feature standardization: Z-score standardization is performed on the 28-dimensional ERP features and the 15-dimensional eye movement features to eliminate dimensional differences:

[0052]

[0053] X erp : 28-dimensional original ERP feature vector; X eye : 15-dimensional original eye movement feature vector;

[0054] μ erp ,μ eye They are: the mean of ERP / eye movement features in the training set; σ erp ,σ eye They are: standard deviation of ERP / eye movement features in the training set; X′ erp ,X′ eye They are: standardized ERP / eye movement features.

[0055] Optionally, the multimodal feature fusion of the feature fusion module includes:

[0056] Weighted splicing fusion: Calculate feature weights based on the Pearson correlation coefficient and splice the standardized features according to the weights:

[0057] r i =Pearson(X i ,PHQ-9)

[0058] Fusion vector:

[0059] r i : Pearson correlation coefficient between the ith feature and the total score of PHQ-9; k: total number of features, 43 dimensions;

[0060] w i : Normalized weight of the i-th feature, ∑w i =1;F fused : 43-dimensional feature vector after weighted fusion;

[0061] When the initial eigenvector F with biostatistical significance is obtained fused Finally, a dual deep learning framework (ANN+Transformer) is used to perform in-depth analysis on the initial fusion features. ANN feature processing is first performed, and then Transformer feature processing is performed on top of the ANN feature processing. The ANN-processed features and the Transformer-processed features are then input into the ANN hidden layer together until the final classification task is performed.

[0062] As a front-end module, ANN first performs nonlinear mapping on the input fusion features (ERP + eye movement):

[0063] y=f(W·F used +b)

[0064] Where W is the weight matrix, b is the bias vector, and f is the nonlinear activation function. Here we use ReLU

[0065] Activation function, y is the output vector (the fused features are transformed according to the set hidden layer size to obtain the flat vector y). After this transformation, the representation ability of the original input data can be improved.

[0066] Since the output of ANN is usually a flat feature vector with no time step and no sequence structure, it cannot be directly input into Transformer. In this patent, before entering the Transformer module, the feature data y processed by ANN is transformed by reshape, and the input data is processed into a data format X that can be recognized by the subsequent Transformer. seq ∈R B ×L×d , where B is the batch size, L is the sequence length, and d is the feature dimension of each position. Then, position encoding is performed to obtain data containing position information:

[0067]

[0068] Where PE is a learnable position code.

[0069] The position-encoded input is fed into a multi-head self-attention layer to model the dependencies between temporal features:

[0070]

[0071] in,

[0072]

[0073] Represent the query vector, key vector, and value vector respectively, and W Q ,W K ,W V They represent the trainable weight matrices corresponding to Q, K, and V, respectively, and are responsible for projecting the original vectors of the input sequence into different spaces to generate query, key, and value vectors. T represents the matrix transpose.

[0074] d k Represents the key vector dimension. Multiple attention heads are executed in parallel and then linearly mapped. Each attention head generates a set of Q i ,K i ,V i, calculate the attention score. In practical applications, the number of attention heads is set according to actual needs. For the output of n attention heads:

[0075]

[0076] Where seq_len represents the sequence length,

[0077]

[0078] Multiple heads are spliced into a whole:

[0079] MultiHead(Q,K,V)=Concat(Head1,Head2,...,Head n )W 0

[0080] Here is a linear transformation matrix.

[0081] The data aggregated by the multi-head attention mechanism goes through the following steps before average pooling:

[0082] Residual connection + LayerNorm, feedforward network, residual + LayerNorm again;

[0083] After these steps, we enter the average pooling to get a double-processed (ANN→Transformer vector, and then combine the Transformer-processed vector with the ANN-processed initial decision vector F fused The vectors are merged and input into the hidden layer of ANN. The features output by the hidden layer are then mapped to the depression risk label space. The label weight is calculated and the probability value of depression risk is output through the activation function Sigmoid. Finally, the classification result is obtained. At the same time, the fusion feature (ERP + eye movement) F is also performed. fused Directly enter the ANN module to obtain the classification results, and directly use the data processed by ANN to Transformer as the input of ANN to obtain the classification results.

[0084] Optionally, the system further comprises:

[0085] A model building module is used to build a depression recognition model, namely a depression binary classification recognition model, in a Python 3.10 environment using the scikit-learn package and other deep learning frameworks. The model combines the Transformer and ANN algorithms.

[0086] The depression binary classification recognition model specifically includes:

[0087] Input layer: receives the weighted concatenated fusion feature vector to capture the rich information of the input data;

[0088] Transformer encoder: First, the input features are reshaped and positionally encoded to capture the position information of the input data; then a multi-head attention mechanism is implemented, with multiple attention heads processed in parallel to generate multiple different context representations. Each attention head processes the input features through query (Q), key (K), and value (V) weight calculations, and an enhanced feature representation is obtained through weighted fusion; then the outputs of multiple heads are aggregated, and the aggregated results are sequentially passed through residual connection + LayerNorm, feedforward network, residual + LayerNorm again, and finally average pooling to generate the output features of the Transformer.

[0089] Hidden layer: The output features of the Transformer and the initial fusion features are passed to the ANN part. The ANN consists of multiple fully connected layers and is configured to process the extracted deep features. It uses the ReLU activation function for nonlinear transformation.

[0090] Decision and output layer: maps the features output by the hidden layer to the depression risk label space, calculates the label weight, and outputs the probability value of depression risk through the Sigmoid activation function;

[0091] The grid search method was used to optimize the hyperparameter configuration of the depression recognition model composed of Transformer and ANN to improve the recognition accuracy of the model.

[0092] Optionally, the loss function used by the model training module in model training is binary cross entropy (BinaryCross-Entropy, BCE Loss):

[0093]

[0094] Where N: the number of training samples;

[0095] y i : The true label of the i-th sample, 0 = healthy, 1 = depressed;

[0096] The depression probability of the i-th sample predicted by the model;

[0097] The optimizer used in the model training module is Adam, with an initial learning rate of 0.001.

[0098] Early stopping method: terminate training when the validation set loss does not decrease for 5 consecutive rounds;

[0099] Performance evaluation parameters after model training: including receiver operating characteristic curve (ROC), area under the curve (AUC), accuracy, precision, recall, and F1 value;

[0100] In a second aspect, an embodiment of the present invention further provides an intelligent depression assessment method based on an EPR and eye movement data fusion model, comprising:

[0101] Collect users' psychological scale data, multiple ERP indicators, and multiple eye movement indicators;

[0102] Screening ERP and eye movement indicators that are significantly associated with depressive symptoms;

[0103] The filtered ERP data and eye movement data are fused and spliced into a fused feature vector as the input feature of the model;

[0104] During the training phase, the fused feature vector of each trainee is input into a depression binary classification recognition model constructed based on the ANN algorithm, the parameters of the depression binary classification recognition model are trained, and the trained depression binary classification recognition model is obtained;

[0105] The trained model is used to predict the fused feature vector of the new user, and the depression risk assessment results are output through a visual interface. The fused feature vector of the new user is obtained by fusion of multiple ERP indicators and multiple eye movement indicators.

[0106] Optionally, collect the user's psychological scale data, multiple ERP indicators, and multiple eye movement indicators, including:

[0107] The psychological scale data of the depression assessment scale PHQ-9 were collected through the WeChat mini program;

[0108] Multiple ERP indicators were obtained through evoked potential monitoring, including:

[0109] Auditory evoked potential (AEP) data obtained by sound stimulation using the oddball paradigm, including: the latency and amplitude of the N100, P200, N200, and P300 components recorded at the parietal electrode Cz; the latency and amplitude of the N100, P200, N200, and P300 components recorded at the forehead midline electrode Fpz; Visual evoked potential (VEP) data obtained using the checkerboard flip pattern, including: the latency and amplitude of the N75, P100, and N145 components of the right eye; the latency and amplitude of the N75, P100, and N145 components of the left eye;

[0110] The eye tracker can be used to obtain multiple eye movement indicators, including:

[0111] Responsiveness Exploration Score RSS: RSS1, RSS2 and RSS1+RSS2;

[0112] Eye fixation parameters: number of viewpoints NEF;

[0113] The formula for calculating the discriminant score D is D = 10.265-0.065×NEF-0.871×RSS;

[0114] Total fixation time TESL and mean fixation time MESL.

[0115] (III) Beneficial effects In the present invention, objective ERP and eye movement data are fused with multimodal data of psychological scales, and an intelligent recognition model for depression is established based on the ANN+Transformer dual feature processing algorithm, thereby achieving objective and efficient screening of depression status in student groups. This overcomes the limitations of traditional methods that cannot objectively evaluate depression and single-modal data recognition methods, effectively solves the problem of shortage of professional psychological staff, improves the accuracy and robustness of depression recognition, and is suitable for psychological health assessment of large-scale student groups. It has certain technical innovation and practical application value. It systematically overcomes the technical difficulties such as poor reliability of subjective evaluation, large limitations of single-modal detection, and bottlenecks in multimodal fusion technology, providing a set of efficient and reliable intelligent solutions for school mental health screening.

[0116] In this invention, by implementing intelligent depression assessment by collecting ERP and eye movement data, it is possible to accurately identify depressive symptoms, thereby improving the accuracy and robustness of depression screening. This solves the problems of strong subjectivity and unimodal data limitations of traditional methods, breaks through the limitations of traditional binary classification models, and realizes automated grading of depression severity based on physiological data, achieving efficient and objective depression screening. Through technological innovation, this invention has achieved a leap forward in depression screening from "subjective experience judgment" to "objective data evaluation", providing reliable technical support for the establishment of a scientific and efficient student mental health monitoring system, and can provide a scientific basis for teachers, education management departments, and government decision-making in school mental health management.

[0117] In the embodiment of the present invention, an advanced artificial intelligence algorithm is used to construct a recognition model, which can improve the accuracy of depression identification in student groups and conduct large-scale mental health screening for student groups. BRIEF DESCRIPTION OF THE DRAWINGS

[0118] Figure 1 A schematic diagram of a module of an intelligent depression assessment system based on an ERP and eye movement data fusion model provided by one embodiment of the present invention;

[0119] Figure 2 This is an architecture diagram of an intelligent depression assessment system based on an ERP and eye movement data fusion model provided by one embodiment of the present invention;

[0120] Figure 3A schematic diagram of the model architecture;

[0121] Figure 4 and Figure 5 These are schematic diagrams of the results of evaluating the evaluation parameters during the model training process;

[0122] Figure 6 This is a schematic diagram of SHAP's interpretability analysis of the model used in this application. DETAILED DESCRIPTION

[0123] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation methods in conjunction with the accompanying drawings.

[0124] ERP and eye tracking are closely related to psychological factors and can reflect the dynamic cognitive processes of brain nerve cells. They are simple to use, non-invasive, and automatically processed by the brain, and are gradually being used as objective biological indicators for assessing cognitive function in individuals with depression. ERP is a series of EEG activity caused by the brain's processing of received information, identification, discrimination, and judgment of stimuli. It primarily reflects changes in the brain's electrical potential when processing information such as attention and memory during cognitive tasks, and is sensitive to changes in an individual's internal response to events or stimuli.

[0125] Eye tracking is a technique used to study individual attention and cognitive processes by recording and analyzing eye movements. Eye movements provide information not only on task processing but also on eye movement control, such as speed, duration, and saccade trajectory, helping to reveal individual cognitive processing processes and cognitive processing deficits.

[0126] With the application and development of artificial intelligence technology, it has become possible to automatically identify depressed student groups by collecting ERP and eye movement data of depressed individuals and performing intelligent analysis.

[0127] The embodiments of the present invention provide a depression identification method and system based on a multimodal data fusion model of ERP, eye movement data and psychological scales to alleviate the problems existing in the prior art.

[0128] Example 1

[0129] The embodiment of the present invention provides an intelligent depression assessment system based on the fusion model of ERP and eye movement data. Figures 1 to 3 As shown, the method of this embodiment may include the following steps:

[0130] The data acquisition module is used to collect the user's psychological scale data, obtain multiple ERP indicators through the evoked potential meter, and obtain multiple eye movement indicators through the eye tracker.

[0131] For example, psychological scale data was collected through the WeChat mini-program, 28 ERP indicators were obtained through the evoked potential meter, and 15 eye movement indicators were obtained through the eye tracker.

[0132] Data analysis module, used to screen ERP and eye movement indicators that are significantly related to depressive symptoms;

[0133] For example, the data from the training phase of this embodiment can be subjected to inter-group difference analysis and correlation analysis between the depression group and the healthy control group to screen ERP and eye movement indicators that are significantly correlated with depressive symptoms.

[0134] The feature fusion module is used to fuse the filtered ERP data and eye movement data and splice them into a fused feature vector, which can be used as an input feature in the model training stage or prediction stage.

[0135] For example, the filtered ERP data and eye movement data may be first subjected to feature normalization processing, and after feature normalization, they may be fused using a weighted splicing fusion method or an attention enhancement fusion method.

[0136] The model training module is used to input the fused feature vector of each trainee into the depression binary classification recognition model constructed by combining the Transformer and ANN algorithms during the training phase, train the parameters of the depression binary classification recognition model, and obtain the trained depression binary classification recognition model.

[0137] It should be noted here that in the training phase, the data acquisition module, data analysis module and feature fusion module all process the data of the training users to obtain the feature fusion vector of the training phase. In the prediction phase, the data acquisition module, data analysis module and feature fusion module all process the data of the predicted users to obtain the feature fusion vector used in the prediction phase.

[0138] The prediction output module uses the trained model to predict the data collected from new users and outputs the depression risk assessment results through a visual interface. Furthermore, the system uses a depression membership function to map the depression probability values output by the model into a continuous risk level, including mild, moderate, and severe depression, providing a quantitative basis for clinical intervention.

[0139] The method of this embodiment can effectively predict depressive symptoms in college students. The promotion and application of this system will help establish a closed-loop management mechanism of "screening-assessment-intervention" and realize a precise mental health service model from group screening to individualized intervention. It has important practical value for improving the early identification rate of student depression and optimizing the allocation of medical resources.

[0140] This example innovatively integrates the high temporal resolution of ERP and the spatial behavioral characteristics of eye movement data, combined with deep learning feature extraction and attention mechanisms, to build an intelligent depression assessment system suitable for school settings. Through multimodal feature fusion and interpretable AI algorithms, it achieves objective assessment of depression risk, effectively addressing the objectivity, timeliness, and operability shortcomings of traditional methods, and providing an efficient and reliable intelligent solution for school mental health screening.

[0141] In this embodiment, the data acquisition module uses an evoked potential meter to collect event-related potential data, for example, auditory evoked potential (AEP) data obtained by sound stimulation using the oddball paradigm, including: the latency and amplitude of the N100, P200, N200, and P300 components recorded by the parietal electrode (Cz); the latency and amplitude of the N100, P200, N200, and P300 components recorded by the mid-frontal electrode (Fpz);

[0142] Visual evoked potential (VEP) data obtained using the checkerboard reversal pattern include: the latency and amplitude of the N75, P100, and N145 components of the right eye; the latency and amplitude of the N75, P100, and N145 components of the left eye.

[0143] The data acquisition module also uses the eye movement detector to obtain eye movement event indicators (EEM), including:

[0144] Responsiveness Scenario Score (RSS): RSS1, RSS2, and RSS(1+2);

[0145] Eye movement fixation parameters: number of viewpoints (NEF); discrimination score (D, calculated as D = 10.265 - 0.065 × NEF - 0.871 × RSS); total fixation time (TESL) and mean fixation time (MESL). The above parameters were recorded in three test phases (EEM1, EEM2, and EEM3);

[0146] In this embodiment, all physiological data collection was performed in a standardized experimental environment with the ambient temperature controlled at 22-25°C. The subjects were required to avoid strenuous exercise and drinking alcohol within 1 hour before the test and to remain awake and relaxed during the test.

[0147] This embodiment combines the Transformer and ANN algorithms to build a depression recognition model, such as using the scikit-learn package and other deep learning frameworks in the Python 3.10 environment to build a depression binary classification recognition model;

[0148] The binary depression classification model specifically includes the following: The input layer receives the weighted concatenated fused feature vector to capture the rich information of the input data; the Transformer encoder first reshapes the input features and performs positional encoding to capture the positional information of the input data; then implements a multi-head attention mechanism, using multiple attention heads in parallel to generate multiple different contextual representations. Each attention head processes the input features by weighting the query (Q), key (K), and value (V), and then achieves an enhanced feature representation through weighted fusion. The outputs of the multiple heads are then aggregated, and the aggregated results are sequentially processed through a residual connection + LayerNorm, a feedforward network, and then a residual + LayerNorm again. Finally, average pooling is performed to generate the Transformer output features. The hidden layer passes the Transformer output features and the features after the initial fusion processed by the ANN to the ANN. The ANN consists of multiple fully connected layers configured to process the extracted deep features and uses the ReLU activation function for nonlinear transformations. The decision and output layer maps the hidden layer output features to the depression risk label space, calculates the label weights, and outputs the depression risk probability value using the Sigmoid activation function.

[0149] A grid search method was used to optimize the hyperparameter configuration of the depression recognition model, which combines a Transformer and an artificial neural network, to improve the model's recognition accuracy (see Table 1). To ensure the model's reliability and stability, reduce the risk of overfitting, and improve its generalization ability, a five-fold cross-validation method was used, with the data divided into an 80% training set and a 20% test set. The training set data was used for training, and the test set data was used for predictions for the tuned model.

[0150] The loss function used by the model training module in model training is binary cross entropy (BCE Loss):

[0151] Where N: the number of training samples;

[0152] y i : The true label of the i-th sample, 0 = healthy, 1 = depressed;

[0153] The depression probability of the i-th sample predicted by the model;

[0154] The optimizer used in the model training module is Adam, with an initial learning rate of 0.001.

[0155] Early stopping method: terminate training when the validation set loss does not decrease for 5 consecutive rounds;

[0156] Performance evaluation parameters after model training: including receiver operating characteristic curve ROC, area under the curve AUC, accuracy, precision, recall rate and F1 value; such as Figures 4 to 6 Schematic diagram of the performance evaluation shown.

[0157] In a specific embodiment, the prediction output module in this embodiment further determines the depression risk level based on the depression membership corresponding to the collected data of the new user;

[0158] Among them, when the depression membership is greater than 0 and less than or equal to 0.3, the depression risk level of the collected data of the new user is determined to be mild; when the depression membership is greater than 0.3 and less than or equal to 0.7, the depression risk level of the collected data of the new user is determined to be moderate; when the depression membership is greater than 0.7 and less than or equal to 1, the depression risk level of the collected data of the new user is determined to be severe;

[0159] The calculation formula of depression membership is:

[0160]

[0161] μ D is depression membership; is the depression probability value (between 0 and 1) output by the depression identification model; k is the adjustment parameter that controls the steepness of membership change, and its value range is 2≤k≤10; θ is the risk critical threshold, defined as the depression risk cutoff point, and its default value is 0.5.

[0162] The system advantages of this embodiment include:

[0163] (1) Multimodal data collaborative collection plan: Develop a WeChat applet to integrate the PHQ-9 scale assessment to realize digital collection of psychological data; use the Nihon Kohden MEB-9404C evoked potential device to collect 16-channel ERP data through the oddball paradigm; configure the DEM-2000 eye tracker to collect 15 eye movement indicators in a standardized manner; establish an environmental control system (constant temperature of 22-25℃, exercise / drinking restrictions) to ensure data comparability.

[0164] (2) Feature screening system: Mann-Whitney U test was used to screen the difference features between groups (p<0.05); Pearson correlation analysis was used to extract physiological indicators that were significantly correlated with PHQ-9 (p<0.05).

[0165] (3) Feature fusion: weighted splicing based on correlation coefficient;

[0166] (4) Interpretable combination of Transformer and ANN model architecture: Input layer: 43-dimensional fused feature vector; Transformer encoder: position encoding the input features to capture the temporal correlation between features; Hidden layer: ReLU activation function is used for nonlinear transformation to enhance the model's ability to learn complex feature relationships; Output layer: Sigmoid function outputs depression probability; Innovative integration: grid search optimization of hyperparameters (learning rate, Dropout rate, etc.), early stopping mechanism (termination after 5 rounds of verification loss does not decrease) and SHAP algorithm to quantify feature contribution.

[0167] (5) End-to-end evaluation system: The ERP and eye movement data of the subjects are input and trained with feature fusion; the binary classification result (depression / no depression) is output. Furthermore, the depression probability value output by the model is mapped into three risk levels through the depression membership function.

[0168] Example 2

[0169] The embodiment of the present invention proposes a depression assessment method based on an ERP and eye movement data fusion model, which includes the following steps.

[0170] Step 1: Obtain psychological scales, ERP, and eye movement data collection.

[0171] 1) General information and depression assessment scale PHQ-9 were collected online through WeChat official account to assess students' mental health status.

[0172] Visual evoked potential (VEP) data obtained using the checkerboard reversal pattern include: the latency and amplitude of the N75, P100, and N145 components of the right eye; the latency and amplitude of the N75, P100, and N145 components of the left eye.

[0173] 2) A Nihon Kohden MEB-9404C evoked potential instrument was used, employing the oddball paradigm. Recording electrode A11 was placed at the top of the skull (Cz), recording electrode A21 was placed at the center of the forehead (Fpz), reference electrodes were placed at the left / right earlobes, and the ground electrode was placed at the hand. The instrument used short tone stimulation, with a probability ratio of 8:2 for low-pitched to high-pitched stimulation, and the two appeared randomly. A11 represents the top of the skull (Cz); A21 represents the center of the forehead (Fpz). The 16 indicators collected were:

[0174] N100 latency and amplitude (A11, A21): Latency: refers to the time from the onset of stimulation to the appearance of fluctuations, reflecting the initiation of sensory attention; Amplitude: refers to the amplitude of the fluctuation, which is usually related to the intensity of sensory processing and attention stimulation.

[0175] P200 latency and amplitude (A11, A21): Latency: usually related to the complex feature analysis of sound stimuli; amplitude: may reflect the degree of perception and processing of sound signals.

[0176] N200 latency and amplitude (A11, A21): Latency: related to conflict detection or the comparison of unknown and known stimuli; Amplitude: reflects the brain's discrimination and classification of stimuli.

[0177] P300 latency and amplitude (A11, A21): Latency: usually related to the evaluation of stimuli in attention and memory; Amplitude: represents the most commonly studied ERPs component, often used to assess attention and cognitive processes, such as novelty and stimulus importance.

[0178] Visual evoked potentials were measured in a checkerboard pattern. The subjects sat 1 meter away from the screen, with the recording electrode placed 2 cm above the occipital protuberance, the reference electrode placed on the forehead approximately 1 cm below the hairline, and the ground electrode placed on the hand. Both eyes were monitored alternately, and the latency and amplitude of the left and right eyes were collected around 75 ms, 100 ms, and 145 ms after stimulation, for a total of 12 visual evoked potential indicators.

[0179] The 12 visual evoked potential indicators include: right eye N75 latency and amplitude, right eye P100 latency and amplitude, right eye N145 latency and amplitude; left eye N75 latency and amplitude, left eye P100 latency and amplitude, left eye N145 latency and amplitude.

[0180] Latency: Reflects nerve conduction velocity: Latency (e.g., N75, P100, N145) is the time from stimulus onset to neural response, reflecting the speed of signal transmission in the visual system. It is commonly used to assess the integrity and function of the visual pathway. Delayed latency may indicate an abnormality in the optic nerve or pathway, such as optic neuritis or multiple sclerosis.

[0181] Amplitude: Reflects the level of neuronal activity. Amplitude indicates the strength of the response, that is, the relative degree of activation of neurons in the visual cortex to visual stimuli. Larger amplitudes generally indicate stronger neural activity, while reduced amplitudes may indicate a decrease in the response function or pathology of the visual cortex.

[0182] 3) Eye Movement Data: Eye movement testing was performed using a Shanghai-made Dikang DEM-2000 eye movement monitor. The subject looked at a small screen in front of them, with both eyes at a distance of 25 cm from the screen. The subject's gaze moved from the left to the right of the screen at an angle of 33°. The first S-shaped pattern (S1) was displayed on the screen for 15 seconds. The instrument automatically recorded the number of fixations within 15 seconds and counted them as the number of eye fixations (NEF). The second and third S-shaped patterns (S2, S3) were then displayed on the screen for 15 seconds each. The subject was asked to observe carefully and asked, "Is there any difference from the first pattern?" When the subject answered, "No difference," the subject's gaze was recorded in seven areas (one point per area) for a total of 5 seconds. This was used as the responsive search score (RSS). RSS: The second and third S-shaped patterns were displayed on the screen in sequence, and the subject's eye movements were measured and recorded for 15 seconds. The metrics RSS1 and RSS2 were recorded. Each image is divided into 7 areas, for a total of 14 areas. Each time the subject fixates on an area, 1 point is scored. There are 15 eye movement metrics in total: RSS1, RSS2, RSS(1+2), EEM1 NEF, EEM1 D, EEM1 TESL, EEM1 MESL, EEM2 NEF, EEM2 D, EEM2 TESL, EEM2 MESL, EEM3 NEF, EEM3 D, EEM3 TESL, and EEM3 MESL. RSS1, RSS2, and RSS(1+2) are used to assess the subject's reactivity and exploratory behavior to different stimuli during visual tasks.

[0183] Eye Movement Event Metrics (EEM):

[0184] NEF (Number of Eye Fixations): Indicates the number of times an observer looks at a specific area within a specific period of time, reflecting the degree of attention concentration.

[0185] Discrimination score D: The calculation formula is D = 10.265-0.065×NEF-0.871×RSS

[0186] TESL (Total Eye Stare Length): The accumulation of total fixation time in an eye movement event, indicating the time of visual information processing.

[0187] MESL (Mean Eye Stare Length): The average fixation time of each eye movement event, indicating the concentration of each gaze.

[0188] Among them, all physiological data collection is carried out in a standardized experimental environment with the ambient temperature controlled at 22-25°C. Subjects are required to avoid strenuous exercise and drinking within 1 hour before the test and remain awake and relaxed during the test.

[0189] Step 2: Screening of ERP and eye movement data.

[0190] Students with PHQ-9 scale results less than 5 points were set as the healthy control group, and those with PHQ-9 scale results greater than or equal to 5 points were set as the depression group. ERP and eye movement data were collected and organized at the same time.

[0191] (1) Intergroup difference analysis (depression group vs. healthy control group):

[0192] Let X be ERP or eye movement index, and define group variable G:

[0193] G = {1, depression group (PHQ-9 ≥ 5); 0, healthy control group (PHQ-9 < 5)}

[0194] Using the Mann-Whitney U test:

[0195] H0: Median(X|G=1) = Median(X|G=0)

[0196] H1: Median(X|G=1)≠Median(X|G=0)

[0197] Screening criteria: statistical significance p value < 0.05

[0198] Output: Significant difference indicator set F_diff={X1,X2,...,X_k}

[0199] (2) Correlation analysis:

[0200] In the depression group, a Pearson correlation analysis was performed between the indicator X_i in F_diff and the PHQ-9 total score Y:

[0201] Calculate the correlation coefficient: r_{X_iY}=Pearson(X_i,Y)

[0202] Test hypothesis: H0: ρ_{X_iY}=0; H1: ρ_{X_iY}≠0;

[0203] Screening criteria: statistically significant p1 value < 0.05;

[0204] Output: significant correlation index set F_corr = {X_{i1}, X_{i2}, ..., X_{im}};

[0205] (3) Feature screening results:

[0206] The final feature for the next fusion is the index in F_corr;

[0207] Step 3: Fusion of ERP and eye movement data.

[0208] Based on the obtained screening features, further feature fusion is performed, including:

[0209] (1) Feature standardization: Z-score standardization is performed on ERP features (28 dimensions) and eye movement features (15 dimensions) to eliminate dimensional differences:

[0210]

[0211] X erp : original ERP feature vector (28 dimensions);

[0212] X eye : original eye movement feature vector (15 dimensions);

[0213] μ erp ,μ eye : the mean of ERP / eye movement features in the training set;

[0214] σ erp ,σ eye : standard deviation of ERP / eye movement features in the training set;

[0215] X′ erp ,X′ eye : Normalized ERP / eye movement characteristics.

[0216] (2) Multimodal feature fusion: weighted splicing fusion is used: feature weights are calculated based on the Pearson correlation coefficient, and the standardized features are spliced according to the weights:

[0217] r i =Pearson(X i ,PHQ-9)

[0218] Fusion vector:

[0219] r i : Pearson correlation coefficient between the i-th feature (ERP or eye movement) and the PHQ-9 total score

[0220] k: total number of features (43 dimensions, including 28 ERPs + 15 eye movements);

[0221] w i : Normalized weight of the i-th feature (∑w i =1).

[0222] F fused : Feature vector after weighted fusion (43 dimensions).

[0223] Step 4: Construction of depression recognition model based on ERP and eye movement fusion data.

[0224] (1) This embodiment uses a combination of Transformer and ANN algorithms to construct a depression recognition model, specifically including: input layer: receiving the feature vector after weighted splicing and fusion to capture the rich information of the input data; Transformer encoder: first reshape the input features and process them by position encoding to capture the position information of the input data; then implement a multi-head attention mechanism, use multiple attention heads for parallel processing, and generate multiple different context representations respectively. Each attention head processes the input features by query (Q), key (K), and value (V) weight calculation, and obtains enhanced feature representation through weighted fusion; then summarize the outputs of multiple heads, and the summary results are sequentially passed through residual connection + LayerNorm, feedforward network, and residual network again.

[0225] +LayerNorm, and finally, through average pooling, the output features of the Transformer are generated. Hidden layer: The output features of the Transformer and the features after the initial fusion features are processed by the ANN are passed to the ANN part. The ANN consists of multiple fully connected layers and is configured to process the extracted deep features. The ReLU activation function is used for nonlinear transformation. The decision and output layer: The features output by the hidden layer are mapped to the depression risk label space, the label weights are calculated, and the probability value of depression risk is output through the Sigmoid activation function. Figure 2 .

[0226] (2) Grid search parameter adjustment:

[0227] For model parameters, a grid search method was used to obtain the optimal parameters, as shown in Table 1. To ensure the reliability and stability of the model, reduce the risk of overfitting, and improve the model's generalization ability, a five-fold cross-validation method was used, dividing the data into an 80% training set and a 20% test set. The training set data was used for training, and the test set data was used for prediction of the tuned model.

[0228] Table 1. ANN model parameters

[0229]

[0230] (3) Model training

[0231] Loss function: binary cross entropy (BCE Loss):

[0232]

[0233] Where N is the number of training samples.

[0234] y i : The true label of the i-th sample, 0 = healthy, 1 = depressed;

[0235] The depression probability of the i-th sample predicted by the model;

[0236] The optimizer used in the model training module is Adam (initial learning rate 0.001);

[0237] Early stopping method: terminate training when the validation set loss does not decrease for 5 consecutive rounds;

[0238] Based on the confusion matrix data, the present invention uses five indicators to evaluate the generalization performance of the classifier, including the receiver operating characteristic (ROC) curve, the area under the curve (AUC), accuracy, precision, recall and F1 value.

[0239] 1) ROC and AUC. The ROC curve is a graph plotted with the true positive rate on the y-axis and the false positive rate on the x-axis. The diagonal line corresponds to the random guessing model, and the closer the curve is to the upper left corner, the better the classification performance. The ROC curve is a qualitative metric that is simple, intuitive, and highly readable. The more convex the ROC curve is and the closer it is to the upper left corner, the greater its diagnostic value. The AUC is a quantitative metric that is more accurate and objective. A larger AUC value indicates better classification performance.

[0240] 2) Accuracy: refers to the ratio of the number of correct predictions to the total number of actual predictions. It is the most widely used classification evaluation indicator.

[0241] Calculation formula:

[0242] 3) Precision P (Precision): The proportion of actual positive cases to the total predicted positive cases.

[0243] The calculation formula is:

[0244] 4) Recall (R) refers to the proportion of cases predicted to be positive that are actually positive.

[0245] Calculation formula:

[0246] 5) F1 score: It is the weighted average of precision and recall.

[0247] Calculation formula:

[0248] The weighted fusion feature vector F fused , as the input feature of the model, and then mark the PHQ-9 scale total score <5 points as 0 (i.e., no depression), and the PHQ-9 scale ≥5 points as 1 (depression), perform binary classification of depression, and establish a depression recognition model based on ERP and eye movement data.

[0249] The model was evaluated using the five-fold cross validation method, with AUC = 0.921, accuracy = 0.838, precision = 0.808, recall = 0.910, and F1 score = 0.855. Figure 4 shown.

[0250] Furthermore, the depression probability value output by the model was mapped into three risk levels through the depression membership function, with 5≤PHQ-9≤9 points marked as 0 (mild depression), 10≤PHQ-9≤14 points marked as 1 (moderate depression), and PHQ-9≥15 points marked as 2 (severe depression). Based on the identified depression, three-category identification was performed.

[0251] AUC for Class 0 = 0.827; AUC for Class 1 = 0.862; AUC for Class 2 = 0.923. Accuracy = 0.769, Precision = 0.769, Recall = 0.770, F1 score

[0252] =0.769. Figure 5 shown.

[0253] Step 5: Interpretability analysis of the prediction model interpretability analysis module.

[0254] Building an interpretable deep learning model is crucial for practical applications. Interpretability refers to the degree to which humans can understand the decisions made by deep learning. The higher the interpretability of the model, the easier it is to understand why the model makes a certain prediction, and the easier it is to promote and apply. This patent applies the game theory method SHAP to analyze the interpretability of the model, and uses SHAP values to visualize the degree of dependence of different voiceprint features and their impact on the prediction results, helping to identify speech features that have an important impact on the model's prediction of positive results, quantifying the contribution of each feature to the model's prediction decisions, further improving the model's credibility and acceptability, and helping to realize the actual transformation and application of the model. The larger the absolute value of SHAP of the ERP and eye movement features, the greater the contribution to the depression recognition model, such as Figure 6 shown.

[0255] like Figure 6 As shown in the figure, SHAP value analysis showed that R2_P100 was the most important factor affecting the effect of music intervention on patients with depression.

[0256] Step 6: The model intelligent evaluation module performs individualized predictive evaluation.

[0257] For each student population to be assessed, a pre-trained deep learning model and features are used to calculate whether the individual's ERP and eye movement features are present in the deep learning model. The results of the personalized recognition module are presented to school staff and individual students, facilitating early screening and intervention for depression. The generalization performance of the student depression recognition model is evaluated through cross-validation using an external dataset encompassing diverse population characteristics.

[0258] In this embodiment, an advanced ANN+Transformer dual-feature processing algorithm is used to first collect ERP and eye movement data, then fuse the ERP and eye movement data, and then input the fused ERP+eye movement features into the ANN+Transformer dual-feature processing algorithm system to construct a depression recognition model. The intelligent depression recognition system for student groups based on ERP and eye movement data established in this patent has certain robustness and accuracy, can automatically and intelligently identify students' depressive states, facilitates large-scale mental health assessment of student groups, and is easy to promote and apply in practical applications.

[0259] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0260] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions.

[0261] It should be noted that, in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims enumerating several means, several of these means may be embodied by one and the same hardware. The use of the words first, second, third etc. is for convenience only and does not indicate any order. These words may be understood as part of the component name.

[0262] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0263] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments after learning the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0264] Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if such modifications and variations fall within the scope of the claims and their equivalents, the present invention shall also include such modifications and variations.

Claims

1. An intelligent depression assessment system based on an ERP and eye movement data fusion model, comprising: The data acquisition module is used to collect the user's psychological scale data, obtain multiple ERP indicators through the evoked potential meter, and obtain multiple eye movement indicators through the eye tracker; Data analysis module, used to screen ERP and eye movement indicators that are significantly related to depressive symptoms; The feature fusion module is used to fuse the filtered ERP data and eye movement data into a fused feature vector as the input feature of the model; The model training module is used to input the fused feature vector of each trainee into the depression binary classification recognition model constructed by combining the Transformer and ANN algorithms during the training phase, train the parameters of the depression binary classification recognition model, and obtain the trained depression binary classification recognition model; The prediction output module is used to predict the fused feature vector of new users using the trained model and output the depression risk assessment results through a visual interface.

2. The system according to claim 1, wherein: The data acquisition module is specifically used for: The psychological scale data of the depression assessment scale PHQ-9 were collected through the WeChat mini program; And / or, obtain multiple ERP indicators through evoked potential monitoring, including: Auditory evoked potential (AEP) data obtained through sound stimulation using the oddball paradigm, including: the latency and amplitude of the N100, P200, N200, and P300 components recorded at the parietal electrode Cz; the latency and amplitude of the N100, P200, N200, and P300 components recorded at the mid-frontal electrode Fpz; Visual evoked potential (VEP) data obtained using the checkerboard flipping mode include: the latency and amplitude of the N75, P100, and N145 components of the right eye; the latency and amplitude of the N75, P100, and N145 components of the left eye; And / or, use an eye tracker to obtain multiple eye movement metrics, including: Responsiveness Exploration Score RSS: RSS1, RSS2 and RSS1+RSS2; Eye fixation parameters: number of viewpoints NEF; Discriminant score D, D = 10.265-0.065×NEF-0.871×RSS; Total fixation time TESL and mean fixation time MESL.

3. The system according to claim 1, wherein: Data analysis module, specifically used for: Analysis of differences between the depression group and the healthy control group: X represents ERP or eye movement index, corresponding to the group variable G: G = {1, depression group (PHQ-9 ≥ 5); 0, healthy control group (PHQ-9 < 5)} Using the Mann-Whitney U test: H0: median(X|G=1) = median(X|G=0); H1: Median(X|G=1)≠Median(X|G=0); The Wilcoxon rank sum test was used to calculate the statistical significance p value, and the screening criterion was p < 0.05; Output: Significant difference indicator set F_diff={X1,X2,...,X_k} And / or, correlation analysis: In the depression group, a Pearson correlation analysis was performed between the indicator X_i in F_diff and the PHQ-9 total score Y: Calculate the correlation coefficient: r_{X_iY}=Pearson(X_i,Y); Test hypothesis: H0: ρ_{X_iY}=0; H1: ρ_{X_iY}≠0; Use Spearman's correlation test to calculate the statistical significance p1 value, with the screening criterion of p1 < 0.05; Output: significant correlation index set F_corr = {X_{i1}, X_{i2}, ..., X_{im}}; Feature screening results: The features that are finally fused in the next step are the indicators in F_corr.

4. The system according to claim 1, wherein: The feature fusion module is specifically used to: first perform feature normalization processing on the filtered ERP data and eye movement data, and then perform multimodal feature fusion to obtain a fused feature vector; Feature standardization: Z-score standardization is performed on ERP features and eye movement features to eliminate dimensional differences: X erp : original ERP feature vector; X eye : original eye movement feature vector; μ erp ,μ eye They are: the mean of ERP / eye movement features in the training set; σ erp ,σ eye They are: standard deviation of ERP / eye movement features in the training set; X′ erp ,X′ eye They are: standardized ERP / eye movement features.

5. The system according to claim 4, characterized in that The multimodal feature fusion of the feature fusion module includes: Weighted splicing fusion: Calculate feature weights based on the Pearson correlation coefficient and splice the standardized features according to the weights: r i =Pearson(X i ,PHQ-9) Fusion vector: r i : Pearson correlation coefficient between the ith feature and the total score of PHQ-9; k: total number of features; w i : Normalized weight of the i-th feature, ∑w i =1;F fused : Feature vector after weighted fusion.

6. The system according to claim 1, wherein: The system further comprises: A model building module for building a depression binary classification recognition model in a Python 3.10 environment using the scikit-learn package and other deep learning frameworks. The model combines the Transformer and ANN algorithms. The depression binary classification recognition model specifically includes: Input layer: receives the feature vector after weighted splicing and fusion, Transformer encoder: The input features are positionally encoded to capture the sequential information of the input data. A multi-head attention mechanism is implemented, with multiple attention heads processing in parallel to generate multiple different contextual representations. Each attention head processes the input features by weighting the query Q, key K, and value V, and an enhanced feature representation is obtained through weighted fusion. Finally, the outputs of multiple heads are averaged to generate the output features of the Transformer. Hidden layer: The output features of the Transformer and the features of the initial fusion features processed by the ANN are combined and passed to the ANN part. The ANN consists of multiple fully connected layers and is configured to process the extracted deep features. The ReLU activation function is used for nonlinear transformation. Decision and output layer: Maps the features output by the hidden layer to the depression risk label space, calculates the label weight, and outputs the probability value of depression risk through the Sigmoid activation function.

7. The system according to claim 1, wherein: The loss function used by the model training module in model training is binary cross entropy in N: number of training samples; y i : The true label of the i-th sample, 0 = healthy, 1 = depressed; The depression probability of the i-th sample predicted by the model; The optimizer used in the model training module is Adam, with an initial learning rate of 0.

001. The performance evaluation parameters after model training include receiver operating characteristic curve (ROC), area under the curve (AUC), accuracy, precision, recall, and F1 value.

8. An intelligent depression assessment method based on an EPR and eye movement data fusion model, characterized in that: include: Collect users' psychological scale data, multiple ERP indicators, and multiple eye movement indicators; Screening ERP and eye movement indicators that are significantly associated with depressive symptoms; The filtered ERP data and eye movement data are fused and spliced into a fused feature vector as the input feature of the model; During the training phase, the fused feature vector of each trainee is input into a depression binary classification recognition model constructed by combining the Transformer and ANN algorithms, the parameters of the depression binary classification recognition model are trained, and the trained depression binary classification recognition model is obtained; The trained model is used to predict the fused feature vector of the new user, and the depression risk assessment results are output through a visual interface. The fused feature vector of the new user is obtained based on the fusion of multiple collected ERP indicators and multiple eye movement indicators.

9. The method according to claim 8, characterized in that Collect user's psychological scale data, multiple ERP indicators, and multiple eye movement indicators, including: The psychological scale data of the depression assessment scale PHQ-9 were collected through the WeChat mini program; Multiple ERP indicators were obtained through evoked potential monitoring, including: Auditory evoked potential (AEP) data obtained through sound stimulation using the oddball paradigm, including: the latency and amplitude of the N100, P200, N200, and P300 components recorded at the parietal electrode Cz; the latency and amplitude of the N100, P200, N200, and P300 components recorded at the mid-frontal electrode Fpz; Visual evoked potential (VEP) data obtained using the checkerboard flipping mode include: the latency and amplitude of the N75, P100, and N145 components of the right eye; the latency and amplitude of the N75, P100, and N145 components of the left eye; The eye tracker can be used to obtain multiple eye movement indicators, including: Responsiveness Exploration Score RSS: RSS1, RSS2 and RSS1+RSS2; Eye fixation parameters: number of viewpoints NEF; The formula for calculating the discriminant score D is D = 10.265-0.065×NEF-0.871×RSS; Total fixation time TESL and mean fixation time MESL.

Images