Cross-session electroencephalogram signal emotion recognition method based on multi-view comparative learning

Abstract

The invention provides a cross-session electroencephalogram signal emotion recognition method based on multi-view comparative learning, and the method comprises the steps: obtaining a differential entropy feature and a brain asymmetric feature of an electroencephalogram signal, carrying out the coding of the differential entropy feature and the brain asymmetric feature through a GRU-AT model, obtaining a differential entropy feature view angle and a brain asymmetric feature view angle, and carrying out the recognition of the emotion of the electroencephalogram signal. Fusing the two features to obtain a fused feature view angle; based on multi-view comparative learning, a differential entropy feature view angle, a brain asymmetric feature view angle and a fusion feature view angle serve as three view angles of comparative learning, a comparative loss function is calculated, a GRU-AT model encoder is trained and optimized by minimizing the comparative loss function, and finally the optimized GRU-AT model encoder is connected with a classifier for fine adjustment. In the fine tuning process, the encoder does not carry out gradient propagation, and emotion recognition is completed. According to the method, the problems of insufficient generalization ability and insufficient feature expression when a traditional model is used for processing electroencephalogram signal cross-session emotion recognition are effectively solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and pattern recognition, and specifically provides a cross-session EEG signal emotion recognition method based on multi-view contrast learning. Background Art

[0002] Emotion recognition using electroencephalogram (EEG) signals objectively identifies emotional states by analyzing brain electrical activity, and has important applications in fields such as psychology and neuroscience. Cross-session emotion recognition aims to process emotional data from the same subject across different time periods (sessions). However, individuals' emotional expressions vary across sessions, making it difficult for traditional models to capture stable emotional features and resulting in insufficient generalization. Current research on cross-session EEG emotion recognition focuses on domain adaptation techniques, feature extraction and fusion, and contrastive learning applications. Domain adaptation techniques use samples from the training set as the source domain and samples from the test set as the target domain. The model transfers knowledge gained in the source domain to the target domain, minimizing the data distribution differences between the two domains. For example, Yu et al. proposed a new cross-session framework based on a fine-grained mutual learning adaptation network. However, this method simply treats different sessions as a single source domain, ignoring the significant differences between source domains. She et al. proposed an emotion recognition method based on a multi-source correlation domain adaptation network, which addresses this issue by branching across multiple source domains. However, it still relies on labeled data for domain alignment, resulting in limited adaptability to unlabeled target domains. Feature extraction and fusion utilize deep learning to extract features relevant to emotion recognition tasks. For example, Pang et al. proposed a multi-scale masked autoencoder, which improves recognition accuracy through multi-scale feature representation. However, this method fails to fully incorporate spatial asymmetry in EEG signals (e.g., differences in left and right EEG activity), resulting in a relatively single feature dimension. In contrastive learning applications, anchor samples are positioned close to positive samples and far from negative samples. For example, Aaron van den Oord et al. proposed the CPC (Contrastive Predictive Coding) model, which compresses high-dimensional data into a more compact latent embedding space. Within this latent space, an autoregressive model is used to predict many steps into the future. Yonglong Tian et al. proposed the CMC (Contrastive Multiview Coding) model, which, within the framework of contrastive learning, explores powerful feature representations that model invariant factors in the data. However, this model fails to design perspectives tailored to the multidimensional nature of EEG signals, preventing it from effectively learning universal feature representations across sessions. Consequently, existing models fail to fully exploit the correlation between features from different perspectives within the same subject, making it difficult to account for the dynamic changes in emotional expression across sessions, resulting in a decrease in recognition accuracy with increasing number of sessions. Summary of the Invention

[0003] In order to address the deficiencies in the prior art, the present invention provides a cross-session EEG signal emotion recognition method based on multi-perspective contrastive learning. Different perspectives are designed according to the characteristics of EEG signals, including differential entropy features and brain asymmetry features. The GRU-AT model is used to obtain the most effective emotion expression features of each perspective and fuse them, maximizing the similarity of different perspectives of the same sample. This effectively overcomes the problems of insufficient generalization ability and insufficient feature expression of traditional models in processing cross-session EEG signal emotion recognition, and improves the adaptability and recognition accuracy of the model.

[0004] The technical solution adopted by the present invention to solve its technical problems is:

[0005] A method for cross-session EEG emotion recognition based on multi-view contrastive learning, comprising the following steps:

[0006] Obtain the differential entropy features and brain asymmetry features of the EEG signal, wherein the brain asymmetry features include left-right asymmetry features and front-back asymmetry features; encode the differential entropy features and brain asymmetry features respectively through the GRU-AT model to obtain the differential entropy feature perspective V DE , Perspective on Brain Asymmetry V AS , the differential entropy feature perspective V DE and the brain asymmetry feature perspective V AS The two are fused to obtain the fusion feature perspective V fu Based on multi-view contrast learning, the differential entropy feature perspective V DE , Perspective on Brain Asymmetry V AS and fusion feature perspective V fu As three perspectives of contrastive learning, the contrastive loss function is calculated, and the GRU-AT model encoder is trained and optimized by minimizing the contrastive loss function. Finally, the optimized GRU-AT model encoder is connected to the classifier for fine-tuning. During this fine-tuning process, the encoder does not perform gradient propagation to complete emotion recognition.

[0007] Furthermore, the GRU-AT model includes:

[0008] The two residual network structure encoders Encoder-1 and Encoder-2 are used to encode the differential entropy features and brain asymmetry features respectively;

[0009] Two GRU modules to capture long-term dependencies in EEG signal time series;

[0010] Three attention mechanism modules are used to weight the features output by the GRU module.

[0011] Furthermore, the specific process implemented in the GRU-AT model is: the differential entropy feature xt Enter the encoder Encoder-1 to get the representation vector Z1(t)=g enc1 (x t ), and then sent to the first GRU module to obtain the hidden layer state C1(t)=g GRU (Z1(t)), and then the differential entropy feature perspective V is obtained through the first attention mechanism module DE =g at1 (c1(t)); the brain asymmetry feature x t 'Send to encoder Encoder-2 to get the representation vector Z2(t)=g enc2 (x t '), and then the brain asymmetric feature perspective V is obtained through the second attention mechanism module As =g at2 (g enc2 (x t ')); the differential entropy feature perspective V DE Asymmetric features of the brain V AS The fused feature V fu =V DE +V As After being sent to the second GRU module for processing, the output is processed by the third attention mechanism module to obtain the final fusion feature perspective V fu .

[0012] Furthermore, the multi-view contrast learning is specifically as follows: given the view angles V1 and V2, one view angle V1 is fixed, and positive samples and negative samples are enumerated from the other view angle V2, and the loss function Expressed as formula (5); fix a perspective V2, and enumerate positive samples and negative samples from another perspective V1, the loss function It is expressed as formula (6);

[0013]

[0014]

[0015] in, is a sample with fixed view angle V1, is a sample with a fixed view angle V2, j = 1 is a positive sample, j > 1 is a negative sample; k represents the number of negative samples; h θ ({v1, v2}) represents the distance between the two views, measured by cosine similarity, and θ represents the parameters that need to be trained in the encoder. The loss of the two views is expressed as L(V1, V2), as shown in Equation (8):

[0016]

[0017] Among them, V1 and V2 represent different perspectives;

[0018] This method has a differential entropy feature perspective V DE , Perspective on Brain Asymmetry V AS and fusion feature perspective V fu There are three perspectives, that is, the number of perspectives M = 3, so the two-perspective loss is calculated by taking two perspectives as a pair, and the final loss function L all It is expressed as formula (9):

[0019]

[0020] Furthermore, the distance h between the two viewing angles θ The calculation method of ({v1,v2}) is as follows:

[0021]

[0022] in Representative sample v1 passes The obtained feature representation is Representative sample v2 passes The obtained feature representation, τ is the temperature hyperparameter.

[0023] The beneficial effects of the present invention include:

[0024] This method effectively integrates differential entropy and spatial asymmetry features of EEG signals through a multi-view contrastive learning framework, significantly improving the accuracy and generalization of cross-session emotion recognition. By designing three complementary perspectives—differential entropy, brain asymmetry, and fusion features—it fully leverages the time-frequency and spatial asymmetry characteristics of EEG signals. The residual encoder and attention mechanism in the GRU-AT model are used to extract deep temporal features from each view. A multi-view contrastive learning strategy maximizes the similarity of similar emotion samples across different views, effectively overcoming the feature drift problem of traditional methods caused by dynamic changes in emotional expression across sessions. By optimizing a loss function based on three-view pairwise comparisons, this method strengthens the learning of universal emotion representations across sessions in an unsupervised manner, reducing reliance on labeled data in the target domain. The introduction of the fusion perspective enhances the model's ability to capture multi-dimensional feature interactions. Compared to existing domain adaptation and single-view contrastive learning methods, this method demonstrates significant advantages in terms of emotion feature stability, adaptability to inter-session distribution differences, and characterization of complex dynamic patterns, providing a more robust solution for cross-session EEG emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is the structural diagram of the GUR-AT model of this application;

[0026] Figure 2It is a diagram of the multi-perspective comparative learning strategy model;

[0027] Figure 3 This is a schematic diagram of the accuracy of different sessions used as test sets in Example 2;

[0028] Figure 4 This is the t-SNE feature map of subject 1 in the SEED dataset of Example 2. DETAILED DESCRIPTION

[0029] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0030] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0031] This application maximizes the correlation information between different perspectives of the same subject, and proposes a cross-session EEG emotion recognition method based on multi-perspective contrastive learning. First, different perspectives are designed according to the characteristics of the EEG signal, including differential entropy features DE and brain asymmetry features. The GRU-AT model is used to obtain the most effective emotion expression features of each perspective, including time domain, frequency domain, and spatial domain features, and fused. Specifically, the GRU module can capture the temporal characteristics of the EEG signal, and the attention mechanism further enhances the expression of effective features. The representation of a single perspective achieves a more fine-grained emotional expression of the EEG signal, while the representation of the fused perspective achieves a more global emotional expression of the EEG signal. The performance of the model is enhanced by combining contrastive learning and a more comprehensive feature expression.

[0032] Example 1: The EEG emotion recognition based on multi-view contrast learning proposed in this application consists of three parts: feature extraction, multi-view contrast learning, and fine-tuning classification. Among them, the feature extraction part makes full use of the time-frequency-space characteristics of the EEG signal, and performs feature encoding and fusion based on the GRU-AT model to comprehensively obtain the overall and fine-grained feature representation of the EEG signal; in the multi-view contrast learning part, by comparing data from different perspectives, the contrast loss is optimized so that the model can capture the features of each perspective, thereby learning high-quality embedding representations. In addition, this application applies multi-view contrast learning to the multi-dimensional feature emotion recognition of the GRU-AT model for the first time; the fine-tuning classification part migrates the encoder to the downstream task of recognition, and only needs to train the classifier to complete the recognition. Specifically, the extracted EEG signal differential entropy features and left-right front-back asymmetric features are first sent to their respective encoders and passed through the gated recurrent network based on the attention mechanism to obtain the perspective V DE 、V AS , then V DE 、V AS The row feature fusion obtains the perspective V including time-frequency domain and spatial domain fu Then, based on the multi-view contrast learning framework, the distance between different viewpoints of the same sample is shortened, while the distance with other samples is increased. Finally, the trained encoder is connected to the classifier for fine-tuning. During this process, the encoder does not perform gradient propagation to complete emotion recognition. Specifically:

[0033] 1. Multi-dimensional feature encoding and fusion based on GRU-AT

[0034] refer to Figure 1 The purpose of the feature encoding design based on the GRU-AT model is to build a feature extraction network. In order to obtain more comprehensive EEG signal features, this application fuses the differential entropy features and asymmetric features of the EEG signal. The differential entropy features of the EEG signal contain rich time-frequency information of the EEG signal, and the asymmetric features contain the physiological spatial relationship of the brain. The fused features contain global feature expressions. These three features together serve as multiple perspectives for comparative learning, which helps to obtain more comprehensive EEG signals.

[0035] First, the DE feature (differential entropy feature) x(t) is sent to the encoder Encoder-1 to obtain the representation vector Z1(t) = g enc1 (x t ), and then sent to the first GRU module to obtain the hidden layer state C1(t)=g GRU (Z1(t)), and then the differential entropy feature perspective V is obtained through the first attention mechanism module DE =g at1 (c1(t)); the brain asymmetry feature x t'Send to encoder Encoder-2 to get the representation vector Z2(t)=g enc2 (x t '), and then the brain asymmetric feature perspective V is obtained through the second attention mechanism module As =g at2 (g enc2 (x t ')); the differential entropy feature perspective V DE Asymmetric features of the brain V AS The fused feature V fu =V DE +V As After being sent to the second GRU module for processing, the output is processed by the third attention mechanism module to obtain the final fusion feature perspective V fu . V DE With V As With a more fine-grained expression of features, V fu It has a more comprehensive expression in the macro sense. DE 、V As 、V fu , which are the three perspectives for subsequent comparative learning.

[0036] 1.1 Differential entropy characteristics and asymmetric characteristics

[0037] Differential entropy represents the logarithm of the energy spectrum of the signal in a specific frequency band, as shown in formula (1). It can reflect the nonlinearity of EEG signals and has a higher recognition effect in emotion recognition than other EEG signal features.

[0038]

[0039] Where x represents the sample value of the continuous feature of the EEG signal; for The EEG signal, σ 2 represents the variance; μ represents the signal mean.

[0040] The left and right electrodes of the brain show significant differences in emotion recognition, including neuronal connection patterns, nerve conduction velocity, and neurochemistry. These factors lead to differences in the spatial electrical signals in the brain, resulting in the left brain being more inclined to positive emotions (such as happiness and excitement), while the right brain is more inclined to negative emotions (such as sadness and fear). Therefore, the left and right brain features extracted from the spatial asymmetry perspective can be used for emotion recognition, as shown in Equation (2). In addition, the front and back electrodes of the brain, including the frontal lobe, temporal lobe, and occipital lobe, also play different roles in emotion processing, as shown in Equation (3). Therefore, extracting the spatial asymmetry of the left and right and front and back directions of the brain helps to obtain the feature expression of the brain in the global space.

[0041] DASM=DE(X left)-DE(X right ) (2)

[0042] DCAU=DE(X frontal )-DE(X posterior ) (3)

[0043] Among them, DE(X frontal ) represents the differential entropy feature obtained from the electrodes in the frontal lobe of the brain, DE(X posterior ) represents the differential entropy features obtained from the temporal and occipital electrodes. DE(X left )、DE(X right ) represent the differential entropy features extracted from the left and right sides of the brain, respectively.

[0044] 1.2GRU-AT model structure setting

[0045] Considering the size and dimension of the input dataset, the encoder Res-net is introduced to first extract local features of the EEG signal to capture the key features in the time series data. Res-net can also solve the problem of gradient explosion caused by overly deep network models.

[0046] The GRU module, a variant of a recurrent neural network, is specifically designed to process sequential data. It uses a core recurrent unit to process input data and update the hidden state at each time step. The GRU incorporates two gating mechanisms: an update gate and a reset gate to control the flow of information. The update gate determines how much of the hidden state from the previous moment is retained, while the reset gate determines how much of the previous moment's input is ignored. In this way, the GRU can capture long-term dependencies in time series, facilitating feature extraction from these sequences.

[0047] The attention mechanism is a technique that enables a model to automatically focus on important parts of input data. In EEG signals, the attention mechanism helps the model focus on historical information that is most relevant to the current situation. In the GRU-AT model, the attention mechanism is used to weight the features output by the GRU module. By calculating the attention weights at each time step, the model can automatically identify the most important features and perform a weighted fusion of them. This allows the model to focus more on features that have a significant impact on the recognition results, thereby improving the accuracy and stability of predictions.

[0048] 2. Multi-view contrastive learning

[0049] This application is based on multi-view contrastive learning and belongs to a self-supervised learning model. Therefore, the label information of the sample is not used. The model is set as a proxy task to distinguish whether the sample comes from the same source. Given perspectives V1 and V2, which contain sample sets The model is trained by comparing positive sample pairs and negative sample pairs. Positive sample pairs are defined as those from the joint distribution x~p(v1,v2) or The negative sample pair is defined as the product y~p(v1)p(v2) from the marginal distribution or By training a discriminant function h θ (·), so that the positive sample outputs high values ​​and the negative sample outputs low values. Its goal is to select a set S = {x, y1, y2, ..., y k} correctly select the positive sample x, as shown in formula (4).

[0050]

[0051] Among them, x represents the positive sample pair, y represents the negative sample pair, and k represents the number of negative samples.

[0052] In multi-view contrastive learning, if one viewpoint (such as V1) is fixed and positive and negative samples are enumerated from another viewpoint (such as V2), the loss function can be expressed as (5). If one viewpoint (such as V2) is fixed and positive and negative samples are enumerated from another viewpoint (such as V1), the loss function can be expressed as (6).

[0053]

[0054]

[0055] in, is a sample with fixed view angle V1, is a sample with a fixed view angle V2, j = 1 is a positive sample, j > 1 is a negative sample; k represents the number of negative samples; h θ ({v1,v2}) represents the distance between the two views, measured by cosine similarity, and θ represents the parameters that need to be trained in the encoder. The distance h between the two views θ The calculation method of ({v1,v2}) is as follows:

[0056]

[0057] in Representative sample v1 passes The obtained feature representation is Representative sample v2 passes The obtained feature representation, τ is the temperature hyperparameter.

[0058] The loss of two views is expressed as L(V1, V2), as shown in formula (8):

[0059]

[0060] Among them, V1 and V2 represent different perspectives;

[0061] This method has a differential entropy feature perspective V DE , Perspective on Brain Asymmetry V AS and fusion feature perspective V fu There are three perspectives, that is, the number of perspectives M = 3, so the two-perspective loss is calculated by taking two perspectives as a pair, and the final loss function L all It is expressed as formula (9):

[0062]

[0063] Preferably, a memory bank is set up to maximize the number of negative samples. This application defines three perspectives, so that multiple perspectives from a single sample are considered positive samples, and the samples in the memory bank are considered negative samples. This allows for efficient retrieval of k negative samples to pair with each positive sample. The memory bank is dynamically updated using dynamically calculated features. The advantage of setting up a memory bank is that it provides more negative sample pairs.

[0064] Example 2: Due to the personalized differences in emotional expressions of individuals in different time periods, the existing models cannot meet the requirements of cross-session emotion recognition. Therefore, establishing a cross-session emotion recognition model has become one of the major challenges of EEG emotion recognition. This application uses the differential entropy features, brain asymmetry features and fusion features of EEG signals, and uses multi-perspective contrastive learning to maximize the similarity of the same sample under different perspectives for feature representation. A cross-session EEG emotion recognition model based on multi-perspective contrastive learning is proposed to achieve cross-session EEG emotion recognition. Simulation results show that the cross-session recognition accuracy of the method proposed in this application on the SEED and SEED-Ⅳ datasets can reach 92.76% and 79.42% respectively, with standard deviations of 3.91 and 9.56, showing stronger recognition accuracy and generalization than other methods. The specific experimental description is given below.

[0065] 1. Dataset Description

[0066] The SEED dataset is a publicly available EEG signal dataset that is widely used for emotion recognition. The dataset contains EEG data from 15 subjects, each of whom was asked to conduct three experiments at different times, with an interval of more than one week between sessions. The inventors' team prepared movie clips that could trigger positive, neutral, and negative emotions in the subjects. There were five clips for each emotion, with an average duration of 226 seconds, for a total of 15 movie clips. Each session involved watching 15 movie clips. The experimental process included: a 5-second prompt, a 4-minute movie clip, a 45-second self-assessment time, and a 15-second rest time, stimulating the corresponding emotions through immersive viewing.

[0067] The inventors' team used a 10-20 system to collect 62 channels of EEG signals at a sampling frequency of 1000Hz. The publicly available SEED dataset had been downsampled to 200Hz and filtered from 0 to 75Hz. The dataset also provided preprocessed experimental data. The differential entropy features, further smoothed using the linear dynamic system (LDS) method, were selected as experimental samples. In the SEED dataset, EEG signals were extracted from each sample in five frequency bands (1-3Hz, 4-7Hz, 8-13Hz, 14-30Hz, and 31-50Hz) using a non-overlapping 1-second time window. Their differential entropy features were calculated separately. This embodiment connects the differential entropy features of 5 frequency bands of 62 channels. To provide a more stable DE feature, reduce the impact of noise randomness, and ensure the number of samples, this embodiment selects the DE features of adjacent 2s for splicing, resulting in a feature vector dimension of 62*10. Due to the large number of 64-channel EEG signal measurement devices currently available, to facilitate subsequent research, pandas (0 padding) is used to uniformly pad the feature vector to 64*10, and the differential entropy feature serves as the input of encoder 1. For the left-right asymmetric feature (27*10) and the front-back asymmetric feature (23*10) after temporal splicing, the two asymmetric features are spliced ​​to obtain a 50*10 feature. The asymmetric feature serves as the input of encoder 2. To ensure the processability and consistency of all data sets, the input is consistent with the input in the differential entropy feature and uniformly padded to 64*10.

[0068] The SEED-IV dataset is also a publicly available EEG signal dataset widely used for emotion recognition. This dataset contains EEG signals from 15 subjects. In each session, participants were presented with movie clips that evoked four emotions: happiness, sadness, neutrality, and fear. Each emotion corresponded to four different clips, and each subject watched 24 movie clips in each session.

[0069] The public SEED-IV dataset uses the same preprocessing method as the SEED dataset, and this example also uses the same preprocessing method. The only difference is that the SEED-IV dataset uses non-overlapping 4-second time windows as a sample, and calculates differential entropy features for different frequency bands. Compared to the SEED dataset, the DE features extracted from the SEED-IV dataset include sufficient temporal features. To ensure sufficient data volume for model training, the input is not spliced ​​in time. The feature dimension of a sample is 62*5, consistent with the SEED dataset, and padded to 64*10. The same applies to asymmetric features.

[0070] 2. Experimental Details

[0071] For each subject, the session is divided into a training set and a test set. One session is selected as the test set, and the other sessions are used as the training set. Repeat the above operation until all sessions are used as test sets. The model training of this embodiment uses Python 3.8 programming language and pytorch development framework to implement the model. For the encoder, according to the dimension of the input data and the size of the dataset, a network model with a 16-layer residual structure is designed. The hidden dimension of the GRU is set to 128. The temperature parameter in the contrastive learning is set to 0.07, the momentum parameter is designed to 0.5, and the learning rate is set to 0.001. When the loss does not decrease for three consecutive times, the learning rate is halved. The batch size is 128, 100 epochs are trained in the contrastive learning stage, and 60 epochs are trained in the second classification stage.

[0072] Table 1 Encoder network parameter structure

[0073]

[0074] 3. Experimental Results and Analysis

[0075] To demonstrate the effectiveness of this method, the results of the cross-session experiment on the SEED dataset were compared with those of the DAN[1](deep adaptive networks), MS-MDA[2](Multisource marginal distribution adaptation), JDA[3](latent representation similarity), MS-ADAN[4](Multisource Associate Domain Adaptation), MS-ADRT[5](Multi-source Attention-based Dynamic Residual Transfer), and FMLAN[6](Fine-grained Mutual Learning Adaptation Network). Table (2) lists the average cross-session accuracy of each subject in the SEED dataset. Tables (3) and (4) list the average cross-session accuracy and standard deviation of different methods on the SEED and SEED-IV datasets.

[0076] [1]Li H,Jin Y M,Zheng W L,et al.Cross-subject emotion recognitionusing deep adaptation networks[C] / / Neural information processing:25thinternational conference,ICONIP 2018,Siem Reap,Cambodia,December 13–16,2018,proceedings,part V 25.Springer International Publishing,2018:403-413.

[0077] [2]Chen H,Jin M,Li Z,et al.MS-MDA:Multisource marginal distributionadaptation for cross-subject and cross-session EEG emotion recognition[J].Frontiers in Neuroscience,2021,15:778488.

[0078] [3]Li J,Qiu S,Du C,et al.Domain adaptation for EEG emotionrecognition based on latent representation similarity[J].IEEE Transactions onCognitive and Developmental Systems,2019,12(2):344-353.

[0079] [4]She Q,Zhang C,Fang F,et al.Multisource associate domain adaptationfor cross-subject and cross-session EEG emotion recognition[J].IEEETransactions on Instrumentation and Measurement,2023,72:1-12.

[0080] [5] Jiang W, Meng G, Jiang T, et al. Generalization across subjects and sessions for EEG-based emotion recognition using multi-source attention-based dynamic residual transfer [C] / / 2023International joint conference onneuralnetworks(IJCNN). IEEE, 2023:1-8.

[0081] [6]Yu P,He

[0082] Table 2. Accuracy of each subject across sessions in the Seed dataset (%)

[0083]

[0084] Table 3 Average accuracy and standard deviation of each method across sessions on the Seed dataset (%)

[0085]

[0086] Table 4 Average accuracy and standard deviation of each method across sessions on the Seed-IV dataset (%)

[0087]

[0088] Table 2 shows that the performance of our method on different subjects varied, with the best performance on Subject 13, reaching 98.23%, and the worst performance on Subject 2, at 88.16%. Tables 3 and 4 show that compared to other methods, our method improved the average accuracy of the best experimental results on the SEED dataset by approximately 0.82 percentage points, and the average accuracy on the SEED-IV dataset by approximately 0.64 percentage points. These results demonstrate that applying a multi-view contrastive learning model to cross-session tasks helps the entire model capture more global and fine-grained features, improving model performance.

[0089] Figure (3) shows the performance of the model using different sessions as test sets. It can be seen that the accuracy of the SEED dataset is significantly higher than that of the SEED-IV dataset. This is because the SEED-IV dataset has four categories while the SEED dataset has three categories. The more categories increase the difficulty of classification. The accuracy of using session 1 as the test set is the highest, and the accuracy decreases gradually. The accuracy of session 3 as the test set is the lowest. The inventors analyzed that the emotional characteristics of the subjects are most obvious when they are exposed to emotional video stimuli in the first session. After multiple exposures to the stimuli, the characteristics will gradually weaken, affecting the recognition results.

[0090] In order to verify that the method of this application extracts emotion features that are more conducive to recognition, the features of the subjects were visualized by t-SNE. As shown in Figure (4), compared with using only multi-dimensional features, the contrastive learning method used in this application is more conducive to emotion recognition. This shows that the proxy task of narrowing the distance between different perspectives of the same sample in contrastive learning helps to complete downstream tasks.

[0091] In addition, this embodiment also conducted an ablation experiment on the feature perspective. The accuracy of using only DE (differential entropy) features and asymmetric features was lower than the cross-session recognition results of the multi-perspective model after the fusion features of this application. This shows that compared with a single feature, the feature fusion of multiple perspectives and data features of more dimensions enable the model to capture signal characteristics more comprehensively.

[0092] Table 5 Feature ablation experimental results

[0093]

[0094] To demonstrate the effectiveness of the network structure GUR-AT of the present application, this embodiment also conducted an ablation experiment. The experimental results show that compared with using only a single network model, GUR-AT can more effectively focus on the important parts of the data while paying attention to the temporal connection of the signal data, reducing the impact of redundant data and more effectively capturing the characteristics of the subjects.

[0095] Table 6 Model ablation experiment results

[0096]

[0097] In summary, this application proposes a cross-session EEG emotion recognition method based on multi-view contrastive learning. The core of this method lies in characterizing EEG signal differential entropy features, brain asymmetry features, and their fusion features using the GUR-AT model, obtaining feature representations in the time, frequency, and spatial domains. Secondly, multi-view contrastive learning maximizes the similarity of the same sample under different viewpoints and minimizes the differences between different samples to achieve feature representation, thus enabling cross-session EEG emotion recognition. This model aims to obtain richer and more comprehensive emotion features, thereby finding an embedding space that can represent the underlying features of the sample and achieve emotion recognition. This application validates the model's effectiveness on the SEED and SEED-IV datasets, and also conducts ablation experiments. The experimental results show that this method achieves high recognition accuracy on both the SEED and SEED-IV datasets, achieving an average recognition accuracy of 92.76% on the SEED dataset and 79.42% on the SEED-IV dataset, surpassing some current state-of-the-art methods. Furthermore, ablation experiments demonstrate the effectiveness of the multi-view features and GUR-AT model designed in this application. Therefore, applying the method of the present application to the cross-session recognition of EEG signals has great application value.

[0098] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A cross-session EEG emotion recognition method based on multi-view contrastive learning, its characteristic steps include: Acquiring differential entropy features and brain asymmetry features of the EEG signal, wherein the brain asymmetry features include left-right asymmetry features and front-back asymmetry features; The differential entropy feature and brain asymmetry feature are encoded respectively through the GRU-AT model to obtain the differential entropy feature perspective V DE , Perspective on Brain Asymmetry V AS , the differential entropy feature perspective V DE and the brain asymmetry feature perspective V AS The two are fused to obtain the fusion feature perspective V fu Based on multi-view contrast learning, the differential entropy feature perspective V DE , Perspective on Brain Asymmetry V AS and fusion feature perspective V fu As three perspectives of contrastive learning, the contrastive loss function is calculated, and the GRU-AT model encoder is trained and optimized by minimizing the contrastive loss function. Finally, the optimized GRU-AT model encoder is connected to the classifier for fine-tuning. During this fine-tuning process, the encoder does not perform gradient propagation to complete emotion recognition.

2. The method for cross-session EEG signal emotion recognition based on multi-view contrastive learning according to claim 1 is characterized in that: The GRU-AT model includes: The two residual network structure encoders Encoder-1 and Encoder-2 are used to encode the differential entropy features and brain asymmetry features respectively; Two GRU modules to capture long-term dependencies in EEG signal time series; Three attention mechanism modules are used to weight the features output by the GRU module.

3. The method for cross-session EEG signal emotion recognition based on multi-view contrastive learning according to claim 2 is characterized in that: The specific process implemented in the GRU-AT model is: the differential entropy feature x t Enter the encoder Encoder-1 to get the representation vector Z1(t)=g enc1 (x t ), and then sent to the first GRU module to obtain the hidden layer state C1(t)=g GRU (Z1(t)), and then the differential entropy feature perspective V is obtained through the first attention mechanism module DE =g at1 (c1(t)); the brain asymmetry feature x t 'Send to encoder Encoder-2 to get the representation vector Z2(t)=g enc2 (x t '), and then the brain asymmetric feature perspective V is obtained through the second attention mechanism module As =g at2 (g enc2 (x t ')); the differential entropy feature perspective V DE Asymmetric features of the brain V AS The fused feature V fu =V DE +V As After being sent to the second GRU module for processing, the output is processed by the third attention mechanism module to obtain the final fusion feature perspective V fu .

4. The method for cross-session EEG signal emotion recognition based on multi-view contrastive learning according to claim 1 is characterized in that: The multi-view contrast learning is specifically as follows: given the view angles V1 and V2, one view angle V1 is fixed, and positive samples and negative samples are enumerated from the other view angle V2. The loss function Expressed as formula (5); fix a perspective V2, and enumerate positive samples and negative samples from another perspective V1, the loss function It is expressed as formula (6); in, is a sample with fixed view angle V1, is a sample with a fixed view angle V2, j = 1 is a positive sample, j > 1 is a negative sample; k represents the number of negative samples; h θ ({v1, v2}) represents the distance between the two views, measured by cosine similarity, and θ represents the parameters that need to be trained in the encoder. The loss of the two views is expressed as L(V1, V2), as shown in Equation (8): Among them, V1 and V2 represent different perspectives; This method has a differential entropy feature perspective V DE , Perspective on Brain Asymmetry V AS and fusion feature perspective V fu There are three perspectives, that is, the number of perspectives M = 3, so the two-perspective loss is calculated by taking two perspectives as a pair, and the final loss function L all It is expressed as formula (9):

5. The method for cross-session EEG signal emotion recognition based on multi-view contrastive learning according to claim 4 is characterized in that: The distance h between the two viewing angles θ The calculation method of ({v1,v2}) is as follows: in Representative sample v1 passes The obtained feature representation is Representative sample v2 passes The obtained feature representation, τ is the temperature hyperparameter.

Images