An electroencephalogram emotion recognition method and device based on an adversarial network and soft negative sample contrast learning

Abstract

This invention relates to a cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning, comprising the following steps: preprocessing an EEG signal dataset; constructing a pre-trained model, wherein the pre-trained model includes an encoder, a projector, and a discriminator of a generative adversarial network using a contrastive learning method; training the pre-trained model; and using the pre-trained encoder to achieve cross-subject EEG emotion recognition. This invention effectively improves the encoder's ability to extract cross-subject generalization features by combining the encoder with the projector and discriminator for adversarial training; and designs a contrastive loss function SDLoss that incorporates soft negative samples, enhancing the encoder's ability to capture shared neural responses under the same video stimuli, resulting in more stable recognition accuracy and stronger generalization performance in cross-subject emotion recognition tasks.

Description

Technical Field

[0001] This invention belongs to the field of EEG signal emotion recognition technology, and specifically relates to a cross-subject EEG emotion recognition method and device based on adversarial networks and soft negative sample contrastive learning. Background Technology

[0002] EEG emotion recognition aims to objectively assess emotional states by analyzing an individual's electroencephalogram (EEG) signals, and has significant application value in fields such as mental health monitoring, human-computer interaction, and clinical diagnosis. However, significant individual differences exist among subjects in emotion perception and neural response patterns, leading to domain bias in data distribution. This makes it difficult for models trained on source subject data to generalize to unseen target subjects, severely limiting the practical application of the technology.

[0003] In recent years, contrastive learning, as an unsupervised representation learning method, has shown potential in EEG emotion recognition. Existing techniques typically employ strategies such as instance discrimination or temporal segment comparison to learn discriminative features by maximizing the consistency between different augmented views of the same sample. However, traditional contrastive learning methods have significant limitations in cross-subject scenarios for EEG emotion recognition: the negative sample construction strategy is too simplistic, usually treating all non-homologous samples as negative samples, resulting in insufficient ability of the model to capture shared neural patterns under the same video stimuli, thus limiting further improvement in the model's generalization performance.

[0004] Meanwhile, generative adversarial networks (GANs) have been introduced into EEG emotion recognition. These typically employ adversarial training between the generator and discriminator to make the generated EEG signals approximate real EEG signals in terms of distribution, thereby expanding the dataset size and improving recognition performance. However, this approach doesn't truly solve the problem of existing models struggling to extract high-quality generalization features. Furthermore, traditional methods often perform classification directly after feature extraction, failing to fully consider the temporal dynamics and noise interference of EEG signals, thus affecting the model's robustness in complex environments. Specifically, existing classifiers mostly employ a simple stacked structure of multilayer perceptrons (MLPs), which, while achieving basic classification functions, lacks an effective constraint mechanism for high-dimensional EEG features.

[0005] In summary, existing technologies mainly suffer from the following problems: First, the negative sample sampling strategy of traditional contrastive learning is rigid and difficult to effectively capture shared emotional representations across subjects; second, the fusion mechanism of adversarial training and contrastive learning is imperfect and lacks collaborative optimization of the encoder and discriminator; third, existing methods have weak temporal denoising and feature smoothing capabilities for EEG signals, and their anti-interference performance needs to be improved; fourth, existing classifiers have a simple structure and lack regularization constraints, resulting in insufficient discriminative stability of the model in complex environments, which limits further improvement in the accuracy of emotion recognition. Summary of the Invention

[0006] The purpose of this invention is to provide a cross-subject EEG emotion recognition method and device based on adversarial networks and soft negative sample contrastive learning, which enhances the model's ability to capture shared neural responses under the same video stimuli, enabling the encoder to effectively extract subject-invariant features, thereby improving the accuracy of emotion recognition.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning includes the following steps:

[0009] Preprocessing EEG signal datasets;

[0010] Construct a pre-trained model, which includes an encoder, a projector, and a discriminator of a generative adversarial network using a contrastive learning method;

[0011] The pre-trained model was trained using the first, second, and third loss functions to train the encoder and projector, and the fourth loss function to train the discriminator. The encoder and projector were trained as one side, and the discriminator as the other side, and the two sides were trained adversarially. The first and second loss functions were contrastive learning loss functions. The positive sample pairs of the second loss function consisted of EEG signals from two subjects under the same emotion induction condition. The negative sample pairs of the second loss function were soft negative sample pairs, which consisted of EEG signals from other emotion induction conditions with the same emotion classification label as the positive sample in the current positive sample pair.

[0012] A pre-trained encoder was used to achieve cross-subject EEG emotion recognition.

[0013] Preferably, training the pre-trained model specifically includes the following steps:

[0014] The preprocessed EEG signals of two subjects are input into the encoder. The EEG signal of one subject is labeled as Subject A and the EEG signal of the other subject is labeled as Subject B. The output of the encoder is used as the input of the projector. The projector then outputs two two-dimensional features, which are then input into the discriminator. The discriminator then outputs two one-dimensional score arrays. The third loss function and the fourth loss function are calculated for each one-dimensional score array. The discriminator is optimized using the two fourth loss function values.

[0015] The preprocessed EEG signals of two subjects are combined and input into the encoder. The output of the encoder is used as the input of the projector. The projector outputs a two-dimensional feature. The two-dimensional feature is normalized and then the dot product is used to calculate the similarity matrix. The contrast loss is calculated using the first loss function and the second loss function. The encoder and the projector are optimized by weighting and adding the contrast loss with two third loss functions.

[0016] Preferably, the calculation of the comparative loss using the second loss function specifically includes the following steps:

[0017] Given a similarity matrix of positive sample pairs Similarity matrix of soft negative samples Where x represents the number of soft negative samples corresponding to each positive sample, calculate the similarity difference matrix:

[0018]

[0019] in, It is a similarity difference matrix. It is a row vector in which all elements are equal to 1;

[0020] Second loss function Similarity difference matrix with ReLU activation Average value:

[0021]

[0022] Preferably, the first loss function is a classic contrastive loss function. The third loss function is the mean squared error loss function. The fourth loss function is the binary cross-entropy loss function. .

[0023] Preferably, the encoder includes a hierarchical normalization layer and a two-dimensional convolutional layer. The hierarchical normalization layer is used to stabilize deep network training, and the two-dimensional convolutional layer is used to simultaneously capture the correlation features between EEG signal channels and between time sequences to form deep emotional representations.

[0024] The projector includes an exponential linear unit activation function, an average pooling layer, a hierarchical normalization layer, and a two-dimensional convolutional layer. The exponential linear unit activation function is used to introduce nonlinearity, the average pooling layer is used to aggregate spatiotemporal information, the hierarchical normalization layer is used to stabilize the distribution, and the two-dimensional convolutional layer is used to map features to a contrastive learning space to generate a low-dimensional vector suitable for calculating similarity.

[0025] Preferably, the discriminator includes a one-dimensional convolutional layer, a leaky ReLU activation function, a max pooling layer, a flattening layer, a fully connected layer, and a sigmoid activation function. The discriminator mainly uses a fully connected layer, supplemented by a one-dimensional convolutional layer to extract local patterns in the sequence. The leaky ReLU activation function avoids neuron inactivation. The max pooling layer filters significant features. After the flattening layer, the probability is output through the sigmoid activation function to determine whether the feature pair comes from the same sample.

[0026] Preferably, the method of using a pre-trained encoder to achieve cross-subject EEG emotion recognition specifically includes the following steps: after the pre-trained model is trained, the optimal parameters of the encoder are transferred to the new encoder;

[0027] The preprocessed EEG signal is standardized and input into a new encoder to extract differential entropy features. An adaptive global-local hybrid normalization method is used to dynamically standardize the differential entropy features to obtain normalized differential entropy features. Then, linear dynamic smoothing is used to process the normalized differential entropy features.

[0028] Build a classification model;

[0029] The smoothed differential entropy features are input into the classification model, and classification training is performed to obtain the trained classification model.

[0030] Repeat the above steps with the EEG signal to be identified to obtain the smoothed differential entropy feature of the EEG signal. Input the smoothed differential entropy feature into the trained classification model to realize the emotion recognition of the EEG signal to be identified.

[0031] Preferably, the classification model includes a classifier, which includes a fully connected layer, a one-dimensional batch normalization layer, a ReLU activation function, and a random dropout layer. The one-dimensional batch normalization layer is used to normalize the inputs of each layer, and the dropout layer is used to randomly drop the outputs of some neurons. During training, the dropout layer randomly drops the outputs from neurons with a predetermined probability. During testing and recognition, the dropout layer does not perform the dropout operation.

[0032] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it performs the steps in the cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning described above.

[0033] In another aspect, the present invention provides a cross-subject EEG emotion recognition device based on adversarial networks and soft negative sample contrastive learning, comprising:

[0034] Memory, used to store software applications.

[0035] A processor is configured to execute the software application, wherein each program of the software application correspondingly performs the steps in the cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning described above.

[0036] This invention employs a joint encoder and projector, employing adversarial training against a discriminator to drive the encoder to extract subject-invariant features with excellent generalization ability. Building upon this, the concept of soft negative samples is introduced. By optimizing the construction strategy of positive and negative sample pairs, the model's ability to distinguish features generated by different videos with the same emotion classification label, as well as its ability to capture shared neural responses under the same video stimuli, is improved. Furthermore, this invention designs an MLP classifier that combines multiple batch normalization layers and lossy layers to enhance the model's generalization ability and prevent overfitting. This method significantly improves the model's generalization performance and has high practical value and application prospects. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the algorithm framework of the present invention;

[0038] Figure 2 A schematic diagram of the structure of the contrastive learning encoder, contrastive learning projector, discriminator, and classifier;

[0039] Figure 3 Two methods for constructing positive and negative sample pairs for contrastive loss functions;

[0040] Figure 4 This is a flowchart illustrating the algorithm of the present invention. Detailed Implementation

[0041] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0042] like Figure 1 As shown, the present invention discloses a cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning, comprising the following steps:

[0043] S1, Preprocessed EEG signal dataset;

[0044] Specifically, the preprocessing of EEG signals includes the following steps:

[0045] The EEG signals were downsampled, with the sampling rate of the SEED and MPED datasets reduced to 200 Hz and the sampling rate of the FACED dataset reduced to 250 Hz.

[0046] The EEG signal was subjected to a 4 Hz high-pass filter and a 47 Hz low-pass filter.

[0047] For channels with poor data quality, interpolation techniques are used to estimate their true signal values. When the proportion of bad channels exceeds 30%, data from adjacent channels are used for interpolation repair.

[0048] The reference electrodes for the original EEG data are redefined. The reference point is the average potential of all electrodes at each time step. After rereference, the sum of the whole brain data at each time step is 0.

[0049] More specifically, this embodiment uses the SEED dataset, which contains three basic emotions (positive, neutral, and negative) and includes 15 participants, each of whom participated in the experiment three times.

[0050] S2. Construct a pre-trained model, which includes an encoder, a projector, and a discriminator of a generative adversarial network using a contrastive learning method.

[0051] Specifically, the structure of the contrastive learning encoder, contrastive learning projector, and discriminator is as follows: Figure 2 As shown, the contrastive learning encoder uses a combination of hierarchical normalization layers and two-dimensional convolutional layers. The normalization stabilizes the deep network training, while the two-dimensional convolution simultaneously captures the correlation features between EEG signal channels (spatial) and between time sequences (temporal), forming a deep emotional representation.

[0052] The contrastive learning projector sequentially aggregates spatiotemporal information through an average pooling layer, introduces nonlinearity through an exponential linear unit activation function, and stabilizes the distribution through a hierarchical normalization layer. Finally, a two-dimensional convolutional layer maps the features to the contrastive learning space, generating a low-dimensional vector suitable for calculating similarity.

[0053] The discriminator is based on a fully connected layer, supplemented by a one-dimensional convolutional layer to extract local patterns of the sequence, a leaky ReLU to avoid neuron inactivation, and a max pooling layer to screen significant features. After flattening, the probability is output through a sigmoid activation function to determine whether feature pairs come from the same sample.

[0054] S3. Execute the training of the pre-trained model. Train the encoder and projector using the first loss function, the second loss function, and the third loss function, and train the discriminator using the fourth loss function. The encoder and projector are treated as one side, and the discriminator is treated as the other side. The two sides are trained adversarially. The first loss function and the second loss function are contrastive learning loss functions. The positive sample pair of the second loss function consists of EEG signals induced by the same video in two subjects. The negative sample pair of the second loss function is a soft negative sample pair, which consists of EEG signals of other videos with the same emotion classification label as the positive sample in the current positive sample pair.

[0055] Specifically, the first loss function is the classic contrastive loss function. The second loss function is the similarity difference loss function that introduces soft negative samples. (Similarity Difference Loss), the third loss function is the mean squared error loss function. (Mean Squared Error Loss), the fourth loss function is the binary cross-entropy loss function. (Binary Cross-Entropy Loss);

[0056] The mean squared error loss function is:

[0057]

[0058] Where N represents the sample size, This represents the probability that the model predicts the i-th sample belongs to the positive class. This indicates a specified value of 0.5.

[0059] The binary cross-entropy loss function is:

[0060]

[0061] N represents the sample size. It is the true label of the i-th sample in the binary classification task. It can take the value 0 or 1. This represents the probability that the model predicts the i-th sample belongs to the positive class.

[0062] More specifically, the SEED dataset uses 15 participants for five-fold cross-training. Each fold contains EEG signals from 12 participants as the training set, and the remaining 3 participants' EEG signals as the test set. During input, one participant's entire trial is considered a participant unit, with a total of 45 participant units. A for loop generates a participant pair index array with a variable step size. EEG signals from two participant units are loaded sequentially according to the participant pair index array. Assuming a participant pair is designated as participant A and participant B, each participant has N video-evoked EEG data points. The batch size for these two participant units can be defined as follows:

[0063]

[0064]

[0065] like Figure 1 As shown in (a), when training the pre-trained model, the encoder needs to be input in two forms: one form is that the EEG signals of two subjects are input separately, in which case the EEG signal of one subject is labeled as subject A and the EEG signal of the other subject is labeled as subject B.

[0066] The encoder's output is used as the projector's input, and the projector then outputs two two-dimensional features. These two features are then input into a discriminator, which outputs two one-dimensional score arrays. For each score array, f1 = ... and f2 = .

[0067] When calculating f1 for subject A's score array, y is 1; when calculating f1 for subject B's score array, y is 0. The two f1 values ​​are then added together to optimize the discriminator.

[0068]

[0069] When calculating f2 for both score arrays, y is set to 0.5. The two f2 values ​​are then summed and weighted together with the contrastive loss to optimize the encoder and projector.

[0070]

[0071] in, for Scaling parameters.

[0072] Contrast loss is calculated in another form, by merging the EEG signals of two subjects and inputting them into the encoder:

[0073]

[0074] Finally, the projector outputs a two-dimensional feature vector, which is then input into the "contrast space of the dual-construction strategy." In this space, the features are first normalized, and then a dot product is performed to calculate the similarity matrix. Since the feature vectors have been normalized, the dot product value equals the cosine similarity between the two vectors. The feature matrix is ​​then used to construct positive and negative sample pairs using two different methods, and the contrastive loss is calculated using two different loss functions. Positive sample pairs typically consist of EEG signals induced by the same video from the EEG signals of two subject units, while negative sample pairs include EEG signals from positive samples and EEG signals induced by other videos.

[0075] Figure 3 This visually demonstrates two contrastive loss functions (the classic contrastive loss function). And the similarity difference loss function that introduces soft negative samples The positive and negative sample pairs are constructed in the same way for the first and second loss functions, consisting of EEG signals induced by the same video from two subjects. The negative sample pair consists of the positive sample in the current positive sample pair and other video-induced EEG signals; The negative sample pairs consist of EEG signals induced by other videos that have the same emotion classification label as the positive samples in the current positive sample pairs.

[0076] like Figure 1 As shown in (b), simulations were performed respectively. and The process of processing the similarity matrix. To represent it more intuitively, the similarity matrix can be imagined as an N×N matrix, where the numbers 1, 2, and 3 represent different sentiment classification labels. Positive sample pairs exist only along the diagonal because only EEG signals induced by the same video from two subjects can form positive sample pairs; all off-diagonal sample pairs are negative sample pairs. Then, all similarity values ​​corresponding to positive sample pairs are shifted backward. After temperature scaling, the cross-entropy is calculated. The formula is as follows:

[0077]

[0078] , where 𝑗+ represents the index of positive sample pairs, 𝑗 represents the index of negative sample pairs, and 𝜏 represents temperature, with a fixed value of 0.07;

[0079] Positive sample pairs remain unchanged, while negative sample pairs are limited to EEG signals that share the same sentiment classification label off-diagonally, i.e., soft negative sample pairs. Identical sample pairs are then merged, an activation function is applied to the similarity difference, and the average is calculated. The calculation process is as follows:

[0080] Given a similarity matrix of positive sample pairs Similarity matrix of soft negative samples Where x represents the number of soft negative samples corresponding to each positive sample, the similarity difference matrix is ​​calculated first:

[0081]

[0082] in, It is a similarity difference matrix. It is a row vector in which all elements are equal to 1, used for expansion. Then Defined as the average difference in similarity between ReLU activations:

[0083]

[0084] EEG segments evoked by different videos but sharing the same emotional label are defined as soft negative samples, forming soft negative sample pairs with their corresponding positive samples. A second loss function explicitly penalizes soft negative sample pairs where the similarity exceeds that of their corresponding positive sample pairs. This loss guides the encoder to actively suppress similarity within soft negative pairs, while maintaining maximum similarity within positive pairs through hierarchical relative similarity constraints, ultimately extracting more stable cross-subject basic emotional representations. The effectiveness of contrastive learning largely depends on the correct construction of positive and negative sample pairs. In the similarity matrix, this manifests as increasing the similarity of positive pairs and decreasing the similarity of negative pairs.

[0085] The formula for calculating the contrast loss is:

[0086]

[0087] in, for Scaling parameters.

[0088] S4. Using a pre-trained encoder, cross-subject EEG emotion recognition is achieved.

[0089] S41. After the pre-trained model is trained, the optimal parameters of the encoder will be transferred to the new encoder, and the projector will no longer be used. At this time, the preprocessed EEG signal will be input into the new encoder to output features suitable for the classification shape. Five-fold cross-input will also be performed to ensure the independence of the test samples.

[0090] S42. The new EEG signals are first standardized to eliminate differences in data distribution among subjects. The goal is to adjust the data distribution to a standard normal distribution with a mean of 0 and a variance of 1. For the input... Standardize the execution path. For each feature size... Calculate the global mean and variance for all samples and time points:

[0091]

[0092] Standardized data is obtained through the following methods:

[0093]

[0094] Where N is the number of batches in the data, T is the length of the time series, and F is the channel dimension. 𝜖 is a very small constant (e.g., 10⁻⁵) used to avoid numerical instability caused by division by zero.

[0095] Given standardized input First, use the encoder. Feature extraction:

[0096]

[0097] in, D and D represent the encoded Feature time length and feature dimension. For each sample i and feature dimension d, the differential entropy DE feature is calculated as follows:

[0098]

[0099] in, It is the time variance of the d-th feature dimension.

[0100] After extracting the differential entropy (DE) features, an adaptive global-local hybrid normalization (AGH-Norm) strategy is used to dynamically standardize these features. This strategy introduces a scalar mixing coefficient φ to balance the pre-computed global statistical priors and the local statistical properties of the current data distribution.

[0101]

[0102] Specifically, for the differential entropy DE feature vector of the t-th input sample First, calculate the mean of the fusion. and variance Here, 𝜆 is used as a global prior weight to control global statistics. Dynamic statistics of the current time step The mixing ratio between them. Initially, is set to 1 to ensure normalization stability. During training, can be gradually adjusted based on an exponential decay strategy, allowing the statistical estimate to smoothly transition from a previous dependence on the global distribution to a local data distribution. Finally, the features are normalized using the fused statistics, where is a minimal constant to prevent division by zero.

[0103] The normalized data is subjected to linear dynamic smoothing. Through bidirectional recursion of forward Kalman filtering and backward RTS smoothing, the state-space model of linear dynamic systems, Gaussian noise assumption, and Bayesian update principle are used to perform time-series modeling and optimal estimation of the input sequence, thereby achieving denoising and smoothing.

[0104] S43. Construct a classification model;

[0105] Specifically, the classification model includes an MLP classifier, the structure of which is as follows: Figure 2 As shown, this classifier combines multiple batch normalization layers and missing data layers, enhancing the model's generalization ability and preventing overfitting. The batch normalization layers stabilize the model's training process and accelerate convergence by normalizing the inputs of each layer. Its calculation formula is as follows:

[0106]

[0107] X is the input data, μ is the average of the mini-batch data, and σ 2 γ is the variance of the mini-batch data, β is the scaling parameter used to adjust the normalized data range, β is the bias parameter used to adjust the normalized data offset, and ε is a small constant used to prevent division by zero and ensure numerical stability. The dropout layer disrupts the complex cooperative adaptation relationships between neurons by randomly discarding the outputs of certain neurons, effectively preventing model overfitting. During training, the dropout layer randomly discards the outputs from neurons with a certain probability (0.5 in this example); during testing, the dropout layer does not perform the dropout operation to ensure model stability.

[0108] S44. Input the smoothed differential entropy features into the classification model, perform classification training, and obtain the trained classification model.

[0109] Specifically, the SEED dataset was used for leave-one-out training with 15 participants. Each fold contained EEG signals from 14 participants as the training set, and the remaining participant's EEG signal as the test set. The smoothed differential entropy (DE) features were input into the MLP classifier for training. The cross-entropy loss function was used. Accuracy is obtained by calculating the difference between the classifier's prediction and the true label.

[0110] S45. Repeat the above steps for the EEG signal to be identified to obtain the smoothed differential entropy feature of the EEG signal. Input the smoothed differential entropy feature into the trained classification model to realize the emotion recognition of the EEG signal to be identified.

[0111] The following experiments demonstrate the method of the present invention.

[0112] Table 1

[0113] Dataset SEED MPED FACED CLISA 86.4 ± 6.4 37.1 ± 6.3 43.7 ± 3.1 MSHCL <![CDATA[ 89.3 ± 4.4 ]]> 38.8 ± 7.1 45.7 ± 3.4 GCPL 80.7 ± 6.1 30.3 ± 5.7 36.9 ± 3.3 DCGNN+CDAN 88.1 ± 4.6 <![CDATA[ 40.6 ± 6.4 ]]> 17.5 ± 1.6 DCENet 87.6 ± 7.6 27.7 ± 5.3 - GMSS 86.5 ± 6.2 28.5 ± 4.4 - CL-CS 88.3 ± 8.9 - 43.4 ± 13.7 EmT-D 80.2 ± 11.5 - <![CDATA[ 57.9 ± 6.8 ]]> Method of the present invention 91.0±4.4 58.4±7.8 58.2±13.0

[0114] In the field of EEG-based emotion recognition, various methods have been proposed to improve recognition accuracy, such as CLISA, MSHCL, GCPL, DCGNN+CDAN, DCENet, GMSS, CL-CS, and EmT-D. The accuracy of the method of this invention was compared with the aforementioned methods on three EEG datasets (SEED, MPED, and FACED), and the results are shown in Table 1. The method of this invention exhibits excellent average accuracy on the three datasets, especially on the SEED dataset, achieving an accuracy of 91.0±4.4%, which is significantly improved compared to CLISA's 86.4±6.4%. This indicates that the method of this invention has significant advantages in extracting subject-invariant features and enhancing the model's generalization ability. Due to the inherent complexity and diversity of emotions in the MPED and FACED datasets, high-accuracy models still need to be developed. Nevertheless, the accuracy of the method of this invention on the MPED and FACED datasets is 58.4±7.8% and 58.2±13.0%, respectively, significantly higher than CLISA's 37.1±6.3% and 43.7±3.1%. This further verifies the effectiveness and robustness of the method of this invention in cross-subject emotion recognition tasks. Experimental results show that the method can effectively handle the diversity and complexity of large-scale datasets while maintaining stable recognition performance. The EmT-D method also achieved a similar accuracy of 57.9±6.8% on the FACED dataset, demonstrating good performance as well.

[0115] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps in the cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning described above.

[0116] In another aspect, the present invention also provides a cross-subject EEG emotion recognition device based on adversarial networks and soft negative sample contrastive learning, comprising:

[0117] Memory, used to store software applications.

[0118] A processor is configured to execute the software application, wherein each program of the software application correspondingly performs the steps in the cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning described above.

Claims

1. A cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning, characterized in that, Includes the following steps: Preprocessing EEG signal datasets; Construct a pre-trained model, which includes an encoder, a projector, and a discriminator of a generative adversarial network using a contrastive learning method; The pre-trained model was trained using the first, second, and third loss functions to train the encoder and projector, and the fourth loss function to train the discriminator. The encoder and projector were trained as one side, and the discriminator as the other side, and the two sides were trained adversarially. The first and second loss functions were contrastive learning loss functions. The positive sample pairs of the second loss function consisted of EEG signals from two subjects under the same emotion induction condition. The negative sample pairs of the second loss function were soft negative sample pairs, which consisted of EEG signals from other emotion induction conditions with the same emotion classification label as the positive sample in the current positive sample pair. A pre-trained encoder was used to achieve cross-subject EEG emotion recognition.

2. The cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning according to claim 1, characterized in that, The specific steps involved in training a pre-trained model are as follows: The preprocessed EEG signals of two subjects were input into the encoder, and the output of the encoder was used as the input of the projector. The projector then output two two-dimensional features, which were then input into the discriminator. The discriminator then output two one-dimensional score arrays. The third loss function and the fourth loss function were calculated for each one-dimensional score array. The discriminator was optimized using the two fourth loss function values. The preprocessed EEG signals of two subjects are combined and input into the encoder. The output of the encoder is used as the input of the projector. The projector outputs a two-dimensional feature. The two-dimensional feature is normalized and then the dot product is used to calculate the similarity matrix. The contrast loss is calculated using the first loss function and the second loss function. The encoder and the projector are optimized by weighting and adding the contrast loss with two third loss functions.

3. The cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning according to claim 1 or 2, characterized in that, The calculation of the comparative loss using the second loss function specifically includes the following steps: Given a similarity matrix of positive sample pairs Similarity matrix of soft negative samples Where x represents the number of soft negative samples corresponding to each positive sample, calculate the similarity difference matrix: in, It is a similarity difference matrix. It is a row vector in which all elements are equal to 1; Second loss function Similarity difference matrix with ReLU activation Average value:

4. The cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning according to claim 1 or 2, characterized in that, The first loss function is the contrastive loss function. The third loss function is the mean squared error loss function. The fourth loss function is the binary cross-entropy loss function. .

5. The cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning according to claim 1 or 2, characterized in that, The encoder includes at least one hierarchical normalization layer and at least one two-dimensional convolutional layer. The hierarchical normalization layer is used to stabilize deep network training, and the two-dimensional convolutional layer is used to simultaneously capture the correlation features between EEG signal channels and between time sequences to form deep emotional representations. The projector includes an exponential linear unit activation function, an average pooling layer, a hierarchical normalization layer, and a two-dimensional convolutional layer. The exponential linear unit activation function is used to introduce nonlinearity, the average pooling layer is used to aggregate spatiotemporal information, the hierarchical normalization layer is used to stabilize the distribution, and the two-dimensional convolutional layer is used to map features to a contrastive learning space to generate a low-dimensional vector suitable for calculating similarity.

6. The cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning according to claim 1 or 2, characterized in that, The discriminator includes a one-dimensional convolutional layer, a leaky ReLU activation function, a max pooling layer, a flattening layer, a fully connected layer, and a sigmoid activation function. The discriminator mainly uses a fully connected layer, supplemented by a one-dimensional convolutional layer to extract local patterns in the sequence. The leaky ReLU activation function avoids neuron inactivation, the max pooling layer filters significant features, and after the flattening layer, the probability is output through the sigmoid activation function to determine whether the feature pair comes from the same sample.

7. The cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning according to claim 1 or 2, characterized in that, The method of using a pre-trained encoder to achieve cross-subject EEG emotion recognition specifically includes the following steps: after the pre-trained model is trained, the optimal parameters of the encoder are transferred to the new encoder. The preprocessed EEG signal is standardized and input into a new encoder to extract differential entropy features. An adaptive global-local hybrid normalization method is used to dynamically standardize the differential entropy features to obtain normalized differential entropy features. Then, linear dynamic smoothing is used to process the normalized differential entropy features. Build a classification model; The smoothed differential entropy features are input into the classification model, and classification training is performed to obtain the trained classification model. Repeat the above steps with the EEG signal to be identified to obtain the smoothed differential entropy feature of the EEG signal. Input the smoothed differential entropy feature into the trained classification model to realize the emotion recognition of the EEG signal to be identified.

8. The cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning according to claim 7, characterized in that, The classification model includes a classifier comprising a fully connected layer, a one-dimensional batch normalization layer, a ReLU activation function, and a random dropout layer. The one-dimensional batch normalization layer is used to normalize the inputs of each layer, and the dropout layer is used to randomly drop the outputs of some neurons. During training, the dropout layer randomly drops the outputs from neurons with a predetermined probability. During testing and recognition, the dropout layer does not perform the dropout operation.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the steps of the cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning as described in any one of claims 1 to 8.

10. A cross-subject EEG emotion recognition device based on adversarial networks and soft negative sample contrastive learning, characterized in that, include: Memory, used to store software applications. A processor for executing the software application, wherein each program of the software application correspondingly performs the steps of the cross-subject EEG emotion recognition method based on adversarial networks and soft negative sample contrastive learning as described in any one of claims 1 to 8.

Images