Dynamic emotion recognition method and device based on unsupervised contrast graph learning electroencephalogram enhancement

Abstract

The invention discloses a dynamic emotion recognition method based on unsupervised contrast graph learning electroencephalogram enhancement, which comprises the following steps: carrying out preprocessing and segmentation processing on unmarked electroencephalogram signals, and segmenting the unmarked electroencephalogram signals into overlapped segments with fixed lengths; differential entropy features of each segment of signal are extracted; establishing enhanced graph data by taking electroencephalogram channels as nodes, differential entropy features as node attributes and Euclidean distances between the channels as edge weights; constructing a pre-training model containing graph convolution and long and short term memory network encoders and projectors; executing pre-training: inputting enhanced graph data into an encoder to obtain spatial-temporal characteristics, and mapping the spatial-temporal characteristics through a projector; constructing a dynamically updated feature queue; optimizing the model by comparing loss functions; migrating the pre-trained encoder to a classification network, and performing fine tuning by using label data; and finally realizing electroencephalogram signal emotion classification. According to the method, through comparison learning pre-training, the classifier can learn rich feature representation from the unlabeled data, and dependence on large-scale manual labeling data is effectively reduced.

Description

Technical Field

[0001] This invention belongs to the field of EEG signal recognition technology, and specifically relates to a dynamic emotion recognition method and device based on unsupervised contrastive image learning to enhance EEG. Background Technology

[0002] With the rapid development of neuroscience and psychology, the understanding and measurement of human emotions has become a cutting-edge field of interdisciplinary research. Electroencephalography (EEG), as a non-invasive neuroimaging technique, has become an important tool for emotion research due to its high temporal resolution and relatively low cost. It can accurately capture subtle changes in the brain's electrical activity under different emotional states, providing objective and reliable physiological indicators for emotion classification.

[0003] In EEG-based emotion recognition research, traditional machine learning methods and supervised deep learning models dominate. However, these methods typically require large amounts of labeled data to train the models, and the labeling process for EEG data is extremely time-consuming and labor-intensive. This not only limits the generalization ability of the models but also affects their application in clinical practice. In recent years, unsupervised learning methods have gradually gained attention, among which contrastive learning, as a powerful unsupervised learning method, has achieved significant results in fields such as computer vision, natural language processing, and medical data analysis. By learning the similarities and differences between samples, it can improve model performance without large-scale manual data labeling, providing a new approach to solving the challenge of EEG data labeling.

[0004] However, most EEG emotion recognition studies based on contrastive learning only focus on classification tasks of a single emotion dimension, and are significantly insufficient in recognizing multiple emotion states simultaneously, making it difficult to effectively meet the actual needs of complex emotion recognition tasks. Summary of the Invention

[0005] The purpose of this invention is to provide a dynamic emotion recognition method and device based on unsupervised contrastive image learning of EEG enhancement, aiming to solve the problems of small labeled data volume, limited model generalization ability, and insufficient ability to recognize multi-dimensional emotional states in existing methods. This method can learn rich feature representations using large-scale unlabeled EEG data, and then fine-tune them on target data to adapt to downstream emotion classification tasks.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A dynamic emotion recognition method based on unsupervised contrast map learning of EEG enhancement includes the following steps:

[0008] Unlabeled EEG signals were preprocessed and segmented to obtain multiple EEG signal fragments;

[0009] Enhanced signal fragments were generated using an EEG slice enhancement method, and differential entropy features were extracted based on the enhanced signal fragments to construct corresponding enhanced map data.

[0010] A pre-trained model and a classification model are constructed. The pre-trained model includes a first GCN-LSTM encoder and a projector. The classification model consists of a second GCN-LSTM encoder and a classifier. Both the first GCN-LSTM encoder and the second GCN-LSTM encoder are used to fuse spatial features extracted by a multi-layer Chebyshev graph convolutional neural network and temporal features extracted by a long short-term memory network.

[0011] The pre-trained model is trained, and the parameters of the second GCN-LSTM encoder are optimized using a contrastive learning method.

[0012] Perform training of the classification model;

[0013] A trained classification model is used to classify EEG signals for emotion.

[0014] Preferably, the training of the pre-trained model includes the following steps:

[0015] The augmented graph data is input into a graph convolutional neural network to extract spatial features. The obtained spatial features are then input into a long short-term memory network to extract temporal features. The temporal features are then input into a projector to obtain projected temporal features. The contrastive loss function is calculated based on the projected temporal features to train the pre-trained model.

[0016] Preferably, the EEG slice enhancement method involves generating two EEG enhancement signal segments with overlapping portions from each segmented EEG signal.

[0017] Preferably, the training of the pre-trained model further includes the following steps: constructing a dynamic memory using augmented graph data, storing the projected temporal features from the current batch and previous batches in a queue, and updating it with each batch iteration.

[0018] Preferably, the process of establishing enhanced map data based on differential entropy characteristics further includes the following steps:

[0019] By using EEG signal channels as nodes, differential entropy features as node features, calculating the Euclidean distance from each node to other nodes as edge weights, and determining the edge index of the node based on K nearest neighbors, the augmented graph data is obtained.

[0020] Preferably, the contrast loss function is:

[0021]

[0022] in, For batch size, For queue size, This is a similarity metric.

[0023] Preferably, the classifier is trained using a sigmoid activation function and a cross-entropy loss function.

[0024] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps in the dynamic emotion recognition method based on unsupervised contrastive image learning for EEG enhancement described above.

[0025] Another aspect of the present invention provides a dynamic emotion recognition device based on unsupervised contrastive image learning with EEG enhancement, comprising:

[0026] Memory, used to store software applications.

[0027] A processor is configured to execute the software application, wherein each program of the software application correspondingly performs the steps in the dynamic emotion recognition method based on unsupervised contrastive graph learning and EEG enhancement described above.

[0028] To address complex emotional tasks, this invention employs contrastive learning pre-training, enabling the classification model to learn rich feature representations from unlabeled data. This effectively reduces reliance on large-scale manually labeled data, improves the model's generalization ability, and enhances its adaptability and robustness across different datasets. Secondly, a dynamic memory mechanism is introduced to optimize negative sample selection. Dynamically updating the feature representations in the memory allows the classification model to better adapt to individual differences in EEG signals, mitigating data distribution bias. Furthermore, this invention combines Graph Convolutional Networks (GCN) and Long Short-Term Memory Networks (LSTM) to extract spatiotemporal features from EEG signal data, providing a more comprehensive understanding of the dynamic representation of emotional states. Finally, an EEG slicing enhancement strategy increases data diversity while maintaining signal functional connectivity, thereby improving emotional representation performance. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the algorithm framework of the present invention.

[0030] Figure 2 Diagram of EEG signal segmentation.

[0031] Figure 3 This is a schematic diagram of the algorithm flow of the present invention. Detailed Implementation

[0032] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0033] like Figure 1As shown, the present invention discloses a dynamic emotion recognition method based on unsupervised contrast map learning and EEG enhancement, comprising the following steps:

[0034] S1. Preprocess and segment the unlabeled EEG signals;

[0035] Specifically, in step S1, the preprocessing of the unlabeled EEG signal includes the following steps: removing the baseline signal of the EEG signal.

[0036] Because EEG signals are often subject to various interferences from the environment, physiology, or the electrodes themselves, such as power supply noise, muscle movements, and eye movements, removing these noises can improve signal quality, making the signal more accurate and reliable.

[0037] In this embodiment, the EEG signal contains 63 seconds of data, of which 3 seconds are baseline data. The baseline data is multiplied by 20, and the original EEG signal is subtracted from the multiplied baseline data to complete the noise reduction operation.

[0038] Specifically, such as Figure 2 As shown, the segmentation process includes the following steps: segmenting the EEG signal into overlapping segments. part.

[0039] More specifically, brainwave signals are divided into Segment, where the sliding window size It can be set to 6, and the sliding step can be set to 3 to complete the segmented operation.

[0040] S3. An enhanced signal fragment is generated using an EEG slice enhancement method, and differential entropy features are extracted based on the enhanced signal fragment to construct the corresponding enhanced map data;

[0041] Specifically, the EEG slice enhancement method involves generating two EEG enhancement signal segments with overlapping parts from each segmented EEG signal, and dividing the EEG slice into 0~T-1 seconds and 1~T seconds;

[0042] The EEG slice enhancement strategy is expressed as follows:

[0043]

[0044] in, and It is a window segmentation strategy that extracts portions of the original time series. The 0 to T-1 second signal and the 1 to T second signal were used to increase the sample size by window slicing, while maintaining the functional connectivity between EEG channels.

[0045] Specifically, the preprocessed slice length is The signal from 0 to T-1 seconds is defined as the enhanced signal. Signals from 1 to T seconds are defined as enhanced signals. In this way, enhanced EEG signals can be generated while maintaining the temporal dependence within the EEG time series. This method does not introduce additional noise, ensuring that the enhanced signal accurately reflects the dynamic characteristics of EEG activity.

[0046] Specifically, the steps for extracting differential entropy features include: extracting differential entropy features in five frequency bands for each segment of EEG signal transformed to the five angles in the time-frequency domain, where the five frequency bands are the δ band (1~3 Hz), θ band (4~7 Hz), α band (8~13 Hz), β band (14~30 Hz), and γ band (31~50 Hz).

[0047] The extracted differential entropy features are represented as follows:

[0048]

[0049] in, Let be the value of the differential entropy, representing the uncertainty of a continuous distribution. It is a continuous random variable, namely, electroencephalogram (EEG) signal. The mean of a Gaussian distribution is given. denoted as the standard deviation of the Gaussian distribution.

[0050] Specifically, an augmented map data is created for each differential entropy feature, that is, N augmented map data are created for the number of EEG signal segments;

[0051] More specifically, enhancement map data is constructed based on the differential entropy characteristics of each EEG signal, including the following steps:

[0052] By using the channels of the EEG signal as nodes, the differential entropy feature as the node feature, the Euclidean distance from each node to other nodes as the edge weight, and the edge index of the node is determined according to the K nearest neighbors, the augmented graph data is obtained.

[0053] More specifically, the diagram can be represented as ,in In this embodiment, each channel is used as a node. Represents the edges between nodes; augmented graph data contains node features and edge features; edge features include edge weights and edge indices, and the extracted differential entropy features are used as node features. The calculation method for edge features is as follows;

[0054] Calculate the Euclidean distance from each node to all other nodes to determine the edge weights. The Euclidean distance is:

[0055]

[0056] Edge selection for each node based on the K-nearest neighbor algorithm: Set a... and Parameters, select distance less than or equal to The value of a node determines the edge; thus, the edge weight and edge index of each node are stored.

[0057] The calculation method is as follows: .

[0058] S4. Construct a pre-trained model and a classification model, wherein the pre-trained model includes a first GCN-LSTM encoder and a projector; the classification model consists of a second GCN-LSTM encoder and a classifier; the first GCN-LSTM encoder and the second GCN-LSTM encoder are used to fuse the spatial features extracted by the multi-layer Chebyshev graph convolutional neural network and the temporal features extracted by the long short-term memory network.

[0059] Specifically, the GCN-LSTM encoder converts graph-structured signals into vectors, where the GCN layer extracts the spatial dependencies between EEG channels, and the LSTM layer analyzes the temporal dependencies between EEG feature vectors at fixed time intervals to obtain more effective temporal features.

[0060] The projector consists of two multilayer perceptrons (MLPs), including two linear layers with ReLU activation and BatchNorm regularization functions in between, to further explore the potential representations of EEG features.

[0061] S5, such as Figure 3 As shown, the pre-trained model is trained, and the parameters of the second GCN-LSTM encoder are optimized by a contrastive learning method.

[0062] Specifically, the training of the pre-trained model includes the following steps:

[0063] The augmented graph data is input into a graph convolutional neural network to extract spatial features. The obtained spatial features are then input into a long short-term memory network to extract temporal features. The temporal features are then input into a projector to obtain projected temporal features. The contrastive loss function is calculated based on the projected temporal features to train the pre-trained model.

[0064] More specifically, after the pre-trained model is trained, the parameters of the first GCN-LSTM encoder are fed into the second GCN-LSTM encoder.

[0065] More specifically, a dynamic memory is constructed, which stores the projected temporal features from the current batch and previous batches through a queue. This queue is updated with each batch iteration to increase the diversity of negative samples and improve the model's learning efficiency and stability. The dynamic memory is dynamically updated using a first-in, first-out (FIFO) principle, and the queue... The size is fixed and initialized to a random value. In this embodiment, a queue is set up. The size is 256 and is updated with each batch iteration to maintain feature diversity and model memory capacity.

[0066] Specifically, the pre-trained model is represented as follows:

[0067]

[0068]

[0069] in, For graph convolution operations, For Long Short-Term Memory (LSTM) network operations, This is a feature splicing operation.

[0070] The graph convolutional neural network (GNN) described above uses spectral filtering, a common operation based on graph signal processing principles. It primarily performs filtering operations on graph-structured data. Graph signals can be Fourier transformed using the graph's Laplacian matrix. This transformation forms the basis for frequency domain analysis of graph signals.

[0071] The Laplace matrix L of the graph: ,in It is an adjacency matrix. For degree matrix, , It is an identity matrix.

[0072] For a spatial signal, its graphical Fourier transform can be represented as: , It is a frequency domain signal. It is an orthogonal matrix, which can be obtained from... The singular value decomposition yields: ;

[0073] The inverse Fourier transform can be expressed as: ;

[0074] Signal and signal In the figure The convolution operation on can be defined as: , It is the Hadamard product.

[0075] It is a filter function, so the signal The filtering operation can be expressed as: ,in , yes eigenvalues.

[0076] because The computation is complex, so a K-order Chebyshev polynomial approximation is introduced. . ,in Chebyshev polynomial coefficients. The method for calculating Chebyshev polynomials can be expressed as follows:

[0077]

[0078] Based on the above, we can conclude that:

[0079] ,in,

[0080] The Long Short-Term Memory (LSTM) network described above can effectively capture the temporal features of sequential data and learn long-term dependencies within the data. LSTM defines three layers: the first layer is the input layer, which receives results from the GCN layer; the second layer is the hidden layer, which memorizes changes in the relationships between EEG signal channels; and the last layer outputs emotion recognition information. LSTM can better capture the dependencies between EEG signal channels.

[0081] The forget gate, input gate, and output gate of LSTM cells can be used to add and delete information to the cell state, and are defined as follows:

[0082]

[0083]

[0084]

[0085] The calculation method for the current memory state of a cell is defined as follows:

[0086]

[0087] Cell status:

[0088]

[0089] The unit's output calculation was defined as follows:

[0090]

[0091] in This represents the activation function. and These are the output and cell state of the previous cell in the LSTM, respectively. It is a bias.

[0092] Specifically, the contrastive loss function is based on the infoMax principle. It minimizes feature vectors from the same EEG slice to make them closer together, while feature vectors from different slices are further apart. The loss function is defined as follows:

[0093]

[0094] in, It refers to the batch size; It is the queue size; It is a similarity metric; in this embodiment, the cosine of the angle between two input vectors is used. (Queue) Perform random initialization and update as the batch iterates.

[0095] S6, such as Figure 3 As shown, the classification model is trained.

[0096] Specifically, the classification model is further trained and optimized using labeled EEG data, thereby enabling the classification model to classify EEG data more accurately.

[0097] More specifically, the classifier uses a non-linear sigmoid activation function to predict the correspondence between hidden vector representations and labels, and its loss function is the cross-entropy loss function, expressed as:

[0098]

[0099] in, For the classifier's prediction results, The classification labels are used. The cross-entropy function is a commonly used loss function used to measure the difference between the predicted values ​​and the true labels.

[0100] S7. Use a trained classification model to classify EEG signals for emotion.

[0101] The following experiments demonstrate the method of the present invention.

[0102] Table 1

[0103] Method Valence Arousal Dominance mean / std mean / std mean / std GCN+LSTM (baseline) 84.45 / 3.11 84.60 / 3.23 85.45 / 3.45 PNN 81.21 / 6.57 81.67 / 7.67 - FLDNet 83.85 / 11.34 78.22 / 10.14 77.52 / 10.14 CNN 85.53 / 2.06 85.88 / 1.62 66.85 / 15.06 Method of the present invention 91.18 / 4.24 92.60 / 3.20 92.55 / 2.42

[0104] Table 1 shows the performance of each method on the DEAP dataset under subject independence. The method of this invention performs best in all three dimensions: Valence accuracy is 91.18 (6.73 higher than GCN+LSTM), Arousal accuracy is 92.60 (7.99 higher), and Dominance accuracy is 92.55 (7.10 higher), with a smaller standard deviation and stronger stability. Among other methods, CNN outperforms the baseline in Valence and Arousal dimensions but performs poorly in Dominance, while PNN and FLDNNet generally underperform the baseline.

[0105] Table 2

[0106] Method Valence Arousal mean / std mean / std GCN+LSTM (baseline) 90.45 / 3.09 90.60 / 2.63 PCRNN 90.80 / 3.08 91.03 / 2.99 MM-ResLSTM 92.30 / 1.55 92.87 / 2.11 ACRNN 93.72 / 3.21 93.38 / 3.73 BiDCNN 94.38 / 2.61 94.72 / 2.57 Method of the present invention 95.58 / 2.69 96.07 / 2.71

[0107] Table 2 shows the performance of each method on the DEAP dataset in the Valence and Arousal dimensions under the subject-dependent condition. The method of this invention achieves the highest accuracy in both dimensions, at 95.58 and 96.07 respectively, with a smaller standard deviation (2.69 / 2.71), demonstrating stable performance and significantly outperforming other methods. BiDCNN is second (94.38 / 94.72), while ACRNN (93.27 / 93.38), MM-ResLSTM (92.30 / 92.87), and PC-RNN (90.80 / 91.03) all lag behind the method of this invention. Compared to the subject-independent scenario (Table 1), the accuracy of each method is significantly improved in the subject-dependent mode, further expanding the advantages of the method of this invention.

[0108] Table 3

[0109] Method Subject-dependent Subject-independent mean / std mean / std GCN+LSTM (baseline) 90.45 / 3.09 90.60 / 2.63 DBN 43.16 / 10.36 58.50 / 10.94 GELM 44.09 / 4.12 70.80 / 0.74 DGCNN 92.27 / 5.32 80.11 / 16.27 DGGN 97.28 / 2.70 83.84 / 10.26 Method of the present invention 97.99 / 2.69 87.50 / 6.97

[0110] Table 3 shows the accuracy and standard deviation of different methods on the SEED dataset under both subject-dependent and subject-independent conditions. Under the subject-dependent condition, the method of this invention achieved the highest accuracy of 97.99%, which is 0.71 higher than the second-ranked DGGN (97.28), and shows a significant advantage over other methods. Under the subject-independent condition, the accuracy of the method of this invention was 87.50%, which is 3.66 higher than DGGN (83.84), and also outperforms other methods. In summary, the method of this invention achieved excellent performance under both test conditions, indicating its strong adaptability and stability in emotion recognition tasks.

[0111] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps in the dynamic emotion recognition method based on unsupervised contrastive image learning for EEG enhancement described above.

[0112] Another aspect of the present invention provides a dynamic emotion recognition device based on unsupervised contrastive image learning with EEG enhancement, comprising:

[0113] Memory, used to store software applications.

[0114] A processor is configured to execute the software application, wherein each program of the software application correspondingly performs the steps in the dynamic emotion recognition method based on unsupervised contrastive graph learning and EEG enhancement described above.

Claims

1. A dynamic emotion recognition method based on unsupervised contrast map learning and EEG enhancement, characterized in that, Includes the following steps: Unlabeled EEG signals were preprocessed and segmented to obtain multiple EEG signal fragments; Enhanced signal fragments were generated using an EEG slice enhancement method, and differential entropy features were extracted based on the enhanced signal fragments to construct corresponding enhanced map data. A pre-trained model and a classification model are constructed. The pre-trained model includes a first GCN-LSTM encoder and a projector. The classification model consists of a second GCN-LSTM encoder and a classifier. Both the first GCN-LSTM encoder and the second GCN-LSTM encoder are used to fuse spatial features extracted by a multi-layer Chebyshev graph convolutional neural network and temporal features extracted by a long short-term memory network. The pre-trained model is trained, and the parameters of the second GCN-LSTM encoder are optimized using a contrastive learning method. Perform training of the classification model; A trained classification model is used to classify EEG signals for emotion.

2. The dynamic emotion recognition method based on unsupervised contrastive image learning and EEG enhancement according to claim 1, characterized in that, The training of the pre-trained model includes the following steps: The augmented graph data is input into a graph convolutional neural network to extract spatial features. The obtained spatial features are then input into a long short-term memory network to extract temporal features. The temporal features are then input into a projector to obtain projected temporal features. The contrastive loss function is calculated based on the projected temporal features to train the pre-trained model.

3. The dynamic emotion recognition method based on unsupervised contrastive image learning and EEG enhancement according to claim 1, characterized in that, The EEG slice enhancement method involves generating two EEG enhancement signal segments with overlapping portions from each segmented EEG signal.

4. The dynamic emotion recognition method based on unsupervised contrastive image learning and EEG enhancement according to claim 2, characterized in that, The training of the pre-trained model further includes the following steps: constructing a dynamic memory using augmented graph data, storing the projected temporal features from the current batch and previous batches in a queue, and updating it with each batch iteration.

5. The dynamic emotion recognition method based on unsupervised contrastive image learning and EEG enhancement according to claim 1, characterized in that, Based on the differential entropy characteristics, the enhancement map data is constructed, which also includes the following steps: By using EEG signal channels as nodes, differential entropy features as node features, calculating the Euclidean distance from each node to other nodes as edge weights, and determining the edge index of the node based on K nearest neighbors, the augmented graph data is obtained.

6. The dynamic emotion recognition method based on unsupervised contrastive image learning and EEG enhancement according to claim 2, characterized in that, The contrastive loss function is: ， in, For batch size, For queue size, This is a similarity metric.

7. The dynamic emotion recognition method based on unsupervised contrastive image learning and EEG enhancement according to claim 1, characterized in that, The classifier is trained using the sigmoid activation function and the cross-entropy loss function.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the steps of the dynamic emotion recognition method based on unsupervised contrastive graph learning and EEG enhancement as described in any one of claims 1 to 7.

9. A dynamic emotion recognition device based on unsupervised contrastive image learning with EEG enhancement, characterized in that, include: Memory, used to store software applications. A processor for executing the software application, wherein each program of the software application correspondingly performs the steps of the dynamic emotion recognition method based on unsupervised contrastive graph learning-enhanced EEG as described in any one of claims 1 to 7.

Images