Graph semi-supervised learning electroencephalogram emotion recognition method based on metric learning
Through the graph semi-supervised learning method based on metric learning, an EEG sentiment recognition model is constructed, which solves the problem of retraining the model in the existing technology, and realizes effective classification and efficient adaptation to new samples.
Patent Information
- Application Number
- CN202510249283.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-04
- Publication Date
- 2025-06-13
AI Technical Summary
The prior art requires all samples to participate in the training of EEG sentiment recognition model. When new samples are added, the training of the model needs to be re-trained, resulting in an increase in time overhead.
The graph semi-supervised learning method based on metric learning is used to construct the initial similar graph through the first metric function, and a semi-supervised EEG sentiment recognition model is constructed using the metric learning layer and graph neural network. When new samples are added, the training second metric function is used to calculate the similarity between the new node and the existing node, thereby expanding the edges of the similar graph and classifying them.
It avoids the time overhead of retraining the model, can be effectively applied to the classification of new emotional EEG samples, and improves the adaptability and efficiency of the model.
Smart Images
Figure CN120145148A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of electroencephalogram (EEG) emotion recognition, and particularly to a graph semi-supervised learning EEG emotion recognition method based on metric learning. Background Art
[0002] In recent years, semi-supervised EEG emotion recognition methods have gradually received extensive attention. Common methods include those based on consistency regularization, proxy labels, generative models, etc.
[0003] However, the existing technologies have the following problems: all samples are required to participate in the training of the EEG recognition model, and when new samples are added, the model needs to be retrained. Summary of the Invention
[0004] The purpose of the present invention is to provide a graph semi-supervised learning EEG emotion recognition method based on metric learning to solve the problem that the model needs to be retrained when new samples are added.
[0005] To achieve the above task, the present invention adopts the following technical solutions:
[0006] A graph semi-supervised learning EEG emotion recognition method based on metric learning, comprising:
[0007] Obtain an EEG dataset; divide the EEG dataset into a training set and an extended set, where some samples in the training set are labeled; use a first metric function to fit the relationship between samples in the training set, thereby constructing an initial similarity graph;
[0008] Construct a semi-supervised EEG emotion recognition model; the EEG emotion recognition model includes a metric learning layer and a graph neural network; use the initial similarity graph as the input of the EEG emotion recognition model, use the metric learning layer to learn a similarity matrix based on a learnable second metric function to update the similarity graph, and jointly input the updated similarity graph and the regularized sample features into the graph neural network to obtain the classification result of the sample;
[0009] Construct a total loss function for training the EEG emotion recognition model, where the total loss function includes a classification loss function for labeled samples and a regularization term for controlling the smoothness of the second metric function; train the EEG emotion recognition model to obtain a trained EEG emotion recognition model;
[0010] Input the extended set into the trained EEG emotion recognition model, construct new nodes with the samples in the extended set, add the new nodes to the similarity graph to expand the nodes of the similarity graph, and use the trained second metric function in the metric learning module to calculate the similarity between the new nodes and the existing nodes in the similarity graph to expand the edges of the similarity graph, and finally use the trained graph neural network to perform label propagation on the new nodes to obtain the corresponding classification result.
[0011] Further, the EEG dataset is a dataset composed of EEG data collected from different subjects under different emotion types; each EEG data contains time-domain information and spatial information; after the EEG data is normalized, it is divided into time windows per second, and each divided EEG data is used as a sample; the EEG dataset is divided into a training set and an extended set, where some samples in the training set have labels, and the label is the emotion type corresponding to the sample.
[0012] Further, the step of using the first metric function to fit the relationship between samples in the training set to construct an initial similarity graph includes:
[0013] Extract the differential entropy feature of each sample, where the differential entropy feature is the number of channels of the EEG data × the number of frequency bands; expand each differential entropy feature into a one-dimensional feature vector and use it as a node in the initial similarity graph;
[0014] Use the first metric function to calculate the similarity between different nodes, and use the similarity as the edge between nodes to obtain the initial similarity graph.
[0015] Further, the metric learning layer first performs regularization processing on all nodes in the initial similarity graph using the Dropout layer to obtain sample features to prevent overfitting; then multiplies the sample features by a learnable weight matrix, and uses the second metric function to calculate the similarity between nodes, thereby outputting the learned similarity matrix; uses the similarity matrix as the new edge between nodes to update the initial similarity graph to obtain the similarity graph after metric learning; inputs the similarity graph into the graph neural network to obtain the classification results of each sample.
[0016] Further, the graph neural network includes two graph convolutional layers. The input features pass through the first graph convolutional layer to obtain embedded features. The embedded features are then activated by the relu function and subjected to Dropout regularization, and then input into the second graph convolutional layer. After the final output embedded features are processed by the Softmax function, the classification results are obtained.
[0017] Further, the first metric function uses a cosine similarity function without parameters; the second metric function uses a cosine similarity function with learnable parameters, which is expressed as:
[0018]
[0019] where represents the similarity between nodes and , and w is a learnable parameter.
[0020] Furthermore, the total loss function is expressed as:
[0021] L = L P + L G ;
[0022] wherein, L P represents the classification loss function, which is the cross-entropy loss function calculated using the classification results of the graph neural network; L G is the regularization term, which is obtained by calculating the second similarity function in the metric learning layer and is expressed as follows:
[0023]
[0024] The electroencephalogram (EEG) emotion recognition model is trained using the backpropagation algorithm through the training set, and the trained EEG emotion recognition model is obtained after ten-fold cross-validation.
[0025] Furthermore, first, the differential entropy features of the samples in the extended set are extracted, and the differential entropy features are expanded into a one-dimensional feature vector and then added as a new node to the similarity graph constructed using the training set, thereby expanding the nodes of the similarity graph; second, the similarity between the new node and the existing nodes in the similarity graph is calculated using the trained second metric function as the edge between the nodes, thereby expanding the edges of the similarity graph.
[0026] A terminal device includes a processor, a memory, and a computer program stored in the memory; when the processor executes the computer program, the graph semi-supervised learning EEG emotion recognition method based on metric learning is implemented.
[0027] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the graph semi-supervised learning EEG emotion recognition method based on metric learning is implemented.
[0028] Compared with the prior art, the present invention has the following technical features:
[0029] The present invention can learn a similarity graph representing the data clustering effect through the graph learning method; by using the metric learning method, a function fitting the data similarity can be learned. When new node samples are added, the similarity graph can be expanded and classified without retraining. The present invention can be applied to the classification of new emotion EEG samples, solving the time overhead caused by retraining. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 is a schematic framework diagram of the method of the present invention;
[0031] Figure 2 is a schematic diagram of the semi-supervised EEG emotion recognition model in the present invention;
[0032] Figure 3 is the metric learning layer of the electroencephalogram (EEG) emotion recognition model;
[0033] Figure 4 is the graph neural network of the EEG emotion recognition model;
[0034] Figure 5 is the training process of the EEG emotion recognition model;
[0035] Figure 6 is the training loss curve in an embodiment of the present invention;
[0036] Figure 7 are the similarity graphs and test results at different stages of the training process in an embodiment of the present invention. Specific Embodiments
[0037] See the appendix Figure 1 The present invention provides an EEG emotion recognition method based on metric learning and graph semi-supervised learning. This method uses a first metric function to fit the relationship between samples, thereby constructing a similarity graph reflecting the clustering effect. When new samples are added, only the learned second metric function is needed to calculate the relationship between the new sample and the existing samples, so as to expand the original similarity graph, thus avoiding the time cost brought by retraining. The present invention specifically includes the following steps:
[0038] Step 1: Obtain an EEG dataset; divide the EEG dataset into a training set and an extended set, where some samples in the training set are labeled; use a first metric function to fit the relationship between the samples in the training set, thereby constructing an initial similarity graph.
[0039] Among them, the EEG dataset is a dataset composed of EEG data collected from different subjects under different emotion types; each EEG data contains time-domain information and spatial information; after the EEG data is normalized, it is divided into time windows per second, and each divided EEG data is used as a sample.
[0040] Divide the EEG dataset into a training set and an extended set, where some samples in the training set have labels, and the labels are the emotion types corresponding to the samples; the emotion categories corresponding to the samples in the extended set are unknown; in this embodiment, the proportion of labeled samples is 5%.
[0041] Construct an initial similarity graph using the samples in the training sample set and the first metric function, specifically as follows:
[0042] (1) Extract the differential entropy features of each sample. The differential entropy features are the number of channels of the EEG data × the number of frequency bands; expand each differential entropy feature into a one-dimensional feature vector and use it as a node in the initial similarity graph.
[0043] (2) Calculate the similarity between different nodes using the first metric function, and use the similarity as the edge between nodes to obtain the initial similarity graph.
[0044] Among them, the first metric function uses a cosine similarity function without parameters, which is expressed as follows:
[0045] S i,j =cos(v i ,v j );
[0046] Among them, S i,j represents the similarity between node v i and node v j .
[0047] Step 2, construct a semi-supervised EEG emotion recognition model; the EEG emotion recognition model includes a metric learning layer and a graph neural network; use the initial similarity graph as the input of the EEG emotion recognition model, use the metric learning layer to learn the similarity matrix based on the learnable second metric function to update the similarity graph, and jointly input the updated similarity graph and the regularized sample features into the graph neural network to obtain the classification results of the samples.
[0048] As Figure 3 shown, using the initial similarity graph constructed by the training set as the input, the metric learning layer first regularizes all nodes in the initial similarity graph using the Dropout layer to obtain sample features to prevent overfitting; then multiply the sample features by a layer of learnable weight matrix (projection matrix), and use the second metric function to calculate the similarity between nodes, so as to output the learned similarity matrix; use the similarity matrix as the new edge between nodes to update the initial similarity graph to obtain the similarity graph after metric learning; input the similarity graph into the graph neural network to obtain the classification results of each sample.
[0049] The second metric function uses a cosine similarity function with learnable parameters, which is expressed as:
[0050]
[0051] Among them, represents the similarity between nodes and , and w is a learnable parameter.
[0052] Among them, the graph neural network includes two layers of graph convolutional layers. The dimension of the input feature (similar graph) is 310. After passing through the first layer of graph convolutional layer, the dimension of the embedded feature obtained is 64. After the embedded feature is activated by the relu function, Dropout regularization is performed, and then it is input into the second layer of graph convolutional layer. The dimension of the second layer of graph convolutional layer is 4 (number of categories). After the final embedded feature is processed by the Softmax function, the classification result is obtained.
[0053] Step 3: Construct a total loss function for training the EEG emotion recognition model. The total loss function includes the classification loss function of labeled samples and a regularization term for controlling the smoothness of the second metric function; train the EEG emotion recognition model to obtain a trained EEG emotion recognition model.
[0054] Among them, the total loss function is expressed as:
[0055] L = L P + L G ;
[0056] Among them, L P represents the classification loss function, which is the cross-entropy loss function calculated using the classification result of the graph neural network; L G is the regularization term, which is obtained by calculating the second similarity function in the metric learning layer and is expressed as follows:
[0057]
[0058] Use the backpropagation algorithm to train the EEG emotion recognition model through the training set, and obtain a trained EEG emotion recognition model after ten-fold cross-validation.
[0059] Step 4: Input the extended set into the trained EEG emotion recognition model. Construct new nodes with the samples in the extended set, add the new nodes to the similar graph to expand the nodes of the similar graph, and use the trained second metric function in the metric learning module to calculate the similarity between the new nodes and the existing nodes in the similar graph to expand the edges of the similar graph. Finally, use the trained graph neural network to perform label propagation on the new nodes to obtain the corresponding classification result.
[0060] In this solution, the unlabeled extended set is used to expand the similar graph, specifically as follows:
[0061] First, extract the differential entropy features of each sample in the expansion set according to the method mentioned in step 1. After expanding the differential entropy features into a one-dimensional feature vector, add it as a new node to the similarity graph constructed using the training set, thereby expanding the nodes of the similarity graph. Secondly, since the second metric function contains learnable parameters, the second metric function is also trained during the training of the EEG emotion recognition model. Then, use the trained second metric function to calculate the similarity between the new node and the existing nodes in the similarity graph as the edge between the nodes, thereby expanding the edges of the similarity graph.
[0062] Embodiment:
[0063] In the embodiment of the present invention, the SEEDIV dataset is used to divide the training set and the expansion set. For this dataset, 4-minute EEG data of 62 leads are collected for different subjects and emotion types respectively, and the sampling rate is 1000 Hz. Use the differential entropy features of the samples in the dataset as the sample features, with a size of 62×5 (number of channels × number of frequency bands); then expand it into a one-dimensional vector (310×1).
[0064] This dataset is four-class EEG data (neutral, sad, fear, happy). Each subject's experiment has three sessions, and each session has 24 trials. There are 6 trials for each emotion type. Select the EEG data of the first four trials as the training set, and the EEG data of the last two trials as the expansion set. Train the EEG emotion recognition model based on metric learning, select the optimal model, and use the samples in the expansion set as new node samples to expand the similarity graph, and finally perform label propagation.
[0065] As Figure 6 shown in the loss function of the training, the loss value tends to be stable after 140 epochs, and the training time cost is low. As Figure 7 shown in the similarity graph of different epoch stages, it can be seen that after metric learning, compared with the initial similarity graph, a sparse low-rank graph that can better reflect the clustering effect can be obtained. Figure 7 Shown in the four-class confusion matrix, it can be seen that the classification accuracy of the test is 81.7%.
[0066] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A graph semi-supervised learning EEG emotion recognition method based on metric learning, characterized in that: include: Obtain EEG dataset; The EEG dataset is divided into a training set and an expansion set, wherein some samples in the training set are labeled; The first metric function is used to fit the relationship between samples in the training set, thereby constructing an initial similarity graph; Constructing a semi-supervised EEG emotion recognition model; the EEG emotion recognition model includes a metric learning layer and a graph neural network; The initial similarity graph is used as the input of the EEG emotion recognition model. The metric learning layer is used to learn the similarity matrix based on the learnable second metric function to update the similarity graph. The updated similarity graph and the regularized sample features are input into the graph neural network to obtain the classification result of the sample. Constructing a total loss function for EEG emotion recognition model training, the total loss function includes a classification loss function of labeled samples and a regularization term for controlling the smoothness of the second metric function; training the EEG emotion recognition model to obtain a trained EEG emotion recognition model; The extended set is input into the trained EEG emotion recognition model, new nodes are constructed with samples in the extended set, the new nodes are added to the similarity graph to expand the nodes of the similarity graph, and the second metric function trained in the metric learning module is used to calculate the similarity between the new nodes and the existing nodes in the similarity graph to expand the edges of the similarity graph. Finally, the trained graph neural network is used to perform label propagation on the new nodes to obtain the corresponding classification results.
2. The graph semi-supervised learning EEG emotion recognition method based on metric learning according to claim 1 is characterized in that: The EEG data set is a data set composed of EEG data collected from different subjects under different emotion types; each EEG data contains time domain information and spatial information; after the EEG data is normalized, the EEG data is divided according to the time window per second, and each EEG data after the division is used as a sample; the EEG data set is divided into a training set and an extended set, wherein some samples in the training set have labels, and the labels are the emotion types corresponding to the samples.
3. The graph semi-supervised learning EEG emotion recognition method based on metric learning according to claim 1 is characterized in that: The method of fitting the relationship between samples in the training set using the first metric function to construct an initial similarity graph includes: Extract the differential entropy feature of each sample, which is the number of channels × the number of frequency bands of the EEG data; expand each differential entropy feature into a one-dimensional feature vector and use it as a node in the initial similarity graph; The first metric function is used to calculate the similarities between different nodes, and the similarities are used as edges between the nodes, thereby obtaining an initial similarity graph.
4. The graph semi-supervised learning EEG emotion recognition method based on metric learning according to claim 1 is characterized in that: The metric learning layer first uses the Dropout layer to regularize all nodes in the initial similarity graph to obtain sample features to prevent overfitting; then the sample features are multiplied by a layer of learnable weight matrix, and the similarity between nodes is calculated using the second metric function, thereby outputting the learned similarity matrix; The initial similarity graph is updated with the similarity matrix as the new edge between nodes to obtain the similarity graph after metric learning; the similarity graph is input into the graph neural network to obtain the classification results of each sample.
5. The graph semi-supervised learning EEG emotion recognition method based on metric learning according to claim 1 is characterized in that: The graph neural network includes two graph convolution layers. The input features are embedded features obtained after passing through the first graph convolution layer. The embedded features are activated by the relu function and then Dropout regularized. Then, they are input to the second graph convolution layer. The final embedded features are output and processed by the Softmax function to obtain the classification result.
6. The graph semi-supervised learning EEG emotion recognition method based on metric learning according to claim 1 is characterized in that: The first metric function uses a cosine similarity function without parameters; the second metric function uses a cosine similarity function with learnable parameters, which is expressed as: in, Representation Node and The similarity between them, w is a learnable parameter.
7. The graph semi-supervised learning EEG emotion recognition method based on metric learning according to claim 1 is characterized in that: The total loss function is expressed as: L=L P +L G ; Among them, L P represents the classification loss function, which is the cross entropy loss function calculated using the classification results of the graph neural network; L G is a regularization term, which is calculated using the second similarity function in the metric learning layer and is expressed as follows: The EEG emotion recognition model was trained using the training set using the back propagation algorithm, and the trained EEG emotion recognition model was obtained after ten-fold cross validation.
8. The graph semi-supervised learning EEG emotion recognition method based on metric learning according to claim 1, characterized in that: Firstly, the differential entropy features of the samples in the extended set are extracted, and the differential entropy features are expanded into a one-dimensional feature vector and then added as a new node to the similarity graph constructed using the training set, thereby expanding the nodes of the similarity graph; secondly, the trained second metric function is used to calculate the similarity between the new node and the existing nodes in the similarity graph as the edge between the nodes, thereby expanding the edge of the similarity graph.
9. A terminal device comprising a processor, a memory and a computer program stored in the memory; characterized in that: When the processor executes the computer program, it implements the graph semi-supervised learning EEG emotion recognition method based on metric learning according to any one of claims 1-8.
10. A computer-readable storage medium, wherein a computer program is stored in the medium; characterized in that: When the computer program is executed by a processor, the graph semi-supervised learning method for EEG emotion recognition based on metric learning according to any one of claims 1 to 8 is implemented.