Cross-subject emotion recognition method and system based on removal of electroencephalogram signal identity information

Abstract

This invention discloses a cross-subject emotion recognition method and system based on removing identity information from EEG signals, comprising: S1, acquiring EEG signals from multiple subjects and performing data preprocessing; S2, constructing an EEG signal decoupling module to remove identity-related information from the EEG signals, wherein the EEG signals consist of task components, identity components, and noise; S3, dividing the EEG data into training, validation, and test sets, training a model using the training set data, and fine-tuning the parameters of the trained model using the validation set through a loss function; S4, constructing a feature clustering module to cluster features in the source and target domains; S5, iteratively training until the total loss function value converges, obtaining a cross-subject emotion recognition model with high accuracy for the target subjects. This invention improves the accuracy of cross-subject emotion recognition.

Description

Technical Field

[0001] This invention relates to electroencephalogram (EEG) signal recognition technology, and more particularly to a method and system for cross-subject emotion recognition based on removing identity information from EEG signals. Background Technology

[0002] Brain-computer interface (BCI) is a technology that enables direct communication between the brain and external devices by decoding brain signals. It is widely used in medical rehabilitation, brain-computer interaction, and disease research. Electroencephalography (EEG) has become an important signal source in BCI research due to its non-invasive, real-time, and low-cost nature.

[0003] EEG-based emotion recognition technology has shown great potential in various application scenarios, including mental health monitoring, user experience optimization, and adaptive systems. However, significant individual differences in EEG signals lead to poor performance in cross-subject usage scenarios, limiting the effectiveness of emotion recognition in practical applications.

[0004] There are many emotion recognition methods based on electroencephalogram (EEG) signals. Traditional methods such as support vector machines, k-nearest neighbors, and random forests are used to classify emotional features in EEG. With the continuous development of deep learning technology, neural network-based emotion recognition methods have gradually become the mainstream in the field of emotion analysis, especially the application of convolutional neural networks (CNNs) in EEG signals.

[0005] Convolutional Neural Networks (CNNs) possess powerful feature extraction capabilities, enabling them to learn temporal and frequency features from EEG signals. Through operations such as convolutional and pooling layers, CNNs can effectively extract useful information from complex EEG data, providing accurate feature representations for sentiment classification. Meanwhile, Recurrent Neural Networks (RNNs), especially variants such as Long Short-Term Memory (LSTM) networks, also have significant advantages in processing the time-series characteristics of EEG signals. RNNs can capture the temporal dependencies and dynamic changes of EEG signals, making them particularly suitable for modeling signal sequences, thereby improving the accuracy of sentiment recognition.

[0006] While these methods can improve recognition accuracy to some extent, they still fail to completely eliminate the impact of individual differences on emotion recognition, especially in cross-subject emotion recognition. To improve cross-subject emotion recognition performance, transfer learning, as an effective solution, has begun to be widely used in EEG emotion recognition. Transfer learning attempts to reduce the differences in EEG signals between different subjects by borrowing knowledge from other subjects. Although transfer learning has shown potential in cross-subject emotion recognition, its cross-domain adaptability remains limited. Summary of the Invention

[0007] The purpose of this invention is to provide a cross-subject emotion recognition method and system based on removing identity information from EEG signals. By decoupling EEG signals, identity information is removed from the EEG data, while emotional information is retained, thereby improving the accuracy of cross-subject emotion recognition.

[0008] To this end, the present invention provides a cross-subject emotion recognition method based on removing identity information from EEG signals, comprising: S1, acquiring EEG signals from multiple subjects and performing data preprocessing; S2, constructing an EEG signal decoupling module to remove identity-related information from the EEG signals, wherein the EEG signals consist of task components, identity components, and noise; S3, dividing the EEG data into training sets, validation sets, and test sets, training a model using the training set data, and fine-tuning the parameters of the trained model using the validation set through a loss function; S4, constructing a feature clustering module to cluster features in the source domain and target domain; S5, converging the total loss function value through iterative training to obtain a cross-subject emotion recognition model with high accuracy for the target subjects, wherein the total loss function value consists of emotion classification loss, identity information removal loss, and feature clustering loss.

[0009] According to another aspect of the present invention, a cross-subject emotion recognition system based on removing identity information from EEG signals is provided, comprising: a data acquisition and preprocessing module for preprocessing EEG signal data from multiple subjects; an EEG signal decoupling module for decoupling the EEG signal data into a task component related to the emotion recognition task and an identity component containing user identity information, while removing the identity component through adversarial training and feature subtraction; an emotion feature clustering module for identifying and aggregating similar emotion features, while separating features of different emotion categories; and an iterative training and output module for converging the total loss function value of the loss function through iterative training to obtain a cross-subject emotion recognition model with high accuracy for the target subjects, wherein the total loss function value consists of emotion classification loss, identity information removal loss, and feature clustering loss.

[0010] Compared with the prior art, the present invention has the following technical advantages / effects:

[0011] (1) By constructing an EEG signal decoupling module to remove identity-related information from the EEG signal, interference caused by individual differences in cross-subject emotion recognition was successfully reduced. By effectively decoupling task-related features and identity-related features, the model can focus on features related to emotional state, thereby significantly improving the accuracy of emotion recognition. In particular, the model's emotion recognition performance was significantly improved when faced with data from different subjects.

[0012] (2) Through multi-source learning and cross-subject training, the method of the present invention enhances the generalization ability of the emotion recognition model. By using leave-one-out cross-validation, the model can be trained and evaluated on data from each target subject, ensuring that the model can effectively adapt to data from different subjects, thereby improving the model's performance on unseen data and enhancing the model's cross-subject adaptability.

[0013] The emotion recognition method of this invention can be widely applied in fields such as brain-computer interfaces (BCI), human-computer interaction, mental health monitoring, and virtual reality. By accurately identifying the emotional states of different subjects, this invention provides effective technical support for intelligent emotion computing systems, promoting the development and popularization of emotion recognition technology in practical applications.

[0014] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0016] Figure 1 This is a flowchart illustrating the cross-subject emotion recognition method based on removing identity information from EEG signals according to the present invention.

[0017] Figure 2 This is a schematic diagram of the overall model of the cross-subject emotion recognition method based on removing identity information from EEG signals;

[0018] Figure 3 This is a structural block diagram of the cross-subject emotion recognition system based on removing identity information from EEG signals. Detailed Implementation

[0019] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] Combined with reference Figure 1 and Figure 2 The cross-subject emotion recognition method in this embodiment includes the following steps S1 to S5.

[0021] S1. Acquire EEG signals from multiple subjects and perform data preprocessing.

[0022] EEG signals are susceptible to the influence of a person's psychological state and device noise, leading to inconsistent data distribution between the training and test sets. In this embodiment, the emotion recognition EEG signal classification task is divided into four categories: positive, negative, neutral, and fear, each with a specific label. For a given subject, the current subject's data is used as the test set, while data from other subjects is used as the training set.

[0023] The preprocessing process includes:

[0024] The original EEG signal was downsampled to 200Hz, and a bandpass filter was used to perform bandpass filtering on the EEG signal in the range of [1, 75]Hz to obtain the EEG signal in the frequency band related to emotion recognition. 1–4Hz, 4–8Hz 8–14Hz, 14–31Hz, The signal was divided into non-overlapping 1-second segments (31–50 Hz) to remove power frequency interference and artifacts and reduce computational complexity. Differential entropy (DE) features were extracted from each segment. The EEG signal data were divided into three sessions, each corresponding to a complete experiment. Euclidean space alignment was applied to the data from different sessions, the covariance matrix of each session was calculated, and the data was standardized using the Z-score standardization method.

[0025] S2. Construct an EEG signal decoupling module to remove identity-related information from EEG signals.

[0026] Electroencephalogram (EEG) signals are extremely complex, containing a wealth of physiological information. Traditional emotion recognition tasks consider EEG signals to be simply divided into two parts: emotional components and noise, as shown in the formula... As shown.

[0027] This invention's innovative research suggests that electroencephalogram (EEG) signals should be divided into three parts: task component, identity component, and noise, as shown in the formula. As shown. The task component is a stable component highly correlated with emotion; while the identity component is an unstable component highly correlated with individual differences among subjects. The noise within... After data preprocessing, it can be considered that it has been removed.

[0028] The role of the EEG signal decoupling module is to remove the identity component from the EEG signal and retain the task component related to emotion. Figure 2 This is an overall model diagram of the present invention.

[0029] The S201 and EEG signal decoupling modules first use a shared feature extractor to extract shared features, such as... Figure 2As shown in the green box, the shared feature extractor is a multilayer perceptron (MLP) structure that progressively extracts task-relevant features from EEG signals through multiple fully connected layers. The network consists of four fully connected layers, each followed by a ReLU activation function, batch normalization, and dropout operation to improve the model's non-linear expressiveness and prevent overfitting. Finally, the network maps the input EEG signal to a 64-dimensional feature space, which is used for subsequent sentiment classification tasks.

[0030] S202. To remove identity information, a multilayer perceptron-structured identity discriminator is set after the shared feature extractor. The discriminator's role is to identify and predict the subject's identity from the shared features and learn identity features. This discriminator supervises the removal of identity information during training by determining whether the features contain identity information. Through adversarial training, a gradient inversion layer is introduced. Its role is to invert the gradient calculated by the identity discriminator and pass the inverted gradient back to the shared feature extractor. Specifically, when the identity discriminator learns identity features, the gradient inversion layer inverts the gradient, so that the shared feature extractor does not depend on identity features when updating, avoiding overfitting to identity information.

[0031] S203. Feature subtraction is a crucial step in removing identity information. In this process, the model removes identity information by subtracting identity-related features extracted by the identity discriminator from shared features. Its core is to solve the problem of task features being destroyed by direct subtraction due to the coupling of task features and identity features through a combination strategy of 'decoupling regularization processing + feature subtraction'. Directly adopting... When performing feature subtraction, due to shared features... Medium task characteristics With identity characteristics Strong coupling and correlation exist, with some features simultaneously containing both emotional and identity information. This can easily disrupt the task's feature structure, thereby affecting the accuracy of emotion recognition. The decoupling regularization term introduced in this invention addresses this issue.

[0032] This precisely addresses this pain point, among which and These are the task features and identity features processed by linear dimensionality reduction. Used to calculate the mean of the eigenvectors to reflect the distribution center. Used to calculate variance to reflect the degree of dispersion of the distribution. The magnitude of the value directly corresponds to the coupling strength between the two types of features; the larger the value, the higher the feature overlap.

[0033] During model training, by minimizing able to force and The distribution centers are separated and the degree of dispersion is independent, ultimately achieving effective decoupling of the two types of features. When the minimum value is taken, the two types of features have almost no overlap. and After decoupling is completed, then use the formula By performing feature subtraction, only purely identity-related features are removed without compromising the structural integrity of the task features. This effectively removes interference from individual differences while preserving the core information required for emotion recognition to the greatest extent possible, thus ensuring the accuracy of subsequent cross-subject emotion recognition.

[0034] S3. Divide the EEG data into a training set, a validation set, and a test set. Use the training set data to train the model. Use the validation set to fine-tune the parameters of the trained model using a loss function.

[0035] In this invention, to effectively train the cross-subject emotion recognition model, we divide the electroencephalogram (EEG) data into training, validation, and test sets, and then train and fine-tune the model based on these sets. The specific implementation steps are as follows:

[0036] S301. To evaluate the model's emotion recognition ability on different subjects, this invention employs leave-one-out cross-validation. In each training round, data from all subjects except the target subject are used as the training set; a subset of data is randomly selected from the training set as the validation set to adjust model hyperparameters and prevent overfitting; and data from one target subject is selected each time as the test set to evaluate the model's performance. This ensures that data from each subject participates in the testing, and that the model can be evaluated on different target subjects.

[0037] S302. After data partitioning, the sentiment recognition model is trained using the training set data. First, task-related features are extracted from the EEG signals in the training set using an EEG signal decoupling module. Then, the extracted task-related features are input into the sentiment classifier to predict the sentiment category. Finally, the cross-entropy loss function is used to calculate the difference between the model output and the true label, thereby optimizing the model parameters.

[0038] S303. During training, the AdamW optimizer is used to update the model parameters, and a mini-batch gradient descent method is employed to improve training efficiency and stability. After model training is complete, the validation set is used to fine-tune the trained model to ensure its generalization ability.

[0039] During fine-tuning, this invention also employs an early stopping strategy. Training is stopped when the accuracy on the validation set no longer improves, thus avoiding overfitting. The core of the fine-tuning process lies in calculating the loss function using the validation set data and adjusting the model's hyperparameters based on the loss value to optimize model performance.

[0040] S4. Construct a feature clustering module to cluster features of the source domain and the target domain.

[0041] In this invention, emotion feature clustering is a key step in the cross-subject emotion recognition model. The main function of this module is to cluster EEG signal features from the source and target domains, optimize feature representation, and enhance the discriminative power between emotion categories, thereby improving the accuracy and generalization ability of emotion recognition.

[0042] S401. Before implementing feature clustering, the EEG signal needs to be preprocessed and its features extracted. This step involves extracting task features through the EEG signal decoupling module and removing identity information.

[0043] S402. Cluster the sentiment features extracted from the source domain (training data) and the target domain (test data), grouping samples of the same sentiment category together while separating samples of different categories to ensure a high degree of distinction between sentiment categories.

[0044] S403. To optimize the clustering effect, this invention designs a clustering loss function, which consists of two parts: minimizing intra-class distance: making features of the same sentiment category as close as possible in the feature space, reducing intra-class differences. Maximizing inter-class distance: making features of different sentiment categories as far apart as possible in the feature space, increasing inter-class differences. The optimization objective of the clustering loss function is to improve the discriminative ability of sentiment features by minimizing intra-class distance and maximizing inter-class distance. The form of the clustering loss function is:

[0045]

[0046] in, and These are the predicted label distribution and auxiliary label distribution of the samples, respectively. Denotes KL divergence, The weight of samples within each category is defined by C, where C is the number of sentiment categories. The calculation result of this clustering loss function will serve as a key component of the overall loss function, passing it to subsequent iterative training processes to achieve synergistic advancement between "feature clustering optimization" and the "sentiment recognition task".

[0047] S5. Through iterative training, the total loss function value converges, resulting in a cross-subject emotion recognition model with high accuracy for the target subjects.

[0048] To ensure that the cross-subject emotion recognition model in this invention can accurately and stably identify the emotional state of the target subjects, an iterative training strategy is adopted. Through multiple rounds of training, the model's loss function is optimized, causing the total loss function value to gradually converge, thereby obtaining a model with high emotion recognition accuracy on the target subjects.

[0049] S501. The core objective of iterative training is to gradually optimize the model's loss function, ultimately stabilizing and optimizing the model's performance on the training, validation, and test sets. After each training round, the model's loss function is calculated, and backpropagation is used to update the model parameters. Through multiple rounds of iterative training, the loss function value gradually decreases, and the model's accuracy continuously improves until convergence.

[0050] S502. The total loss function of this invention comprises several components, primarily used to optimize the sentiment classification task and remove interference from identity information. The total loss function consists of sentiment classification loss, identity information removal loss, and feature clustering loss. Sentiment classification loss function: This is used to measure the model's performance on sentiment classification tasks, and typically uses the cross-entropy loss function to calculate the difference between the predicted and true labels. These are the true labels of the samples. This represents the model's predicted probability for the sample, where C is the number of sentiment categories. Identity removal loss function: The accuracy of the identity discriminator in predicting identity labels is measured, among which, For the number of subjects, Representing the The real identity label of each sample This represents the predicted identity label output by the identity discriminator. The gradient is backpropagated to the shared feature extractor, thus eliminating identity information. The total loss function is calculated as follows:

[0051]

[0052] in These are the weighting coefficients.

[0053] S503. To accelerate the training process and ensure that the model can finely adjust parameters in later training stages, this invention uses a dynamic learning rate adjustment strategy. Specifically, the learning rate gradually decreases with each training round, allowing the model to converge more stably in later stages. After several rounds of iterative training, the model will gradually converge, the loss function value will stabilize, and the sentiment recognition accuracy will reach its optimal level. The convergence of the model is determined by monitoring the loss function values ​​on the training and validation sets. After training is complete, the model is evaluated using a test set, and the final sentiment recognition accuracy is calculated.

[0054] Using the method described in this invention, experiments were conducted on two public datasets, SEED and SEED-IV. To quantitatively evaluate the emotion recognition results, average accuracy was used as the evaluation metric.

[0055] Table 1

[0056] Several experiments were conducted on two different datasets using the method described in this invention, and the results were compared with state-of-the-art methods. To ensure that these models have a similar experimental paradigm, this invention fine-tuned them during testing and conducted experiments on open-source code to obtain the results of the aforementioned method.

[0057] Table 1 shows the classification performance of each subject on the SEED and SEED-IV datasets, as well as the overall average results. For the SEED dataset, the model of this invention performs exceptionally well, achieving an average accuracy of 90.27%, the highest accuracy, surpassing the second-best method, CL-CS, with an average accuracy of 88.30%. On the SEED-IV dataset, the method of this invention again maintains its leading position, achieving an average accuracy of 76.26%, also the highest accuracy, surpassing the second-best method, ST-DADGAT, with an average accuracy of 74.97%. In both datasets, the method of this invention demonstrates consistent superiority, proving its effectiveness over previous methods.

[0058] See Figure 3 The present invention also provides a cross-subject emotion recognition system based on removing identity information from EEG signals, comprising: a data acquisition and preprocessing module, an EEG signal decoupling module, an emotion feature clustering module, and an iterative training and output module.

[0059] The data acquisition and preprocessing module is used to preprocess the EEG signal data of multiple subjects.

[0060] The EEG signal decoupling module decouples EEG signal data into a task component related to the emotion recognition task and an identity component containing user identity information, while removing the identity component through feature subtraction.

[0061] The emotion feature clustering module is used to identify and aggregate similar emotion features, while separating features of different emotion categories, thereby improving the accuracy of emotion recognition and cross-subject adaptability.

[0062] The iterative training and output module is used to converge the total loss function value of the loss function through iterative training, preventing the training loss from decreasing excessively, and obtaining a cross-subject emotion recognition model with high accuracy for the target subjects.

[0063] For those skilled in the art, the various modules of the aforementioned system can be implemented entirely or partially in the form of software, hardware, or a combination thereof. Specifically, these modules can be embedded in or independent of the processor of a computer device in hardware form, or they can be directly stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding functional operations of each module at any time.

[0064] The present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the cross-subject emotion recognition method based on removing identity information from EEG signals described above.

[0065] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for cross-subject emotion recognition based on removing identity information from electroencephalogram (EEG) signals, characterized in that, include: S1. Acquire EEG signals from multiple subjects and perform data preprocessing; S2. Construct an EEG signal decoupling module to remove identity-related information from the EEG signal, which consists of task component, identity component and noise. S3. Divide the EEG data into training set, validation set and test set. Use the training set data to train the model, and use the validation set to fine-tune the parameters of the trained model through the loss function. S4. Construct a feature clustering module to cluster features from the source domain and the target domain; S5. Through iterative training, the total loss function value converges, resulting in a cross-subject emotion recognition model with high accuracy for the target subjects. The total loss function value consists of emotion classification loss, identity information removal loss, and feature clustering loss.

2. The cross-subject emotion recognition method based on removing identity information from EEG signals according to claim 1, characterized in that, In step S1, the data preprocessing process includes: The original EEG signal was downsampled to 200Hz, and a bandpass filter was used to perform bandpass filtering on the EEG signal in the range of [1,75]Hz to obtain the EEG signal in the frequency band related to emotion recognition. The signal was segmented into non-overlapping 1-second time periods. Differential entropy features were extracted from each time period. Euclidean space alignment was applied to the multi-session data, the covariance matrix of each session was calculated, and the data was standardized using the Z-score standardization method.

3. The cross-subject emotion recognition method based on removing identity information from EEG signals according to claim 1, characterized in that, Step S2 includes: S201. Use a shared feature extractor to extract shared features from EEG signals in the source and target domains; S202. After the shared feature extractor, an identity discriminator is introduced to predict the identity information in the features. If the features contain identity information, the identity discriminator can accurately classify them. S203. Use a gradient inversion layer for adversarial training. Invert the gradient calculated by the identity discriminator during training and pass the inverted gradient back to the shared feature extractor. S204. Remove identity information using feature subtraction while retaining task-related information required for emotion recognition. The feature subtraction formula is as follows: , Task characteristics and identity characteristics The decoupling regularization term, F, represents the shared features.

4. The cross-subject emotion recognition method based on removing identity information from EEG signals according to claim 1, characterized in that, Step S3 includes: S301. Select one set of data from multiple subjects as the test set, select two sets of data from the remaining subjects as the validation set, and use the remaining data as the training set. S302. In order to improve the generalization ability of the model, these multiple subjects will be used as the test set in turn. S303. Fine-tune the trained model using validation set data. After each round of training, calculate the model's loss function using validation set data and adjust the model's hyperparameters based on the loss value to optimize model performance.

5. The cross-subject emotion recognition method based on removing identity information from EEG signals according to claim 4, characterized in that, During fine-tuning, an early stopping strategy is applied to prevent overfitting based on performance on the validation set.

6. The cross-subject emotion recognition method based on removing identity information from EEG signals according to claim 1, characterized in that, Step S4 includes: S401. Extract features from the EEG signals in the source and target domains, and cluster the features in the source and target domains. S402. Cluster the source domain features to group features of similar emotion categories together, while ensuring that features of different emotion categories maintain a large distance. S403. Cluster the features of the target domain to ensure that the sentiment category features in the target domain are consistent with the features in the source domain.

7. The cross-subject emotion recognition method based on removing identity information from EEG signals according to claim 1, characterized in that, Step S5 includes: through multiple rounds of iterative training, gradually reducing the loss function value so that the model can achieve a high accuracy rate in emotion recognition on the target subject data.

8. The cross-subject emotion recognition method based on removing identity information from EEG signals according to claim 1, characterized in that, The shared feature extractor is a multilayer perceptron, consisting of four fully connected layers. Each layer is followed by ReLU activation, batch normalization, and Dropout processing to map the input EEG signal to a 64-dimensional feature space.

9. A cross-subject emotion recognition system based on removing identity information from EEG signals, characterized in that, include: The data acquisition and preprocessing module is used to preprocess the EEG signal data of multiple subjects; The EEG signal decoupling module is used to decouple EEG signal data into a task component related to the emotion recognition task and an identity component containing user identity information, while removing the identity component through adversarial training and feature subtraction. The sentiment feature clustering module is used to identify and aggregate similar sentiment features, while separating features from different sentiment categories; The iterative training and output module is used to converge the total loss function value of the loss function through iterative training, so as to obtain a cross-subject emotion recognition model with high accuracy for the target subjects. The total loss function value consists of emotion classification loss, identity information removal loss, and feature clustering loss.

10. The cross-subject emotion recognition system based on removing identity information from EEG signals according to claim 9, characterized in that, The data acquisition and preprocessing module includes: The Euclidean alignment unit is used to align data from multiple sessions to eliminate cross-session differences between different subjects. This includes: first, calculating the covariance matrix of the input data to obtain a matrix representing the relationship between data features; then, mapping the data of each session onto this mean-covariance matrix so that the data from different sessions have the same distribution; and finally, standardizing the aligned data using Z-score to eliminate scale differences between different sessions.

Images