A cross-domain electroencephalogram emotion recognition method based on two-stage domain alignment and two-classifier collaborative confrontation
By employing a two-level domain alignment and dual-classifier collaborative adversarial approach, the problem of insufficient model generalization ability in cross-domain EEG emotion recognition is solved, achieving high accuracy and stable emotion recognition, applicable to various practical application scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU DIANZI UNIV
- Filing Date
- 2026-03-13
- Publication Date
- 2026-06-26
AI Technical Summary
Existing cross-domain EEG emotion recognition technologies suffer from insufficient model generalization ability when faced with differences in subjects, devices, and scenarios. Furthermore, traditional methods struggle to balance classification accuracy with domain adaptability, resulting in insufficient recognition accuracy and stability.
We employ a two-level domain alignment and dual-classifier collaborative adversarial approach. By constructing a dual-classifier module and a multi-stage training strategy, we design subdomain alignment loss and dual-classifier consistency-deterministic constraint loss to achieve deep adaptation between the source and target domains.
It improves the accuracy and stability of cross-domain emotion recognition, enhances the model's generalization ability, is applicable to EEG data from different acquisition devices and subject groups, and is suitable for a variety of practical application scenarios.
Smart Images
Figure CN121817895B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of EEG signal processing and emotion recognition technology, specifically to a cross-domain EEG emotion recognition method based on bi-level domain alignment and dual-classifier collaborative adversarial approach. Background Technology
[0002] Emotion recognition, as a core supporting technology in affective computing and natural human-computer interaction, has irreplaceable application value in scenarios such as real-time monitoring of mental health, emotional interaction of intelligent devices, and auxiliary assessment of clinical emotional disorders. Compared with facial expressions and voice, which are easily faked emotional carriers, electroencephalogram (EEG) signals can directly reflect brain neural activity and have unique advantages such as high temporal resolution, strong objectivity, and high reliability, making them an important data source for emotion recognition. The core logic of EEG-based emotion recognition technology is to extract time-domain, frequency-domain, or time-frequency-domain features of the signal (such as differential entropy, power spectral density, etc.) and combine them with machine learning or deep learning models to achieve accurate classification of emotional states. This is currently a research hotspot in the interdisciplinary field of artificial intelligence and neuroscience.
[0003] However, in practical applications, EEG signals are easily affected by various factors, leading to significant differences in the data distribution between the source and target domains. On the one hand, different subjects have inherent differences in physiological structure, neural response patterns, and emotional expression habits, resulting in significant differences in the distribution of EEG signal features corresponding to the same emotion. On the other hand, differences in the model of the acquisition device, the experimental setting, and the data acquisition session can cause a systematic shift in the distribution of EEG signals, thus causing a significant deviation between the feature distributions of the source and target domains. This cross-domain distribution difference directly weakens the model's generalization ability, ultimately leading to a significant decrease in the accuracy of emotion recognition on target domain data for models that perform well in the source domain.
[0004] To address the aforementioned cross-domain adaptation problem, existing technologies primarily employ global domain alignment methods, aiming to improve model generalization by minimizing the global distribution difference between the source and target domains. However, these methods only focus on global domain differences, neglecting sub-domain differences within the same emotion category, leading to insufficient intra-class feature alignment. Furthermore, traditional single-classifier training models struggle to balance classification accuracy and domain adaptation, easily resulting in overfitting or underfitting. Moreover, the loss function designs of existing cross-domain emotion recognition methods often lack constraints on classifier consistency and determinism, further impacting the stability and accuracy of cross-domain recognition. Summary of the Invention
[0005] The purpose of this invention is to provide a cross-domain EEG emotion recognition method based on dual-level domain alignment and dual-classifier collaborative adversarial approach. By constructing a dual-classifier module, designing a multi-stage training strategy, and innovating sub-domain alignment loss and dual-classifier consistency-deterministic constraint loss, the method achieves deep adaptation between the source domain and the target domain, thereby improving the accuracy and stability of cross-domain emotion recognition and solving the problems mentioned in the background art.
[0006] The technical solution provided by this invention is as follows: A cross-domain EEG emotion recognition method based on two-level domain alignment and dual-classifier collaborative adversarial approach, comprising the following operational steps:
[0007] Step S1: Acquire EEG signals and preprocess them.
[0008] Preferably, the preprocessing includes: using filtering and denoising technology to filter out power frequency interference and high-frequency noise, and using artifact removal algorithms to eliminate physiological artifacts such as electrooculography and electromyography to obtain purified EEG signals; then extracting EEG differential entropy features according to preset frequency bands to obtain a 310-dimensional differential entropy feature vector for each EEG signal sample.
[0009] Step S2: Construct a cross-domain recognition model that includes a feature extractor module and a dual classifier module.
[0010] Preferably, the feature extraction module includes a common feature extractor and M domain-specific feature extractors (M being the number of source domains); the common feature extractor maps 310-dimensional EEG features to 64-dimensional common features, and each domain-specific feature extractor corresponds to a unique source domain, converting the 64-dimensional common features into 32-dimensional domain-specific features; the dual classifier module includes two independent sets of domain-specific classifiers, each set containing M linear classifiers, which respectively map the 32-dimensional domain-specific features of the corresponding source domain to the emotion category probability space.
[0011] Step S3: Use a multi-stage adversarial training strategy to train and optimize the cross-domain recognition model.
[0012] Preferably, the specific implementation includes:
[0013] Step S31: Optimize the parameters of the feature extractor module and the dual classifier module to minimize the source domain cross-entropy loss, the maximum mean difference loss, and the subdomain alignment loss;
[0014] Step S32: Fix the parameters of the feature extractor module, optimize the parameters of the classifier, and minimize the source domain cross-entropy loss and the consistency-determinism constraint loss of the dual classifier;
[0015] Step S33: Fix the parameters of the classifier and optimize only the parameters of the feature extractor module to maximize the consistency-deterministic constraint loss of the two classifiers.
[0016] Step S4: Define the key loss function.
[0017] Preferably, the definition of the key loss function includes:
[0018] Subdomain alignment loss: After the target domain features are processed by the feature extraction module, they are input into the dual classifier module to obtain two sets of logits. Two sets of predicted probabilities and two entropy values are calculated separately. Target domain samples with at least one entropy value less than or equal to a set value are selected as high-confidence pseudo-label samples. The pseudo-label is determined by the category corresponding to the highest probability among the two sets of predicted probabilities. The subdomain alignment loss is then divided according to the true label of the source domain. The source domain feature set is used to calculate the true class center; then, the pseudo-labels are used to divide the class into the first class. The target domain feature set is used, with the predicted probability corresponding to the sample pseudo-label as the confidence weight, to calculate the first... The confidence-weighted class center of the target domain is calculated; the cosine distance loss between the class centers of the source and target domains is calculated to obtain the subdomain alignment loss.
[0019] The dual-classifier consistency-deterministic constraint loss is composed of a weighted sum of a joint consistency term and a local classification deterministic term. The joint consistency term measures the predictive consistency of the two classifiers on high-confidence samples, while the local classification deterministic term measures the deterministic synergistic effect of the two classifiers.
[0020] Step S5: Input the preprocessed target domain EEG data into the trained cross-domain recognition model and output the target domain emotion recognition result.
[0021] Preferably, the trained cross-domain recognition model calculates the weighted fusion weights using the consistency-deterministic constraint loss values of the dual classifiers corresponding to each source domain, and obtains the final emotion recognition result.
[0022] Compared with the prior art, the beneficial effects achieved by the present invention are:
[0023] (1) Strong cross-domain adaptation capability: This invention achieves global distribution alignment between the source domain and the target domain by jointly optimizing the maximum mean difference loss and the subdomain alignment loss, and solves the problem of subdomain differences within the same category, thus greatly improving the generalization capability of the model.
[0024] (2) High recognition accuracy: The present invention adopts a dual classifier adversarial training strategy, combined with consistency-determinism constraint loss, which effectively improves the classifier's discrimination ability and output stability, and reduces classification errors in cross-domain scenarios;
[0025] (3) High practicality: The preprocessing process of this invention is simple and efficient, and the feature extraction and model structure are easy to implement in engineering. It can be adapted to EEG data of different acquisition devices and different subject groups and is suitable for a variety of practical application scenarios. Attached Figure Description
[0026] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0027] Figure 1 This is a schematic diagram of the steps of the cross-domain EEG emotion recognition method based on dual-level domain alignment and dual-classifier collaborative adversarial provided in the embodiments of the present invention;
[0028] Figure 2 This is a schematic diagram of the cross-domain recognition model framework provided in an embodiment of the present invention. Detailed Implementation
[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0030] The embodiments of the present invention are combined with Figures 1-2 As shown, the following technical solution is provided: a cross-domain EEG emotion recognition method based on two-level domain alignment and dual-classifier collaborative adversarial approach, including the following operational steps:
[0031] Step S1: Acquire and preprocess EEG signals;
[0032] In this embodiment, data preparation is carried out by setting up the following experimental environment.
[0033] Hardware environment: Computer equipment with general-purpose CPU and GPU computing capabilities;
[0034] Software environment: Windows operating system, with PyTorch framework and EEG signal processing dependency library;
[0035] Dataset: The publicly available EEG emotion dataset SEED was used, containing EEG data from 15 participants. Each participant participated in 3 experimental sessions. In each session, 15 movie clips were used to induce three emotions: positive, negative, and neutral. 62 channels of EEG signals were collected (sampling rate 1000Hz). The experiment adopted a "leave one participant" strategy: each time, the data of 1 participant was used as the target domain (unlabeled), and the data of the remaining 14 participants were used as the source domain (with real labels). The experiment was repeated 15 times to cover all participants.
[0036] For example, preprocessing includes: using filtering and denoising techniques to remove power frequency interference and high-frequency noise; eliminating physiological artifacts such as electrooculography (EOG) and electromyography (EMG) through artifact removal algorithms to obtain purified EEG signals; and then extracting EEG differential entropy features according to preset frequency bands for the EEG signals. Its differential entropy characteristics The definition is as follows:
[0037]
[0038] in, This is the probability density function of the EEG signal. After bandpass filtering, the EEG signal follows a Gaussian distribution. ,in The mean, Since it is variance, the differential entropy feature can be simplified to:
[0039]
[0040] Based on the above formula, the 310-dimensional differential entropy feature vector of each EEG signal sample is calculated.
[0041] Step S2: Construct a cross-domain recognition model that includes a feature extractor module and a dual classifier module.
[0042] In this embodiment, the feature extraction module includes one common feature extractor and M domain-specific feature extractors (M being the number of source domains). The common feature extractor maps 310-dimensional EEG features to 64-dimensional common features. Each domain-specific feature extractor corresponds to a unique source domain and converts the 64-dimensional common features into 32-dimensional domain-specific features. The dual classifier module includes two independent sets of domain-specific classifiers, each set containing M linear classifiers, which respectively map the 32-dimensional domain-specific features of the corresponding source domain to the emotion category probability space. Specific parameter settings are as follows:
[0043] Common feature extractor: A 3-layer fully connected network is used, with an input dimension of 310 and an output dimension of 64. The activation function is LeakyReLU and the dropout rate is 0.3.
[0044] Domain-specific feature extractors: 14 in total (consistent with the number of source domains), each is a 1-layer fully connected network with an input dimension of 64 and an output dimension of 32, and the activation function is LeakyReLU;
[0045] The dual classifier module is divided into two groups, each containing 14 domain-specific classifiers. Each classifier is a linear layer with an input dimension of 32 and an output dimension of 3, corresponding to the three emotion categories.
[0046] Step S3: Train and optimize the model using a multi-stage adversarial training strategy:
[0047] Step S31: Optimize the parameters of the feature extractor module and the dual classifier module to minimize the source domain cross-entropy loss, the maximum mean difference (MMD) loss, and the subdomain alignment loss.
[0048] Instantaneous, with the following goal:
[0049]
[0050] in, , , For feature extractor module, classifier Classifier Parameters; For cross-entropy loss, , Classifiers Classifier Predicted output for source domain data, The true labels for the source domain data. , For hyperparameters, For the maximum mean difference loss, For the subdomain alignment loss based on feature cosine distance, the experiment fixed the high-confidence pseudo-label selection threshold as "entropy value ≤ 0.6" and the number of emotion categories. The category center calculation uses the source domain true labels and target domain pseudo labels from the SEED dataset.
[0051] Step S32: Fix the parameters of the feature extractor module and optimize the classifier. Classifier The parameters are set to minimize the source domain cross-entropy loss and the bi-classifier consistency-deterministic constraint loss.
[0052] For example, the goal is:
[0053]
[0054] in, , For hyperparameters, , Classifiers Classifier Predicted output for target domain data, The loss is the consistency-deterministic constraint loss for the dual classifier.
[0055] Step S33: Fix the classifier Classifier The parameters are optimized only for the feature extractor module to maximize the bi-classifier consistency-deterministic constraint loss.
[0056] For example, the goal is:
[0057]
[0058] in, , This is a hyperparameter.
[0059] This invention achieves global domain alignment and subdomain alignment to the greatest extent possible, i.e., two-level domain alignment. It utilizes an adversarial mechanism of "minimizing the loss of the dual classifier module and maximizing the loss of the feature extractor module" to repeatedly iterate the above three sub-steps during the training process.
[0060] Step S4: Definition of the key loss function:
[0061] Step S41: Subdomain Alignment Loss Intra-class subdomain adaptation is achieved through category center alignment.
[0062] In this embodiment, the calculation process is as follows: the target domain features are processed by the feature extraction module and then input into the dual classifier module to obtain logits and calculate two sets of predicted probabilities and two entropy values respectively. At least one target domain sample with an entropy value less than or equal to 0.6 is selected as a high-confidence pseudo-label sample, and its pseudo-label is determined according to the category corresponding to the highest probability among the two sets of predicted probabilities.
[0063] Classified by source domain real label Source domain feature set Calculate the true category center :
[0064]
[0065] in, for The number of samples in the sample;
[0066] For example, the first is divided according to pseudo-tags. Class Target Domain Feature Set The predicted probability corresponding to the pseudo-label of the sample is used as the confidence weight. Calculate the first Target Domain Confidence Weighted Class Center :
[0067]
[0068] Calculate the cosine distance loss between the center of the same class in the source domain and the target domain to obtain the subdomain alignment loss:
[0069]
[0070] in For the number of emotion categories, This is the cosine similarity function.
[0071] Step S42: Dual Classifier Consistency - Deterministic Constraint Loss : Composed of "joint consensus items" "and "local classification deterministic items" "Weighted composition, the calculation formula is:"
[0072]
[0073] Among them, joint consensus items The formula used to measure the predictive consistency of two classifiers on high-confidence samples is as follows:
[0074]
[0075] in This represents the total number of samples in this batch. Classifiers Classifier For the sample Predicted as category The probability, For the sample The weights are jointly determined by the maximum predicted probabilities of the two classifiers for that sample, and are calculated using the following formula:
[0076]
[0077] Local classification deterministic terms The formula used to measure the deterministic collaborative effect of two classifiers is as follows:
[0078]
[0079] in, , , Classifiers Classifier The average L2 norm squared of the output probability is calculated using the following formula:
[0080]
[0081]
[0082] Step S5: Cross-domain emotion recognition and prediction.
[0083] In this embodiment, the preprocessed target domain EEG data is input into the trained model, based on the corresponding source domains. The weighted fusion weights are calculated using the following formula:
[0084]
[0085] in, For the first The weights of each source domain dual classifier module are weighted and fused together to obtain the final emotion recognition result.
[0086] In this invention, the average results of 15 experiments on the SEED dataset across subjects were as follows: emotion recognition accuracy: 89.75% ± 8.38%; F1 score: 0.88; Kappa coefficient: 0.82.
[0087] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0088] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A cross-domain EEG emotion recognition method based on two-level domain alignment and dual-classifier collaborative adversarial approach, characterized in that: The following steps are included: Step S1: Acquire and preprocess EEG signals; Step S2: Construct a cross-domain recognition model that includes a feature extractor module and a dual classifier module; Step S3: Train and optimize the cross-domain recognition model using a multi-stage adversarial training strategy; The implementation of step S3 includes: Step S31: Optimize the parameters of the feature extractor module and the dual classifier module to minimize the source domain cross-entropy loss, the maximum mean difference loss, and the subdomain alignment loss; Step S32: Fix the parameters of the feature extractor module, optimize the parameters of the classifier, and minimize the source domain cross-entropy loss and the consistency-determinism constraint loss of the dual classifier; Step S33: Fix the parameters of the classifier and optimize only the parameters of the feature extractor module to maximize the consistency-deterministic constraint loss of the two classifiers; Step S3 also includes: based on global domain alignment and subdomain alignment, using the adversarial mechanism of "minimizing the loss of the dual classifier module and maximizing the loss of the feature extractor module", step S31-step S33 are repeatedly iterated during the training process; Step S4: Define the key loss function; The definition of the key loss function includes: Subdomain alignment loss: Intra-class subdomain adaptation is performed by aligning with the category center; Dual classifier consistency-deterministic constraint loss: composed of a weighted average of the joint consistency term and the local classification deterministic term; Step S5: Input the preprocessed target domain EEG data into the trained cross-domain recognition model and output the target domain emotion recognition result.
2. The cross-domain EEG emotion recognition method based on dual-domain alignment and dual-classifier collaborative adversarial approach according to claim 1, characterized in that: The preprocessing in step S1 includes: Filtering and noise reduction techniques are used to remove power frequency interference and high-frequency noise; Physiological artifacts in electrooculography (EOG) and electromyography (EMG) are eliminated by artifact removal algorithms to obtain purified EEG signals. The differential entropy features of the EEG are extracted according to the preset frequency band, resulting in a 310-dimensional differential entropy feature vector for each EEG signal.
3. The cross-domain EEG emotion recognition method based on dual-domain alignment and dual-classifier collaborative adversarial approach according to claim 2, characterized in that: The feature extractor module in step S2 includes: a common feature extractor and multiple domain-specific feature extractors. The common feature extractor maps 310-dimensional EEG features to 64-dimensional common features. Each domain-specific feature extractor corresponds to a unique source domain and converts the 64-dimensional common features into 32-dimensional domain-specific features.
4. The cross-domain EEG emotion recognition method based on dual-domain alignment and dual-classifier collaborative adversarial approach according to claim 3, characterized in that: In step S2, the dual classifier module contains two independent sets of domain-specific classifiers, each set containing multiple linear classifiers, which respectively map the 32-dimensional domain-specific features of the corresponding source domain to the emotion category probability space.
5. The cross-domain EEG emotion recognition method based on dual-domain alignment and dual-classifier collaborative adversarial approach according to claim 4, characterized in that: The calculation process for subdomain alignment loss includes: After the target domain features are processed by the feature extraction module, they are input into the dual classifier module to obtain two sets of logits. Two sets of predicted probabilities and two entropy values are calculated respectively. Target domain samples with at least one entropy value less than or equal to a set value are selected as high-confidence pseudo-label samples. The pseudo-label is determined by the category corresponding to the highest probability among the two sets of predicted probabilities. Classified by source domain real label Calculate the true class center from the source domain feature set; Then divide the pseudo-labels as described above. The target domain feature set is used, with the predicted probability corresponding to the sample pseudo-label as the confidence weight, to calculate the first... Target domain confidence weighted category center; The cosine distance loss between the source and target domain centers of the same class is calculated to obtain the subdomain alignment loss.
6. The cross-domain EEG emotion recognition method based on dual-domain alignment and dual-classifier collaborative adversarial approach according to claim 5, characterized in that: In the dual-classifier consistency-deterministic constraint loss: The joint consistency term is used to measure the consistency of predictions between two classifiers on high-confidence samples. The local classification determinism term is used to measure the deterministic synergistic effect of two classifiers.
7. The cross-domain EEG emotion recognition method based on dual-domain alignment and dual-classifier collaborative adversarial approach according to claim 6, characterized in that: The cross-domain recognition model trained in step S5 calculates the weighted fusion weights using the consistency-deterministic constraint loss values of the dual classifiers corresponding to each source domain, and obtains the final emotion recognition result.