A dual contrast learning transformer cross-dataset electroencephalogram emotion recognition method

Abstract

The application relates to a dual contrast learning Transformer cross-dataset electroencephalogram emotion recognition method, and belongs to the technical field of electroencephalogram emotion recognition and source domain independent domain self-adaption. The method comprises the following steps: obtaining a source domain electroencephalogram dataset and a target domain electroencephalogram dataset; constructing an electroencephalogram emotion recognition model, including a lightweight time domain Transformer feature extractor and a classifier; performing end-to-end supervised training on the electroencephalogram emotion recognition model by using the source domain electroencephalogram dataset to obtain a pre-training model; inputting the target domain electroencephalogram dataset into the frozen pre-training model to obtain initial target domain features and initial pseudo labels; optimizing the pre-training model through a loss function to obtain a trained electroencephalogram emotion recognition model; the loss function comprises a first stage loss, a second stage loss and a third stage loss; inputting electroencephalogram data to be detected into the trained electroencephalogram emotion recognition model to obtain an electroencephalogram emotion recognition result. The application can improve the accuracy of electroencephalogram emotion recognition.

Description

Technical Field

[0001] This invention belongs to the field of EEG emotion recognition and source-domain-independent domain adaptation technology, specifically involving a dual-contrast learning Transformer cross-dataset EEG emotion recognition method. Background Technology

[0002] Electroencephalogram (EEG) signals, as objective physiological indicators reflecting the electrical activity of the brain's nerves, have significant value in areas such as emotion recognition, cognitive state monitoring, and auxiliary diagnosis of neurological diseases. However, due to their strong individual specificity and context-dependent nature, the accuracy of high-performance models trained in the source domain significantly declines when applied to the target domain. Domain adaptation technology has become crucial for the practical application of EEG decoding. Traditional models assume that training and testing data are distributed in the same way. Early domain adaptation methods required continuous access to the original source domain data. However, EEG data contains sensitive individual information and is strictly constrained by privacy regulations, making source domain data sharing difficult. This limitation restricts its application.

[0003] In the context of deep learning, while domain adaptation methods based on complex neural networks such as Transformer (e.g., CDTrans) have improved feature alignment, they still require real-time reading of source domain data during inference or adaptation phases. This results in high storage overhead and computational latency during deployment, and also poses privacy compliance risks. The current core challenge is to achieve target domain adaptation while completely isolating source domain data and utilizing only trained source domain models. This requires constructing a "source domain-independent" paradigm, condensing source domain knowledge, and designing a model self-evolution mechanism based on target domain data to achieve robust cross-domain generalization while protecting privacy. Summary of the Invention

[0004] To achieve the above objectives, the present invention employs the following technical solution:

[0005] This invention provides a dual-contrast learning Transformer cross-dataset EEG emotion recognition method, comprising the following steps:

[0006] S1. Obtain the source domain EEG dataset and the target domain EEG dataset;

[0007] S2. Construct an EEG emotion recognition model, including a lightweight temporal Transformer feature extractor and a classifier; the lightweight temporal Transformer feature extractor is composed of multiple layers of temporal convolution and standard Transformer encoder blocks concatenated; the classifier includes a global average pooling layer and a fully connected layer; the EEG emotion recognition model is trained end-to-end using a source domain EEG dataset to obtain a pre-trained model; the target domain EEG dataset is input into the frozen pre-trained model, the lightweight temporal Transformer feature extractor outputs the initial target domain features for each target domain EEG data sample, the classifier outputs the predicted probability distribution for each target domain EEG data sample, and the category corresponding to the maximum probability is taken as the initial pseudo-label;

[0008] S3. Optimize the pre-trained model using a loss function to obtain a trained EEG emotion recognition model. The loss function includes the first-stage loss, the second-stage loss, and the third-stage loss. The first-stage loss is the information maximization loss. The second-stage loss uses the triple contrastive learning loss and the cross-entropy classification loss based on high-confidence pseudo-labels. The third-stage loss uses the local consistency loss.

[0009] S4. Input the EEG data to be detected into the trained EEG emotion recognition model to obtain the EEG emotion recognition result.

[0010] Furthermore, the source domain EEG dataset is denoted as... ,in, Indicates the first [item] in the source domain EEG dataset One EEG test; Indicates the first [item] in the source domain EEG dataset The actual emotion category label corresponding to each EEG test; Total number of categories; The target domain EEG dataset is denoted as the total number of EEG trials in the source domain dataset; ,in, Indicates the first EEG data in the target domain One EEG test; This represents the total number of EEG trials in the target domain EEG dataset.

[0011] Furthermore, during pre-training, the source domain prototype for each category is calculated and stored by forward propagating all source domain EEG data samples. The formula is expressed as follows:

[0012] ,

[0013] in, Indicates the category in the source domain EEG dataset. The set of all samples; Indicates the category in the source domain EEG dataset. The total number of all samples; Indicates belonging to a category Source domain EEG data samples; This represents a lightweight temporal Transformer feature extractor; Feature extractor The set of learnable parameters.

[0014] Furthermore, initial pseudo-labels for all target domain EEG data samples A dynamic threshold strategy is used for purification, taking the 75th percentile of the maximum predicted probability of all samples as the dynamic adaptive threshold. The high-confidence sample set is obtained by retaining samples with confidence scores higher than the dynamic adaptive threshold. .

[0015] Furthermore, the formula for maximizing information loss is expressed as follows:

[0016] ,

[0017] in, This indicates a loss of information maximization. This represents the average value of all samples in the target domain. Represents target domain EEG data samples The predicted probability distribution; Represents target domain EEG data samples Category The conditional probability value; Represents the entropy function; Indicates the equilibrium hyperparameters; This represents the average predicted distribution of the batch.

[0018] Furthermore, the triple contrastive learning loss consists of instance contrastive loss. Category comparison loss Loss compared to cross-domain prototype The weighted composition is expressed by the following formula:

[0019] ,

[0020] in, , and This represents three non-negative hyperparameters.

[0021] Furthermore, the example compares the loss. To encourage different augmented views of the same sample to be close together in the feature space, the formula is as follows:

[0022] ,

[0023] in, and This represents two different features that have undergone random enhancement of the same initial target domain features; This represents the characteristics of all samples in the batch; Indicates cosine similarity; Indicates temperature parameter; Indicates batch size;

[0024] Category contrast loss Using the purified pseudo-labels, we can bring similar samples closer together and push away the features of dissimilar samples. The formula is as follows:

[0025] ,

[0026] in, , The pseudo-labels represent the j-th and i-th samples in the same class. , Let represent the initial target domain features of the i-th sample and the initial target domain features of the j-th sample in the same class, respectively; Indicates that the high-confidence sample set is... Initial target domain features of samples of different categories;

[0027] Cross-domain prototype contrast loss Align the feature centers of each category in the target domain with the corresponding pre-stored source domain prototypes, as shown in the following formula:

[0028] ,

[0029] in, This represents the prototype of the target domain, where the pseudo-labels belong to the target domain. The feature mean of all samples in the class; This represents the square of the Euclidean distance.

[0030] Furthermore, the total losses in the second phase The formula is expressed as follows:

[0031] ,

[0032] in, This represents the average value of the sample set for the high-confidence target domain. Indicates the balance weighting coefficient; This represents the cross-entropy loss term.

[0033] Furthermore, in the third stage, for each initial target domain feature, the following calculations are performed: nearest neighbors For each initial target domain feature, Euclidean distance is used as the similarity measure, and the K nearest neighbor algorithm is used to calculate its K nearest neighbor set in the feature space, thus obtaining the reliable neighbor set; local consistency loss. The similarity between sample features and their nearest neighbor features is encouraged, as expressed by the following formula:

[0034] ,

[0035] in, Represents the initial target domain features of Pseudo-labels in neighboring regions A set of reliable neighbors that are identical and have a confidence level higher than the dynamically adaptive threshold; This represents the initial target domain features in the reliable neighbor set.

[0036] The advantages of this invention are:

[0037] This invention proposes a triple contrastive learning framework to achieve multi-level feature alignment without requiring source domain data: instance contrastive comparison improves robustness, class contrastive comparison sharpens decision boundaries, and prototype contrastive comparison maintains cross-domain semantic consistency, significantly improving cross-dataset recognition accuracy. A dynamic reliability filtering mechanism is designed: high-confidence pseudo-labels are filtered through adaptive thresholds, and nearest neighbors are selected based on dual verification, effectively suppressing error propagation during training and improving model robustness. A lightweight and practical deployment scheme is constructed, employing a lightweight temporal Transformer to reduce computational overhead, and combining a freeze-fine-tuning strategy to achieve efficient knowledge transfer while ensuring data privacy, balancing performance and feasibility. Attached Figure Description

[0038] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0039] Figure 1 This is a flowchart of the steps of the method of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Example 1

[0042] In this embodiment, as Figure 1 As shown, this invention provides a dual-contrast learning Transformer cross-dataset EEG emotion recognition method, the specific steps of which include:

[0043] S1. Obtain the source domain EEG dataset and the target domain EEG dataset;

[0044] Specifically, the source domain EEG dataset is denoted as ,in, Indicates the first [item] in the source domain EEG dataset One EEG test; Indicates the first [item] in the source domain EEG dataset The actual emotion category label corresponding to each EEG test; Total number of categories; This represents the total number of EEG trials in the source domain EEG dataset;

[0045] The target domain EEG dataset is denoted as ,in, Indicates the first EEG data in the target domain One EEG test; This represents the total number of EEG trials in the target domain EEG dataset.

[0046] S2. Construct an EEG emotion recognition model, including a lightweight temporal Transformer feature extractor. and classifier ,in, Feature extractor The set of learnable parameters Represents classifier The set of learnable parameters; the lightweight temporal Transformer feature extractor is composed of multiple temporal convolutional layers and standard Transformer encoder blocks concatenated; the classifier includes a global average pooling layer and a fully connected layer.

[0047] Specifically, the source domain EEG dataset is used to perform end-to-end supervised training on the EEG emotion recognition model to obtain a pre-trained model;

[0048] During pre-training, source domain prototypes for each category are calculated and stored by forward propagating all source domain EEG data samples. The formula is expressed as follows:

[0049] ,

[0050] in, Indicates the category in the source domain EEG dataset. The set of all samples; Indicates the category in the source domain EEG dataset. The total number of all samples; Indicates belonging to a category Source domain EEG data samples; This represents a lightweight temporal Transformer feature extractor; Feature extractor The set of learnable parameters;

[0051] Specifically, the target domain EEG dataset is input into the frozen pre-trained model, and the lightweight temporal Transformer feature extractor outputs the initial target domain features for each target domain EEG data sample. The classifier outputs the predicted probability distribution of each target domain EEG data sample, and the class corresponding to the maximum probability is taken as the initial pseudo-label. ; Initial pseudo-labels for all target domain EEG data samples A dynamic threshold strategy is used for purification, taking the 75th percentile of the maximum predicted probability of all samples as the dynamic adaptive threshold. The high-confidence sample set is obtained by retaining samples with confidence scores higher than the dynamic adaptive threshold. ;

[0052] S3. Optimize the pre-trained model using a loss function to obtain a trained EEG emotion recognition model; the loss function includes the first-stage loss, the second-stage loss, and the third-stage loss.

[0053] Unlabeled EEG data in the target domain can lead to weak feature discrimination during direct training, resulting in low confidence and poor class differentiation in the model's prediction of emotion categories. This invention addresses this by minimizing label entropy and maximizing marginal entropy, enabling the model to output highly confident and discriminative predictions for the target domain data, thus laying the foundation for subsequent pseudo-label cleanup and comparative learning.

[0054] Specifically, the lightweight temporal Transformer feature extractor is improved through the first-stage loss. The parameters are fine-tuned to enhance the discriminative power of target domain features by maximizing information. The first-stage loss is the information maximization loss, which consists of two parts: minimizing label entropy and maximizing marginal entropy. It aims to enable the model to make high-confidence and highly discriminative predictions of the target domain data. The formula is as follows:

[0055] ,

[0056] in, This indicates a loss of information maximization. This represents the average value of all samples in the target domain. This indicates the target domain EEG data sample The predicted probability distribution; Represents target domain EEG data samples Category The conditional probability value; Represents the entropy function; Indicates the equilibrium hyperparameters; This represents the average predicted distribution of the batch. .

[0057] Specifically, in the second-stage loss, through contrast constraints across three dimensions—instance, category, and prototype—the feature distribution of the target domain is strongly pulled towards the source domain knowledge space, and the inter-class decision boundaries within the target domain are optimized. The second-stage loss employs a triple contrastive learning loss and a cross-entropy classification loss based on high-confidence pseudo-labels. Cross-domain alignment between the target domain prototype and the source domain prototype is achieved through triple contrastive learning. The triple contrastive learning loss consists of an instance contrastive loss... Category comparison loss Loss compared to cross-domain prototype The weighted composition is expressed by the following formula:

[0058] ,

[0059] in, , and These represent three non-negative hyperparameters used to balance the weights of different alignment objectives. The values ​​of all three hyperparameters range from 0.1 to 1.0.

[0060] EEG signals are susceptible to individual differences and environmental noise interference, and the feature representation of the same sample lacks robustness. To make different augmented views of the same sample closer together in the feature space and improve the model's resistance to noise and perturbations, this invention employs instance contrast loss. To encourage different augmented views of the same sample to be close together in the feature space, the formula is as follows:

[0061] ,

[0062] in, and This represents two different features that have undergone random enhancement of the same initial target domain features; This represents the initial target domain features of all samples in the batch; Indicates cosine similarity; Indicates temperature parameter; This indicates the batch size; random augmentation applies perturbation operators to the initial target domain features, specifically including superimposing Gaussian white noise with a mean of 0 and a standard deviation of 0.01 into the signal, and randomly zeroing out 10% of the time-domain sampling points. By generating different augmented views with semantic consistency, the model is guided in... Under the constraint of loss, it learns interference-resistant sentiment discrimination features.

[0063] The initial pseudo-labels are noisy, and the boundaries between class features are blurred, making it difficult for the model to accurately distinguish different emotion categories; the class contrast loss of this invention... By using purified high-confidence pseudo-labels, we can bring together the features of similar samples and push away the features of dissimilar samples, thus sharpening the inter-class decision boundary. The formula is as follows:

[0064] ,

[0065] in, , The pseudo-labels represent the j-th and i-th samples in the same class. , Let represent the initial target domain features of the i-th sample and the initial target domain features of the j-th sample in the same class, respectively; Indicates that the high-confidence sample set is... Initial target domain features for samples of different categories;

[0066] Significant differences in EEG signal distribution across datasets hinder the effective transfer of knowledge from the source domain to the target domain; this invention employs a cross-domain prototype contrast loss mechanism. Align the feature centers of each category in the target domain with the corresponding pre-stored source domain prototypes to achieve cross-domain semantic consistency and complete source domain knowledge transfer; the formula is expressed as follows:

[0067] ,

[0068] in, This represents the prototype of the target domain, where the pseudo-labels belong to the target domain. The feature mean of all samples in the class; Represents the square of the Euclidean distance;

[0069] Total losses in the second phase The formula is expressed as follows:

[0070] ,

[0071] in, This represents the average value of the sample set for the high-confidence target domain. This represents the balancing weight coefficient, used to adjust the importance of the high-confidence classification loss, and is set to 0.1-1.0. This represents the cross-entropy loss term.

[0072] In the third stage, for each initial target domain feature, its... nearest neighbors For each initial target domain feature, Euclidean distance is used as a similarity measure, and the K nearest neighbor set in the feature space is calculated using the K nearest neighbor algorithm to obtain a reliable neighbor set; specifically, the K value is set to 5, and the Ball-tree algorithm is used to accelerate the search for high-dimensional features.

[0073] The number of high-confidence samples is limited, the local feature distribution in the target domain is inconsistent, the model's generalization ability is insufficient, and classification errors are prone to occur in small sample regions; the third-stage loss in this invention adopts local consistency loss. This encourages sample features to be similar to their nearest neighbor features, and to bring sample features close to their K nearest neighbor features that match their pseudo-labels and meet the confidence threshold. This enhances the consistency of local feature distribution and improves the model's generalization performance in the target domain. The formula is as follows:

[0074] ,

[0075] in, express of Pseudo-labels in neighboring regions A set of reliable neighbors that are identical and have a confidence level higher than the dynamic adaptive threshold;

[0076] S4. Input the EEG data to be detected into the trained EEG emotion recognition model to obtain the EEG emotion recognition result.

[0077] Example 2

[0078] This embodiment conducts comparative experiments on the DEAP and SEED datasets to compare the method of this invention with existing technologies, as shown in Table 1:

[0079] Table 1. Comparative experimental results of the method of the present invention and the prior art.

[0080]

[0081] DEAP SEED indicates that the DEAP dataset is used as the source domain dataset and SEED is used as the target domain dataset; DEAP indicates that the DEAP dataset is used as the source domain dataset and DEAP is used as the target domain dataset; Source-Free Domain indicates that no source domain is required; SHOT: source domain-independent benchmark, G-SFDA: source domain-independent improvement method; DANN: Domain Adversarial Neural Network, CDTrans: Cross-Domain Transformer, both of which require access to both source and target domain data; SHOT and G-SFDA methods are source domain-independent cross-domain adaptation methods, which only require a pre-trained source domain model and a small amount of stored source domain knowledge during the adaptation phase, without requiring the original source domain data; DANN and CDTrans methods are source domain data-based cross-domain adaptation methods, which require access to both source and target domain data during the adaptation phase.

[0082] The method of this invention achieves almost the same performance as the CDTrans method that requires source domain data without requiring source domain data, and is significantly better than other source domain-independent methods. In source domain-independent cases, this invention is significantly better than similar methods (SHOT, G-SFDA). Even compared with methods that require source domain data, this invention is still competitive, or even better, proving the effectiveness of the "triple contrastive learning + dynamic filtering" framework of this invention.

[0083] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dual-contrast learning Transformer cross-dataset EEG emotion recognition method, characterized in that, Includes the following steps: S1. Obtain the source domain EEG dataset and the target domain EEG dataset; S2. Construct an EEG emotion recognition model, including a lightweight temporal Transformer feature extractor and a classifier; the lightweight temporal Transformer feature extractor is composed of multiple layers of temporal convolution and standard Transformer encoder blocks concatenated; the classifier includes a global average pooling layer and a fully connected layer; the EEG emotion recognition model is trained end-to-end using a source domain EEG dataset to obtain a pre-trained model; the target domain EEG dataset is input into the frozen pre-trained model, the lightweight temporal Transformer feature extractor outputs the initial target domain features for each target domain EEG data sample, the classifier outputs the predicted probability distribution for each target domain EEG data sample, and the category corresponding to the maximum probability is taken as the initial pseudo-label; S3. Optimize the pre-trained model using a loss function to obtain a trained EEG emotion recognition model; the loss function includes the first-stage loss, the second-stage loss, and the third-stage loss. The first-stage loss is the information maximization loss; the formula for the information maximization loss is as follows: ， in, This indicates a loss of information maximization. This represents the average value of all samples in the target domain. Represents target domain EEG data samples The predicted probability distribution; Represents target domain EEG data samples Category The conditional probability value; Represents the entropy function; Indicates the equilibrium hyperparameters; This represents the average predicted distribution of the batch; The second-stage loss employs a triple contrastive learning loss and a cross-entropy classification loss based on high-confidence pseudo-labels; the triple contrastive learning loss consists of an instance contrastive loss. Category comparison loss Loss compared to cross-domain prototype The weighted composition is expressed by the following formula: ， in, This indicates the learning loss due to triple contrast. , and Represents three non-negative hyperparameters; total loss in the second stage. The formula is expressed as follows: ， in, This represents the average value of the sample set for the high-confidence target domain. Indicates the balancing weight coefficient; Represents the cross-entropy loss term; The third-stage loss employs local consistency loss; in the third stage, for each initial target domain feature, the following calculation is performed. nearest neighbors For each initial target domain feature, Euclidean distance is used as the similarity measure, and the K nearest neighbor algorithm is used to calculate its K nearest neighbor set in the feature space, thus obtaining the reliable neighbor set; local consistency loss. The similarity between sample features and their nearest neighbor features is encouraged, as expressed by the following formula: ， in, Represents the initial target domain features of Pseudo-labels in neighboring regions A set of reliable neighbors that are identical and have a confidence level higher than the dynamic adaptive threshold; Represents the initial target domain features in the reliable neighbor set; S4. Input the EEG data to be detected into the trained EEG emotion recognition model to obtain the EEG emotion recognition result.

2. The dual-contrast learning Transformer cross-dataset EEG emotion recognition method according to claim 1, characterized in that, The source domain EEG dataset is denoted as ,in, Indicates the first [item] in the source domain EEG dataset One EEG test; Indicates the first [item] in the source domain EEG dataset The actual emotion category label corresponding to each EEG test; Total number of categories; The target domain EEG dataset is denoted as the total number of EEG trials in the source domain dataset; ,in, Indicates the first [item] in the target domain EEG dataset One EEG test; This represents the total number of EEG trials in the target domain EEG dataset.

3. The dual-contrast learning Transformer cross-dataset EEG emotion recognition method according to claim 2, characterized in that, During pre-training, source domain prototypes for each category are calculated and stored by forward propagating all source domain EEG data samples. The formula is expressed as follows: ， in, Indicates the category in the source domain EEG dataset. The set of all samples; Indicates the category in the source domain EEG dataset. The total number of all samples; Indicates belonging to a category Source domain EEG data samples; This represents a lightweight temporal Transformer feature extractor; Feature extractor The set of learnable parameters.

4. The dual-contrast learning Transformer cross-dataset EEG emotion recognition method according to claim 3, characterized in that, Initial pseudo-labels for all target domain EEG data samples A dynamic threshold strategy is used for purification, taking the 75th percentile of the maximum predicted probability of all samples as the dynamic adaptive threshold. The high-confidence sample set is obtained by retaining samples with confidence scores higher than the dynamic adaptive threshold. .

5. The dual-contrast learning Transformer cross-dataset EEG emotion recognition method according to claim 4, characterized in that, The example compares the loss. To encourage different augmented views of the same sample to be close together in the feature space, the formula is as follows: ， in, and This represents two different features that have undergone random enhancement of the same initial target domain features; This represents the characteristics of all samples in the batch; Indicates cosine similarity; Indicates temperature parameter; Indicates batch size; Category contrast loss Using the purified pseudo-labels, we can bring similar samples closer together and push away the features of dissimilar samples. The formula is as follows: ， in, , The pseudo-labels represent the j-th and i-th samples in the same class. , Let represent the initial target domain features of the i-th sample and the initial target domain features of the j-th sample in the same class, respectively; Indicates that the high-confidence sample set is... Initial target domain features for samples of different categories; Cross-domain prototype contrast loss Align the feature centers of each category in the target domain with the corresponding pre-stored source domain prototypes, as shown in the following formula: ， in, This represents the prototype of the target domain, where the pseudo-labels belong to the target domain. The feature mean of all samples in the class; This represents the square of the Euclidean distance.

Images