Cross-subject eeg emotion recognition method based on course learning and multi-source domain adaptation

Abstract

The application discloses a cross-individual EEG emotion recognition method based on course learning and multi-source domain adaptation, which constructs a multi-modal feature extraction network, integrates time sequence, frequency domain and brain network connectivity feature extraction modules, so as to fully mine and fuse multi-dimensional emotion-related information. In view of the challenges of large individual difference and inconsistent data distribution in cross-subject emotion recognition, the application introduces a multi-source domain adaptation mechanism, effectively aligns the feature distribution of the source domain and the target domain, and improves the discrimination ability and migration performance of the model in the target domain. In addition, in order to alleviate the convergence instability and local optimum problem caused by difficult samples in the early stage of model training, the application introduces a course learning strategy, guides the target domain data to participate in training according to the sample difficulty from shallow to deep, thereby significantly enhancing the convergence, generalization ability and robustness of the model, and finally effectively improving the overall performance of cross-subject EEG emotion recognition.

Description

Technical Field

[0001] This invention belongs to the field of EEG emotion recognition technology, specifically relating to a cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation. Background Technology

[0002] Emotion recognition has broad application prospects in contemporary society, playing an increasingly important role, especially in key areas such as autonomous driving and intelligent healthcare. Among the various existing emotion recognition methods, those based on electroencephalography (EEG) signals have attracted much attention due to their unique advantages. Compared to traditional methods that rely on external features such as facial expressions or voice, EEG can provide a more objective, real-time, and difficult-to-fake pathway for emotion expression, demonstrating higher physiological reliability and recognition potential. However, due to the significant individual variability of EEG signals themselves, cross-individual emotion recognition still faces serious challenges.

[0003] Existing EEG emotion recognition methods are mainly divided into two categories: one is based on traditional machine learning, relying on manual extraction of features in the time domain, frequency domain, nonlinearity, and connectivity, combined with classifiers such as SVM (Support Vector Machine) and KNN (K Nearest Neighbors) for modeling; the other uses deep neural networks to automatically extract spatiotemporal features, such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), and Transformer structures, or integrates multimodal information to improve recognition performance. Although deep methods perform well in the same subject setting, in cross-subject scenarios, due to significant differences in neural responses between individuals, model performance generally declines, limiting its practicality.

[0004] To address the issue of insufficient generalization ability across subjects, transfer learning and domain adaptation techniques have been widely introduced into the EEG field to reduce the differences in feature distribution among different subjects and improve the transfer performance of the model on the target individual. Among them, strategies such as MMD (Maximum Mean Difference), DANN (Domain Adversarial Neural Network), CDAN (Conditional Adversarial Domain Adaptation Network), and contrastive learning are widely used in cross-domain modeling tasks. For example, the literature [Ran Sen, Zhong Wei, Hu Feng. A Coarse-to-Fine Domain Adaptation Method for Cross-Subject EEG Emotion Recognition Based on Contrastive Learning [C] / / China Conference on Pattern Recognition and Computer Vision. Singapore: Springer Nature, 2024: 406-419] combines domain adaptation and contrastive learning, which reduces the differences in distribution between domains at the coarse-grained level while enhancing intra-class consistency and inter-class separability at the fine-grained level, thereby significantly improving the generalization performance of cross-subject EEG emotion recognition. To further enhance the adaptability of models, Multi-Source Domain Adaptation (MSDA) has been proposed. For example, the literature [Guo Wei, Xu Gang, Wang Ying. Multi-Source Domain Adaptation EEG Emotion Recognition Method Based on Spatiotemporal Feature Extractor [J]. Biomedical Signal Processing and Control, 2023, 84:104998] utilizes the multi-source domain adaptation method to integrate the knowledge of multiple source subjects, thereby learning the diverse features of emotion expression more comprehensively and achieving stronger generalization ability. However, recent studies have found that when using unlabeled target domain data for training, it is susceptible to pseudo-label errors or difficult samples, leading to unstable model training or even getting stuck in local optima. Summary of the Invention

[0005] In view of the above, the present invention provides a cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation. It divides subjects into different groups through similarity, uses multi-source domain adaptation for the target domain and combines curriculum learning to enable the neural network to learn the distribution of the target domain from easy to difficult, improves the model's generalization ability on the target domain, and thus achieves personalized emotion recognition.

[0006] A cross-individual EEG sentiment recognition method based on curriculum learning and multi-source domain adaptation includes the following steps:

[0007] (1) Obtain an emotional EEG dataset, which is generated by stimulating the subject’s emotional state through emotional induction and simultaneously collecting the subject’s EEG signals and their corresponding emotional state labels.

[0008] (2) Standardize and preprocess the EEG signals in the dataset, obtain a large number of EEG segments by slicing and extract frequency domain data and connectivity data, and then divide these EEG segments and their frequency domain data and connectivity data into multiple source domains and one target domain;

[0009] (3) Cluster the subjects according to the similarity between individuals, divide all subjects into several groups, and merge the source domains of the same group of subjects into one source domain;

[0010] (4) Construct a source domain shared multimodal feature extraction network to extract features from the input EEG fragments and their frequency domain data and connectivity data. Then, concatenate the extracted multimodal features to obtain feature vectors with multiple modal characteristics.

[0011] (5) Set up a classification head independently for each source domain. The classification head predicts the confidence of each emotional state category through nonlinear mapping based on the feature vector, and constructs a cross-individual EEG emotion recognition model composed of a multimodal feature extraction network and multiple classification heads.

[0012] (6) A certain amount of unlabeled target domain data is selected using a course learning strategy and used together with the source domain data to participate in model training. The model is trained by combining classification loss and multi-moment matching loss.

[0013] (7) Input the EEG fragments in the target domain and their frequency domain data and connectivity data into the trained cross-individual EEG emotion recognition model, and take the emotion state category with the highest confidence after weighted summation of the prediction results of each classifier as the recognition result.

[0014] Furthermore, the emotional EEG dataset in step (1) uses either the SEED dataset (Shanghai Jiao Tong University Emotional EEG Dataset) or the DEAP dataset (Physiological Signal Dataset for Emotion Analysis). The SEED dataset uses movie clips as stimulus material and collects 62-lead EEG signals from 15 subjects, labeling them with three emotional state labels: positive, neutral, and negative. The DEAP dataset uses music videos as emotional evoked stimulus material and collects 32-lead EEG signals from 32 subjects, labeling them with corresponding labels from four dimensions: pleasure, arousal, dominance, and liking.

[0015] Furthermore, the specific implementation of the standardization preprocessing in step (2) is as follows: For the EEG signals in the SEED dataset, the signal is first downsampled to 200Hz, and then a bandpass filter is used to limit the signal frequency range to 0.3~49Hz. Subsequently, the ICA (Independent Component Analysis) algorithm is used to remove artifact interference, including eye movement and muscle movement, from the signal. Finally, the common average reference method is applied to complete the standardization preprocessing of the signal. For the EEG signals in the DEAP dataset, the signal is first downsampled from 512Hz to 128Hz, and then a bandpass filter of 4~45Hz is used to remove low-frequency drift and high-frequency noise from the signal. Then, the resting signal portion of the first 3 seconds of each EEG signal is baseline corrected to reduce the baseline differences between individuals.

[0016] Further, in step (2), the preprocessed EEG signal is sliced ​​to obtain multiple EEG segments of 1s length and without overlap. Then, feature data is extracted from each EEG segment to obtain the corresponding frequency domain data (differential entropy) and connectivity data (phase-locked value). A leave-one-out cross-validation strategy is adopted. For the SEED dataset, the EEG segments of 15 subjects and their frequency domain data and connectivity data are divided into 15 independent groups. One group is selected as the target domain and the other 14 groups are used as the source domain. A total of 15 experiments are conducted. For the DEAP dataset, the EEG segments of 32 subjects and their frequency domain data and connectivity data are divided into 32 independent groups. One group is selected as the target domain and the other 31 groups are used as the source domain. A total of 32 experiments are conducted.

[0017] Furthermore, in step (3), the similarity between individuals is obtained by calculating and weighting the Pearson correlation coefficients between subjects in the source domain regarding EEG fragments, frequency domain data, and connectivity data, and then clustering based on the similarity to divide all subjects into several groups.

[0018] Furthermore, the multimodal feature extraction network in step (4) includes

[0019] The temporal feature extraction module is used to first extract local features from the input EEG segment, then model long-term dependencies and inter-channel relationships, and finally obtain the temporal features of the EEG segment.

[0020] The frequency domain feature extraction module is used to obtain frequency domain features by performing high-order nonlinear mapping and compression on the frequency domain data of EEG segments;

[0021] The connectivity feature extraction module is used to extract local connectivity patterns and spatial structure from the connectivity data of EEG segments, thereby obtaining connectivity features of inter-channel relationships.

[0022] The feature combination module is used to concatenate temporal features, frequency domain features, and connectivity features to obtain feature vectors with multiple modal characteristics.

[0023] Furthermore, the temporal feature extraction module first processes the input EEG segment sequentially through two layers of one-dimensional convolution, batch normalization layer, and ReLU (corrected linear unit) activation function. Then, position encoding is added to the processed result before it is input into a Transformer for temporal modeling. Finally, the output of the Transformer is processed through adaptive average pooling and flattened to obtain the temporal features of the EEG segment. The frequency domain feature extraction module flattens the frequency domain data of the EEG segment and inputs it into an MLP (multilayer perceptron) for processing to obtain frequency domain features. The connectivity feature extraction module first processes the connectivity data of the EEG segment sequentially through two layers of two-dimensional convolution, ReLU activation function, and max pooling layer. Then, the processed result is flattened and input into an MLP for processing to obtain connectivity features.

[0024] Furthermore, in step (6), the model is trained using the following total loss function:

[0025]

[0026] Where: L all Let L be the total loss function. mmm_i For the multi-moment matching loss of the i-th source domain, L cls_i Let λ be the classification loss (using cross-entropy) for the i-th classifier, which is calculated by inputting the feature vector of the corresponding source domain data into the i-th classifier. mmm λ represents the weighting coefficients of the multi-moment matching loss. i Let be the weight coefficient of the i-th source domain, where i is the index of the source domain.

[0027] Furthermore, the multi-order moment matching loss L mmm_i The expression is as follows:

[0028]

[0029] ,

[0030]

[0031]

[0032]

[0033] Wherein: F si_j Let F be the feature vector of the j-th EEG segment in the i-th source domain. ti_k Let μ be the feature vector of the j-th EEG segment in the target domain data selected for the i-th source domain. s_i and μ t_iLet Σ be the mean of the i-th source domain and the mean of the target domain data selected for the i-th source domain, respectively. s_i and Σ t_i Let Loss be the covariance matrix of the i-th source domain and the covariance matrix of the target domain data selected for the i-th source domain, respectively. mean_i and Loss conv_i Let n be the first-order moment matching loss and the second-order moment matching loss of the i-th source domain, respectively. si Let n be the number of EEG fragments in the i-th source domain. ti This represents the number of EEG fragments selected from the target domain data for the i-th source domain. Represents the Euclidean norm. The superscript indicates the Frobenius norm. T This indicates transpose.

[0034] Furthermore, the specific implementation of the course learning strategy in step (6) is as follows: For the i-th source domain, in each round of training, the feature vectors of all EEG segments in the target domain are input into the corresponding classification head of the i-th source domain to predict the confidence of each emotional state category. The category with the highest confidence is the predicted category. All EEG segments in the target domain are sorted from high to low according to the confidence of the predicted category. The EEG segments with the highest ratio are selected to participate in training. The ratio is a set ratio value and gradually increases with the training rounds.

[0035] Based on the above technical solution, this invention utilizes multimodal EEG signal data and innovatively solves the core challenge of personalized EEG emotion recognition across subjects by combining a multi-source domain adaptation method with the principle of learning from easy to difficult in a curriculum: First, through curriculum learning, samples from the target domain are selected for training based on confidence levels, progressing from easy to difficult; second, the dataset is grouped according to the Pearson correlation coefficient, and the multi-source domain adaptation method is used to align the multimodal features of the source domain data and the target domain data, eliminating the differences between subjects in the source and target domains. This invention not only utilizes multimodal EEG data but also significantly improves the robustness of cross-subject emotion recognition by combining a multi-source domain adaptation method with curriculum learning. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the cross-individual EEG emotion recognition method based on course learning and multi-source domain adaptation of the present invention.

[0037] Figure 2 This is a schematic diagram of the structure of the multimodal feature extraction network in this invention.

[0038] Figure 3 This is a schematic diagram of the overall framework of the cross-individual EEG emotion recognition model in this invention. Detailed Implementation

[0039] To describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] like Figure 1 As shown, the present invention provides a cross-individual EEG sentiment recognition method based on curriculum learning and multi-source domain adaptation, comprising the following steps:

[0041] (1) Obtain a standardized emotional EEG dataset. The dataset stimulates the emotional state of the subjects through emotional stimuli (such as video or audio) and simultaneously collects multi-lead EEG signals and corresponding emotional labels.

[0042] In this embodiment, the datasets come from two different research institutions: SEED and DEAP. Each dataset contains multi-lead EEG signals and corresponding emotional labeling information from different subjects. The SEED dataset was developed by the BCMI (Brain-Inspired Computing and Machine Intelligence Research Center) laboratory at Shanghai Jiao Tong University. It uses 15 standardized film clips as standardized stimuli, recruits 15 healthy subjects, and collects signals through a 62-lead EEG system, collecting labeled data including three emotional dimensions: positive, neutral, and negative. The DEAP dataset was jointly developed by KU Leuven in Belgium and Queen Mary University of London in the UK. It uses 40 music videos as emotional stimuli, recruits 32 subjects, and collects signals through a 32-lead EEG and various physiological sensors, recording and labeling four emotional dimensions: pleasure, arousal, dominance, and liking.

[0043] (2) Standardize the original EEG signals in the dataset to generate normalized time series data suitable for deep neural network input.

[0044] In this implementation, for the SEED dataset, data containing 62 standard EEG recording channels was used. First, the signal was downsampled to 200Hz. Then, a bandpass filter was used to limit the signal frequency range to 0.3–49Hz. Subsequently, independent component analysis (ICA) was used to remove artifacts such as eye movement and muscle movement. Finally, a weighted average reference method based on independent component values ​​was applied to complete the standardization preprocessing. For the DEAP dataset, the original EEG signal was downsampled from 512Hz to 128Hz, and a bandpass filter of 4–45Hz was used to remove low-frequency drift and high-frequency noise. Baseline correction was performed using the resting signal from 3 seconds before each trial. Then, the processed EEG data from the DEAP and SEED datasets were sliced ​​to obtain non-overlapping EEG segments of 1 second in length. Manual feature extraction was then performed on these EEG segments to obtain the corresponding frequency domain features (differential entropy) and connectivity features (phase-locked values).

[0045] Next, training and test sets were divided, employing a leave-one-out cross-validation strategy: For the SEED dataset, 15 experiments were conducted, sequentially selecting one subject from the SEED dataset as the test set and the remaining 14 subjects as the training set. For the DEAP dataset, 32 experiments were conducted, sequentially selecting one subject from the DEAP dataset as the test set and the remaining 31 subjects as the training set. In these experiments, it was ensured that data from the same subject would not appear in both the training and test sets simultaneously.

[0046] (3) Calculate the Pearson correlation coefficient on the EEG data, differential entropy data and phase-locked value data on the SEED and DEAP datasets respectively, and perform weighted clustering to obtain a domain composed of five groups of subjects.

[0047] (4) Construct a multimodal feature extraction network shared by multiple source domains, such as Figure 2 As shown, its architecture includes:

[0048] The temporal feature extraction network is used to first extract local features from the input EEG signal, then model long-term dependencies and inter-channel relationships, and finally obtain temporal features. The specific expression is as follows:

[0049]

[0050]

[0051]

[0052] Where: x time This represents the input time-series EEG data, x cnn x represents time The result after CNN processing, x tr x represents cnn The result after processing by Transformer, f time The output represents the temporal features. CNN consists of two layers of one-dimensional convolution + batch normalization + ReLU activation function. Transformer represents temporal modeling. P represents adding positional encoding. AvgPool represents adaptive average pooling. Flatten represents flattening. Superscript indicates... T Indicates transpose;

[0053] The frequency domain feature extraction network is used to perform high-order nonlinear mapping and compression on the input EEG frequency domain data to obtain its frequency domain features. The specific expression is as follows:

[0054]

[0055] Where: x fre This represents the input frequency domain data, i.e., the differential entropy data, f.fre The frequency domain characteristics of the output are represented by MLP, which stands for two fully connected layers plus the ReLU activation function.

[0056] The connectivity feature extraction network is used to extract local connectivity patterns and spatial structure from the input EEG connectivity data, thereby obtaining feature vectors of inter-channel relationships. The specific expression is as follows:

[0057]

[0058] Where, x con This represents the input connectivity data, i.e., the phase-locked loop (PLL) value data, f. con The output connectivity features are represented by Convs, which represents two sets of two-dimensional convolutions + ReLU activation function + max pooling.

[0059] (5) Construct a classification head for each source domain. Based on the course learning, select target domain data and update the network using classification loss and distance loss of multimodal features between the source and target domains. The specific overall framework and process are as follows: Figure 3 As shown.

[0060] First, the three modal features obtained through the multimodal feature extraction network are fused together to obtain a feature vector, as shown in the following expression:

[0061]

[0062] Where: F represents the feature vector with three modalities, and Concat represents the concatenation function that concatenates the three features together.

[0063] The feature vector F can be obtained through a multimodal feature extraction network and Concat. si and F t , where F si F represents the feature vector of the i-th source domain sample. t This represents the feature vector of the target domain.

[0064] For each source domain, construct a classification head, with the following specific expression:

[0065]

[0066] in: This represents the original predicted score for each category, which reflects the model's confidence level for each category.

[0067] Then, a course learning strategy is used to progressively select target domain samples for training: for any source domain, in each round of training, the extracted target domain feature vector F is... t The prediction results can be obtained after inputting the classification header. All target domain samples are sorted according to prediction confidence, and samples F with the highest ratios are selected from highest to lowest. ti Used for training; where the ratio gradually increases with each training round to achieve progressive optimization from "easy samples" to "difficult samples".

[0068] Because the network needs to be able to adapt to the data distribution of the target domain, it is necessary to reduce F. si and F ti The distribution differences between them. Therefore, the loss L between them is calculated using a multi-moment matching function. mmm_i The specific expression is as follows:

[0069]

[0070] ,

[0071]

[0072]

[0073]

[0074] Wherein: F si_j Let F be the feature vector of the j-th EEG segment in the i-th source domain. ti_k Let μ be the feature vector of the j-th EEG segment in the target domain data selected for the i-th source domain. s_i and μ t_i Let Σ be the mean of the i-th source domain and the mean of the target domain data selected for the i-th source domain, respectively. s_i and Σ t_i Let Loss be the covariance matrix of the i-th source domain and the covariance matrix of the target domain data selected for the i-th source domain, respectively. mean_i and Loss conv_i Let n be the first-order moment matching loss and the second-order moment matching loss of the i-th source domain, respectively. si Let n be the number of EEG fragments in the i-th source domain. ti This represents the number of EEG fragments selected from the target domain data for the i-th source domain. Represents the Euclidean norm. Denotes the Frobenius norm, superscript T This indicates transpose.

[0075] Finally, F si The data is input into the corresponding classification header, based on the sample's label and the obtained... Calculate the cross-entropy loss function L cls_i , using the obtained L cls_iUpdate the classification header corresponding to the source domain, using the total loss L. all Update the shared multimodal feature extraction network, specifically L all The expression is as follows:

[0076]

[0077] Where: λ i λ represents the proportion of the i-th source domain (set according to the similarity between different source domains and the target domain). mmm This indicates the proportion of the loss due to multi-order moment matching.

[0078] (6) Deploy an emotional reasoning system across individual subjects.

[0079] Using the trained cross-subject emotion recognition model, the emotion label is predicted on the test set data corresponding to each fold. The prediction accuracy and Macro-F1 score are calculated. The average value of all fold results is calculated and compared with other existing methods. The comparison results are shown in Table 1. It can be seen from the table that the method of the present invention has a significant improvement in recognition effect compared with other existing recognition methods.

[0080] Table 1

[0081]

[0082] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. Those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A cross-individual EEG sentiment recognition method based on curriculum learning and multi-source domain adaptation, characterized in that, Includes the following steps: (1) Obtain an emotional EEG dataset, which is generated by stimulating the subject’s emotional state through emotional induction and simultaneously collecting the subject’s EEG signals and their corresponding emotional state labels. (2) Standardize and preprocess the EEG signals in the dataset, obtain a large number of EEG segments by slicing and extract frequency domain data and connectivity data, and then divide these EEG segments and their frequency domain data and connectivity data into multiple source domains and one target domain. (3) Cluster the subjects according to the similarity between individuals, divide all subjects into several groups, and merge the source domains of the same group of subjects into one source domain; (4) Construct a source-domain shared multimodal feature extraction network to extract features from the input EEG fragments and their frequency domain data and connectivity data. The extracted multimodal features are then concatenated to obtain a feature vector with multiple modal characteristics. The multimodal feature extraction network includes: The temporal feature extraction module first processes the input EEG fragments sequentially through two layers of one-dimensional convolution, batch normalization layers, and ReLU activation function. Then, it adds positional encoding to the processed results and inputs them into the Transformer for temporal modeling. Finally, it obtains the temporal features of the EEG fragments by applying adaptive average pooling and flattening the output of the Transformer. The frequency domain feature extraction module is used to flatten the frequency domain data of the EEG segment and input it into the MLP for processing to obtain frequency domain features; The connectivity feature extraction module first processes the connectivity data of the EEG fragment through two layers of two-dimensional convolution, ReLU activation function, and max pooling layer, and then flattens the processed result before inputting it into the MLP to obtain connectivity features. The feature combination module is used to concatenate temporal features, frequency domain features, and connectivity features to obtain feature vectors with multiple modal characteristics; (5) Set up a classification head independently for each source domain. The classification head predicts the confidence of each emotional state category based on the feature vector through nonlinear mapping, and constructs a cross-individual EEG emotion recognition model composed of a multimodal feature extraction network and multiple classification heads. (6) A certain amount of unlabeled target domain data is selected using a course learning strategy and used together with the source domain data to participate in model training. The model is trained by combining classification loss and multi-moment matching loss. (7) Input the EEG fragments in the target domain and their frequency domain data and connectivity data into the trained cross-individual EEG emotion recognition model, and take the emotion state category with the highest confidence after weighted summation of the prediction results of each classifier as the recognition result.

2. The cross-individual EEG sentiment recognition method based on curriculum learning and multi-source domain adaptation according to claim 1, characterized in that: The emotional EEG dataset in step (1) uses either the SEED dataset or the DEAP dataset. The SEED dataset uses movie clips as stimulus material and collects 62-lead EEG signals from 15 subjects, labeling them with three emotional state labels: positive, neutral, and negative. The DEAP dataset uses music videos as emotional evoked stimulus material and collects 32-lead EEG signals from 32 subjects, labeling them with corresponding labels from four dimensions: pleasure, arousal, dominance, and liking.

3. The cross-individual EEG sentiment recognition method based on curriculum learning and multi-source domain adaptation according to claim 2, characterized in that: The specific implementation of the standardization preprocessing in step (2) is as follows: For the EEG signal in the SEED dataset, the signal is first downsampled to 200Hz, and then a bandpass filter is used to limit the signal frequency range to 0.3 to 49Hz. Subsequently, the ICA algorithm is used to remove artifact interference, including eye movement and muscle movement, from the signal. Finally, the common average reference method is applied to complete the standardization preprocessing of the signal. For the EEG signal in the DEAP dataset, the signal is first downsampled from 512Hz to 128Hz, and then a bandpass filter of 4 to 45Hz is used to remove low-frequency drift and high-frequency noise from the signal. Then, the resting signal portion of the first 3 seconds of each EEG signal is baseline corrected.

4. The cross-individual EEG sentiment recognition method based on curriculum learning and multi-source domain adaptation according to claim 2, characterized in that: In step (2), the preprocessed EEG signal is sliced ​​to obtain multiple EEG segments of 1s length without overlap. Then, feature data is extracted from each EEG segment to obtain the corresponding frequency domain data and connectivity data. A leave-one-out cross-validation strategy is adopted. For the SEED dataset, the EEG segments, frequency domain data and connectivity data of 15 subjects are divided into 15 independent groups. One group is selected as the target domain and the other 14 groups are used as the source domain. A total of 15 experiments are conducted. For the DEAP dataset, the EEG segments, frequency domain data and connectivity data of 32 subjects are divided into 32 independent groups. One group is selected as the target domain and the other 31 groups are used as the source domain. A total of 32 experiments are conducted.

5. The cross-individual EEG sentiment recognition method based on curriculum learning and multi-source domain adaptation according to claim 1, characterized in that: In step (3), the similarity between individuals is obtained by calculating and weighting the Pearson correlation coefficients between subjects in the source domain regarding EEG fragments, frequency domain data, and connectivity data. Then, clustering is performed based on the similarity to divide all subjects into several groups.

6. The cross-individual EEG sentiment recognition method based on curriculum learning and multi-source domain adaptation according to claim 1, characterized in that: In step (6), the following total loss function is used to train the model: Where: L all Let L be the total loss function. mmm_i For the multi-moment matching loss of the i-th source domain, L cls_i Let λ be the classification loss of the i-th classifier, which is calculated by inputting the feature vector of the corresponding source domain data into the i-th classifier. mmm λ represents the weighting coefficients of the multi-moment matching loss. i Let be the weight coefficient of the i-th source domain, where i is the index of the source domain.

7. The cross-individual EEG sentiment recognition method based on curriculum learning and multi-source domain adaptation according to claim 6, characterized in that: The multi-order moment matching loss L mmm_i The expression is as follows: L mmm_i =Loss mean_i +Loss conv_i Wherein: F si_j Let F be the feature vector of the j-th EEG segment in the i-th source domain. ti_k Let μ be the feature vector of the k-th EEG segment in the target domain data selected for the i-th source domain. s_i and μ t_i Let Σ be the mean of the i-th source domain and the mean of the target domain data selected for the i-th source domain, respectively. s_i and Σ t_i Let Loss be the covariance matrix of the i-th source domain and the covariance matrix of the target domain data selected for the i-th source domain, respectively. mean_i and Loss conv_i Let n be the first-order moment matching loss and the second-order moment matching loss of the i-th source domain, respectively. si Let n be the number of EEG fragments in the i-th source domain. ti Let ||||2 represent the number of EEG fragments in the target domain data selected for the i-th source domain, and ||||2 represent the Euclidean norm. F Denotes the Frobenius norm, superscript T This indicates transpose.

8. The cross-individual EEG sentiment recognition method based on curriculum learning and multi-source domain adaptation according to claim 1, characterized in that: The specific implementation of the course learning strategy in step (6) is as follows: For the i-th source domain, in each round of training, the feature vectors of all EEG segments in the target domain are input into the corresponding classifier of the i-th source domain to predict the confidence of each emotional state category. The category with the highest confidence is the predicted category. All EEG segments in the target domain are sorted from high to low according to the confidence of the predicted category. The EEG segments with the highest ratio are selected to participate in training. The ratio is a set ratio value and gradually increases with the training rounds.

Images