Cross-individual EEG emotion recognition method based on course learning and multi-source domain adaptation

Abstract

The invention discloses a cross-individual EEG emotion recognition method based on course learning and multi-source domain adaptation, and the method comprises the steps: constructing a multi-modal feature extraction network, and integrating feature extraction modules of time sequence, frequency domain, brain network connectivity and the like, so as to fully mine and fuse multi-dimensional emotion related information. Aiming at challenges of large individual difference, inconsistent data distribution and the like in cross-subject emotion recognition, a multi-source domain adaptation mechanism is introduced, and the discrimination ability and migration performance of a model on a target domain are improved by effectively aligning feature distribution of a source domain and a target domain. Besides, in order to relieve the problems of unstable convergence and local optimization caused by difficult samples in the initial training stage of the model, a course learning strategy is introduced, target domain data is guided to participate in training from shallow to deep according to the sample difficulty, and therefore the convergence, generalization ability and robustness of the model are remarkably enhanced, and the training efficiency is improved. And finally, the overall performance of cross-subject EEG emotion recognition is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of EEG emotion recognition, and in particular relates to a cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation. Background Art

[0002] Emotion recognition holds broad application prospects in today's society, playing an increasingly important role in key areas such as intelligent driving and smart healthcare. Among the various existing emotion recognition methods, those based on electroencephalogram (EEG) signals have attracted considerable attention due to their unique advantages. Compared to traditional methods that rely on external features such as facial expressions or voice, EEG offers a more objective, real-time, and less susceptible to disguised emotional expression, demonstrating greater physiological reliability and recognition potential. However, due to the significant individual variability of EEG signals, cross-individual emotion recognition still faces significant challenges.

[0003] Existing EEG emotion recognition methods fall into two main categories: one is based on traditional machine learning, relying on manual extraction of features such as time domain, frequency domain, nonlinearity, and connectivity, combined with classifiers such as SVM (support vector machine) and KNN (k-nearest neighbor algorithm) for modeling; the other uses deep neural networks to automatically extract spatiotemporal features, such as CNN (convolutional neural network), RNN (recurrent neural network), Transformer (converter) and other structures, or integrate multimodal information to improve recognition performance. Although deep methods perform well in the same-subject setting, in cross-subject scenarios, due to significant differences in neural responses between individuals, model performance generally declines, limiting their practicality.

[0004] To address the issue of insufficient cross-subject generalization, transfer learning and domain adaptation techniques have been widely introduced in the EEG field to reduce inter-subject feature distribution differences and improve the transfer performance of the model to the target individual. Strategies such as MMD (Maximum Mean Difference), DANN (Domain Adversarial Neural Network), CDAN (Conditional Adversarial Domain Adaptation Network), and contrastive learning have been widely used in cross-domain modeling tasks. For example, the paper [Ran Sen, Zhong Wei, Hu Feng. A Coarse-to-Fine Cross-Subject EEG Emotion Recognition Domain Adaptation Method Based on Contrastive Learning [C] / / China Conference on Pattern Recognition and Computer Vision. Singapore: Springer Nature, 2024: 406-419] combines domain adaptation and contrastive learning to reduce inter-domain distribution differences at the coarse granularity level while enhancing intra-class consistency and inter-class separability at the fine granularity level, significantly improving the generalization performance of cross-subject EEG emotion recognition. To further enhance the model's adaptability, MSDA (Multi-Source Domain Adaptation) has been proposed. For example, [Guo Wei, Xu Gang, Wang Ying. Multi-Source Domain Adaptation for EEG Emotion Recognition Based on Spatiotemporal Feature Extractor [J]. Biomedical Signal Processing and Control, 2023, 84:104998] utilizes MSDA to integrate knowledge from multiple source subjects, more comprehensively learning the diverse characteristics of emotional expression and thus achieving stronger generalization capabilities. However, recent studies have found that when training with unlabeled target domain data, it is susceptible to pseudo-label errors or interference from difficult samples, resulting in unstable model training and even trapped in local optimality. Summary of the Invention

[0005] In view of the above, the present invention provides a cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation, which divides subjects into different groups by similarity, uses multiple sources to perform domain adaptation for the target domain and combines it with curriculum learning, so that the neural network learns the distribution of the target domain from easy to difficult, improves the generalization ability of the model in the target domain, and thus achieves personalized emotion recognition.

[0006] A cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation includes the following steps: (1) Obtain an emotional EEG dataset, which stimulates the subject's emotional state through emotion-induced stimulation and synchronously acquires the subject's EEG signal and its corresponding emotional state label; (2) The EEG signals in the dataset are standardized and preprocessed. A large number of EEG segments are obtained by slicing and frequency domain data and connectivity data are extracted. These EEG segments and their frequency domain data and connectivity data are then divided into multiple source domains and one target domain. (3) Cluster the subjects according to the similarity between individuals, divide all subjects into several groups, and merge the source domains of the same group of subjects into one source domain; (4) Construct a source domain shared multimodal feature extraction network to extract features from the input EEG segments and their frequency domain data and connectivity data, and concatenate the extracted multimodal features to obtain a feature vector with multiple modal characteristics; (5) A classification head is independently set for each source domain. The classification head predicts the confidence of each emotional state category through nonlinear mapping based on the feature vector, and constructs a cross-individual EEG emotion recognition model consisting of a multimodal feature extraction network and multiple classification heads; (6) Using the curriculum learning strategy to select a certain amount of unlabeled target domain data, together with the source domain data, to participate in model training, and combining classification loss and multi-order moment matching loss to train the model; (7) The EEG segments in the target domain and their frequency domain data and connectivity data are input into the trained cross-individual EEG emotion recognition model. The prediction results of each classification head are weighted and summed, and the emotional state category with the highest confidence is taken as the recognition result.

[0007] Furthermore, the emotional EEG dataset in step (1) adopts the SEED dataset (Shanghai Jiaotong University emotional EEG dataset) or the DEAP dataset (physiological signal dataset for emotion analysis). The SEED dataset uses movie clips as stimulus materials, collects 62-lead EEG signals from 15 subjects, and annotates them with three types of emotional state labels: positive, neutral, and negative. The DEAP dataset uses music videos as emotion-inducing stimulus materials, collects 32-lead EEG signals from 32 subjects, and annotates them with corresponding labels from four dimensions: pleasure, arousal, dominance, and preference.

[0008] Furthermore, the specific implementation method of the standardization preprocessing in step (2) is as follows: for the EEG signals in the SEED dataset, the signals are first downsampled to 200 Hz, and then a band-pass filter is used to limit the signal frequency range to 0.3~49 Hz, and then the ICA (independent component analysis) algorithm is used to remove artifact interference including eye movement and muscle movement in the signal, and finally the common mean reference method is applied to complete the standardization preprocessing of the signal; for the EEG signals in the DEAP dataset, the signals are first downsampled from 512 Hz to 128 Hz, and then a 4~45 Hz band-pass filter is used to remove low-frequency drift and high-frequency noise in the signal, and then the resting signal portion of the first 3 seconds of each EEG signal is baseline corrected to reduce the baseline differences between individuals.

[0009] Furthermore, in step (2), the preprocessed EEG signal is sliced to obtain multiple EEG segments with a length of 1 s and no overlap, and then feature data is extracted for each EEG segment to obtain corresponding frequency domain data (differential entropy) and connectivity data (phase locking value); a leave-one-out cross-validation strategy is adopted, and for the SEED dataset, the EEG segments and their frequency domain data and connectivity data of 15 subjects are divided into 15 independent groups, one group is selected as the target domain, and the other 14 groups are selected as source domains, and a total of 15 experiments are performed; for the DEAP dataset, the EEG segments and their frequency domain data and connectivity data of 32 subjects are divided into 32 independent groups, one group is selected as the target domain, and the other 31 groups are selected as source domains, and a total of 32 experiments are performed.

[0010] Furthermore, the similarity between individuals in step (3) is obtained by calculating and weighting the Pearson correlation coefficients of the EEG segments, frequency domain data, and connectivity data between the subjects in the source domain, and then clustering all the subjects into several groups based on the similarity.

[0011] Furthermore, the multimodal feature extraction network in step (4) includes The temporal feature extraction module is used to first extract local features of the input EEG segment, then model the long-term dependencies and inter-channel relationships, and finally obtain the temporal features of the EEG segment; The frequency domain feature extraction module is used to obtain frequency domain features after performing high-order nonlinear mapping and compression on the frequency domain data of the EEG segment; Connectivity feature extraction module, which is used to extract local connection patterns and spatial structures from the connectivity data of EEG segments, thereby obtaining connectivity features of inter-channel relationships; The feature combination module is used to combine time series features, frequency domain features, and connectivity features to obtain feature vectors with multiple modal characteristics.

[0012] Furthermore, the temporal feature extraction module first processes the input EEG segment through two layers of one-dimensional convolution, batch normalization layer, and RELU (rectified linear unit) activation function in sequence, then adds position encoding to the processed result and inputs it into Transformer for temporal modeling, and finally obtains the temporal features of the EEG segment after adaptive average pooling and flattening the output result of Transformer; the frequency domain feature extraction module flattens the frequency domain data of the EEG segment and inputs it into MLP (multi-layer perceptron) to obtain frequency domain features; the connectivity feature extraction module first processes the connectivity data of the EEG segment through two layers of two-dimensional convolution, RELU activation function, and maximum pooling layer in sequence, then flattens the processed result and inputs it into MLP to obtain connectivity features.

[0013] Furthermore, in step (6), the following total loss function is used to train the model: Where: L all is the total loss function, L mmm_i is the multi-order moment matching loss of the i-th source domain, L cls_i is the classification loss of the i-th classification head (using cross entropy), which is calculated by inputting the feature vector of the corresponding source domain data into the i-th classification head, λ mmm is the weight coefficient of multi-order moment matching loss, λ i is the weight coefficient of the i-th source domain, and i is the number of the source domain.

[0014] Furthermore, the multi-order moment matching loss L mmm_i The expression is as follows: , Among them: F si_j is the feature vector of the jth EEG segment in the i-th source domain, F ti_k is the feature vector of the jth EEG segment in the target domain data selected for the i-th source domain, μ s_i and μ t_i are the mean of the i-th source domain and the mean of the target domain data selected for the i-th source domain, Σ s_i and Σ t_i are the covariance matrix of the i-th source domain and the covariance matrix of the target domain data selected for the i-th source domain, Loss mean_i and Loss conv_i are the first-order moment matching loss and second-order moment matching loss of the i-th source domain, n si is the number of EEG segments in the i-th source domain, n ti is the number of EEG segments in the target domain data selected for the i-th source domain, represents the Euclidean norm, represents the Frobenius norm, the superscript T Indicates transpose.

[0015] Furthermore, the specific implementation of the course learning strategy in step (6) is as follows: for the i-th source domain, in each round of training, the feature vectors of all EEG segments in the target domain are input into the classification head corresponding to the i-th source domain to predict the confidence of each emotional state category, where the category with the highest confidence is the predicted category, and all EEG segments in the target domain are sorted from high to low according to the confidence of the predicted category, and the EEG segments with the top ratio are selected to participate in the training, where ratio is a set ratio value and gradually increases with the training rounds.

[0016] Based on the above technical solution, the present invention utilizes multimodal EEG signal data and innovatively solves the core challenge of personalized EEG emotion recognition across subjects through a multi-source domain adaptation method combined with the principle of starting with the easy and then the difficult in course learning: First, through course learning, samples of the target domain are selected for training based on the confidence level as the standard, starting with the easy and then the difficult; second, the data set is grouped according to the Pearson correlation coefficient, and the multi-source domain adaptation method is used to align the multimodal features of the source domain data and the target domain data, eliminating the differences between the source domain and target domain subjects. The present invention not only utilizes multimodal EEG data, but also significantly improves the robustness of emotion recognition across subjects by combining the multi-source domain adaptation method with course learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of the cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation of the present invention.

[0018] Figure 2 Schematic diagram of the structure of the multimodal feature extraction network in the present invention.

[0019] Figure 3 Schematic diagram of the overall framework of the cross-individual EEG emotion recognition model in the present invention. DETAILED DESCRIPTION

[0020] In order to describe the present invention more specifically, the technical solution of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 1 As shown, the cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation of the present invention includes the following steps: (1) Obtain a standardized emotional EEG dataset, which stimulates the subject's emotional state through emotion-inducing stimuli (such as video or audio) and simultaneously collects multi-lead EEG signals and corresponding emotion labels.

[0022] In this embodiment, the data sets come from two different research institutions, namely SEED and DEAP. Each data set contains multi-lead EEG signals and corresponding emotional label information from different subjects. The SEED data set was developed by the BCMI (Center for Brain-like Computing and Machine Intelligence) Laboratory of Shanghai Jiao Tong University. It uses 15 standardized movie clips as standardized stimulus materials, recruits 15 healthy subjects and collects signals through a 62-lead EEG system, and collects labeled data containing three emotional dimensions: positive, neutral, and negative. The DEAP data set was jointly developed by the University of Leuven in Belgium and Queen Mary University of the United Kingdom. It uses 40 music videos as emotion-inducing stimuli, recruits 32 subjects and collects signals through 32-lead EEG and a variety of physiological sensors, and records and annotates the four emotional dimensions of pleasure, arousal, dominance, and liking.

[0023] (2) The original EEG signals in the dataset are standardized and preprocessed to generate normalized time series data suitable for deep neural network input.

[0024] In this embodiment, for the SEED dataset, data containing 62 standard EEG recording channels is used. The signal is first downsampled to 200 Hz, and then a bandpass filter is used to limit the signal frequency range to 0.3~49 Hz. Subsequently, an independent component analysis algorithm is used to remove artifacts such as eye movements and muscle movements. Finally, the independent component value weighted average reference method is applied to complete the standardization preprocessing. For the DEAP dataset, the original EEG signal is downsampled from 512 Hz to 128 Hz, and a 4~45 Hz bandpass filter is used to remove low-frequency drift and high-frequency noise. The resting signal 3 seconds before the start of each trial is used for baseline correction. Then, the processed EEG data on the DEAP and SEED datasets are sliced to obtain non-overlapping EEG segments of 1 s in length. The obtained EEG segments are then manually feature extracted to obtain the corresponding frequency domain features (differential entropy) and connectivity features (phase locking value).

[0025] Next, we divided the data into training and test sets, using a leave-one-subject cross-validation strategy. For the SEED dataset, we performed 15 experiments, selecting one subject from the SEED dataset as the test set and the remaining 14 subjects as the training set. For the DEAP dataset, we performed 32 experiments, selecting one subject from the DEAP dataset as the test set and the remaining 31 subjects as the training set. Throughout these experiments, we ensured that data from the same subject did not appear in both the training and test sets.

[0026] (3) The Pearson correlation coefficient was calculated on the EEG data, differential entropy data, and phase-locked value data on the SEED and DEAP datasets, and weighted clustering was performed to obtain domains consisting of five groups of subjects.

[0027] (4) Construct a multimodal feature extraction network shared by multiple source domains, such as Figure 2 As shown, its architecture includes: The temporal feature extraction network is used to first extract local features from the input EEG signal, then model long-term dependencies and inter-channel relationships, and finally obtain temporal features. The specific expression is as follows: Where: x time Represents the input time series EEG data, x cnn Represents x time The result after CNN processing, x tr Represents x cnn The result after Transformer processing, f time Represents the temporal features of the output. CNN includes two layers of one-dimensional convolution + batch normalization + RELU activation function. Transformer represents temporal modeling. P represents adding position encoding. AvgPool represents adaptive average pooling. Flatten represents flattening. T represents transpose; The frequency domain feature extraction network is used to perform high-order nonlinear mapping and compression on the input EEG frequency domain data to obtain its frequency domain features. The specific expression is as follows: Where: x fre Represents the input frequency domain data, namely differential entropy data, f fre Represents the frequency domain features of the output, and MLP represents two layers of fully connected layers + RELU activation function.

[0028] The connectivity feature extraction network is used to extract the local connection pattern and spatial structure of the input EEG connectivity data, thereby obtaining the feature vector of the relationship between channels. The specific expression is as follows: Among them, x con Indicates the input connectivity data, i.e., phase lock value data, f con Represents the connectivity features of the output, and Convs represents two sets of two-dimensional convolution + RELU activation function + maximum pooling.

[0029] (5) Build a classification head for each source domain, combine course learning, select target domain data, and update the network with classification loss and distance loss of multimodal features between source domain and target domain. The specific overall framework and process are as follows: Figure 3 shown.

[0030] First, the three modal features obtained by the multimodal feature extraction network are fused to obtain a feature vector. The specific expression is as follows: Among them: F represents the obtained feature vector with three modes, Concat represents the splicing function, which splices the three features together.

[0031] The feature vector F can be obtained by multimodal feature extraction network and Concat si and F t , where F si represents the feature vector of the i-th source domain sample, F t The feature vector representing the target domain.

[0032] Build a classification head for each source domain. The specific expression is as follows: in: Indicates the original prediction score corresponding to each category, which can reflect the confidence of the model for each category.

[0033] Then, a curriculum learning strategy is used to gradually select target domain samples for training: for any source domain, in each round of training, the extracted target domain feature vector F t After inputting the classification head, the prediction results can be obtained ; Sort all target domain samples according to the prediction confidence, and select the samples with the highest ratio F from high to low ti Used for training; the ratio gradually increases with each training round to achieve progressive optimization from "easy samples" to "hard samples".

[0034] Because the network needs to be able to adapt to the data distribution of the target domain, it is necessary to reduce F si and F ti Therefore, a multi-order moment matching function is used to calculate the loss L between them. mmm_i , the specific expression is as follows: , Among them: F si_j is the feature vector of the jth EEG segment in the i-th source domain, F ti_kis the feature vector of the jth EEG segment in the target domain data selected for the i-th source domain, μ s_i and μ t_i are the mean of the i-th source domain and the mean of the target domain data selected for the i-th source domain, Σ s_i and Σ t_i are the covariance matrix of the i-th source domain and the covariance matrix of the target domain data selected for the i-th source domain, Loss mean_i and Loss conv_i are the first-order moment matching loss and second-order moment matching loss of the i-th source domain, n si is the number of EEG segments in the i-th source domain, n ti is the number of EEG segments in the target domain data selected for the i-th source domain, represents the Euclidean norm, represents the Frobenius norm, the superscript T Indicates transpose.

[0035] Finally, F si Input into the corresponding classification head, according to the sample label and the obtained Calculate the cross entropy loss function L cls_i , the available L cls_i To update the classification head corresponding to the source domain, use the total loss L all To update the shared multimodal feature extraction network, specifically L all The expression is as follows: Where: i is the proportion of the i-th source domain (set according to the degree of similarity between different source domains and target domains), λ mmm Indicates the proportion of multi-order moment matching loss.

[0036] (6) Deploy an emotional reasoning system across subjects.

[0037] The above-trained cross-subject emotion recognition model is used to predict the emotion label of the test set data corresponding to each fold, and the prediction accuracy and Macro-F1 (macro average F1 score) are calculated. The average of the results of all folds is counted and compared with other existing methods. The comparison results are shown in Table 1. It can be seen from the table that the method of the present invention has a significant improvement in recognition effect compared with other existing recognition methods.

[0038] Table 1 The above description of the embodiments is intended to facilitate understanding and application of the present invention by those skilled in the art. It is apparent that those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without requiring creative effort. Therefore, the present invention is not limited to the above embodiments. Any improvements or modifications made by those skilled in the art based on the disclosure of the present invention should fall within the scope of protection of the present invention.

Claims

1. A cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation, characterized by: The steps include: (1) Obtain an emotional EEG dataset, which stimulates the subject's emotional state through emotion-induced stimulation and synchronously acquires the subject's EEG signal and its corresponding emotional state label; (2) The EEG signals in the dataset are standardized and preprocessed. A large number of EEG segments are obtained by slicing and frequency domain data and connectivity data are extracted. These EEG segments and their frequency domain data and connectivity data are then divided into multiple source domains and one target domain. (3) Cluster the subjects according to the similarity between individuals, divide all subjects into several groups, and merge the source domains of the same group of subjects into one source domain; (4) Construct a source domain shared multimodal feature extraction network to extract features from the input EEG segments and their frequency domain data and connectivity data, and concatenate the extracted multimodal features to obtain a feature vector with multiple modal characteristics; (5) A classification head is independently set for each source domain. The classification head predicts the confidence of each emotional state category through nonlinear mapping based on the feature vector, and constructs a cross-individual EEG emotion recognition model consisting of a multimodal feature extraction network and multiple classification heads; (6) Using the curriculum learning strategy to select a certain amount of unlabeled target domain data, together with the source domain data, to participate in model training, and combining classification loss and multi-order moment matching loss to train the model; (7) The EEG segments in the target domain and their frequency domain data and connectivity data are input into the trained cross-individual EEG emotion recognition model. The prediction results of each classification head are weighted and summed, and the emotional state category with the highest confidence is taken as the recognition result.

2. The cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation according to claim 1, characterized in that: The emotional EEG dataset in step (1) uses the SEED dataset or the DEAP dataset. The SEED dataset uses movie clips as stimulus materials, collects 62-lead EEG signals from 15 subjects, and annotates them with three types of emotional state labels: positive, neutral, and negative. The DEAP dataset uses music videos as emotion-inducing stimulus materials, collects 32-lead EEG signals from 32 subjects, and annotates them with corresponding labels based on four dimensions: pleasure, arousal, dominance, and preference.

3. The cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation according to claim 2, characterized in that: The specific implementation method of the standardization preprocessing in step (2) is as follows: for the EEG signals in the SEED dataset, the signals are first downsampled to 200 Hz, and then a band-pass filter is used to limit the signal frequency range to 0.3~49 Hz. Subsequently, the ICA algorithm is used to remove artifact interference including eye movement and muscle movement in the signal, and finally the common mean reference method is applied to complete the standardization preprocessing of the signal; for the EEG signals in the DEAP dataset, the signals are first downsampled from 512 Hz to 128 Hz, and then a 4~45 Hz band-pass filter is used to remove low-frequency drift and high-frequency noise in the signal, and then the resting signal portion of the first 3 seconds of each EEG signal is baseline corrected.

4. The cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation according to claim 2, characterized in that: In the step (2), the preprocessed EEG signal is sliced to obtain multiple EEG segments with a length of 1s and no overlap, and then feature data is extracted for each EEG segment to obtain corresponding frequency domain data and connectivity data; a leave-one-out cross-validation strategy is adopted. For the SEED dataset, the EEG segments and their frequency domain data and connectivity data of 15 subjects are divided into 15 independent groups, one group is selected as the target domain, and the other 14 groups are selected as the source domain, and a total of 15 experiments are performed; for the DEAP dataset, the EEG segments and their frequency domain data and connectivity data of 32 subjects are divided into 32 independent groups, one group is selected as the target domain, and the other 31 groups are selected as the source domain, and a total of 32 experiments are performed.

5. The cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation according to claim 1, characterized in that: The similarity between individuals in step (3) is obtained by calculating and weighting the Pearson correlation coefficients of the EEG segments, frequency domain data, and connectivity data between the subjects in the source domain, and then clustering all the subjects into several groups based on the similarity.

6. The cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation according to claim 1, characterized in that: The multimodal feature extraction network in step (4) includes The temporal feature extraction module is used to first extract local features of the input EEG segment, then model the long-term dependencies and inter-channel relationships, and finally obtain the temporal features of the EEG segment; The frequency domain feature extraction module is used to obtain frequency domain features after performing high-order nonlinear mapping and compression on the frequency domain data of the EEG segment; Connectivity feature extraction module, which is used to extract local connection patterns and spatial structures from the connectivity data of EEG segments, thereby obtaining connectivity features of inter-channel relationships; The feature combination module is used to combine time series features, frequency domain features, and connectivity features to obtain feature vectors with multiple modal characteristics.

7. The cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation according to claim 6, characterized in that: The temporal feature extraction module first processes the input EEG segment through two layers of one-dimensional convolution, batch normalization layer, and RELU activation function, then adds position encoding to the processed result and inputs it into Transformer for temporal modeling, and finally obtains the temporal features of the EEG segment after adaptive average pooling and flattening the output result of Transformer; the frequency domain feature extraction module flattens the frequency domain data of the EEG segment and inputs it into MLP to obtain frequency domain features; the connectivity feature extraction module first processes the connectivity data of the EEG segment through two layers of two-dimensional convolution, RELU activation function, and maximum pooling layer, then flattens the processed result and inputs it into MLP to obtain connectivity features.

8. The cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation according to claim 1, characterized in that: In step (6), the following total loss function is used to train the model: Where: L all is the total loss function, L mmm_i is the multi-order moment matching loss of the i-th source domain, L cls_i is the classification loss of the i-th classification head, which is calculated by inputting the feature vector of the corresponding source domain data into the i-th classification head, λ mmm is the weight coefficient of multi-order moment matching loss, λ i is the weight coefficient of the i-th source domain, and i is the number of the source domain.

9. The cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation according to claim 8, characterized in that: The multi-order moment matching loss L mmm_i The expression is as follows: ， Among them: F si_j is the feature vector of the jth EEG segment in the i-th source domain, F ti_k is the feature vector of the jth EEG segment in the target domain data selected for the i-th source domain, μ s_i and μ t_i are the mean of the i-th source domain and the mean of the target domain data selected for the i-th source domain, Σ s_i and Σ t_i are the covariance matrix of the i-th source domain and the covariance matrix of the target domain data selected for the i-th source domain, Loss mean_i and Loss conv_i are the first-order moment matching loss and second-order moment matching loss of the i-th source domain, n si is the number of EEG segments in the i-th source domain, n ti is the number of EEG segments in the target domain data selected for the i-th source domain, represents the Euclidean norm, represents the Frobenius norm, the superscript T Indicates transpose.

10. The cross-individual EEG emotion recognition method based on curriculum learning and multi-source domain adaptation according to claim 1, characterized in that: The specific implementation method of the course learning strategy in step (6) is as follows: for the i-th source domain, in each round of training, the feature vectors of all EEG segments in the target domain are input into the classification head corresponding to the i-th source domain to predict the confidence of each emotional state category, where the category with the highest confidence is the predicted category, and all EEG segments in the target domain are sorted from high to low according to the confidence of the predicted category, and the EEG segments with the top ratio are selected to participate in the training, where the ratio is a set ratio value and gradually increases with the training rounds.

Images