Cross-session-tested emotion recognition method in combination with feature decomposition and transfer learning

Abstract

The invention discloses a cross-session-tested emotion recognition method combining feature decomposition and transfer learning, which comprises the following steps: preprocessing an original EEG (electroencephalogram) signal, and extracting EEG signal features; decomposing EEG signal features to obtain brain common features and individual private features between a source domain and a target domain; performing cross-domain fine-grained alignment and intra-domain optimization processing on brain common characteristics between the source domain and the target domain; the brain common characteristics of the source domain are decoded, and then emotion recognition and classification are achieved; and constructing a joint optimization objective function based on cross-domain fine-grained alignment loss, intra-domain optimization loss and balance classification loss so as to optimize model training and emotion recognition classification. According to the method, through common-private feature decomposition, cross-domain fine-grained alignment and intra-domain optimization strategies, comprehensive alignment of feature space distribution and effective optimization of an intra-domain structure are realized, and generalization ability and classification accuracy of EEG emotion recognition are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of emotion recognition, and particularly relates to a cross-session-subject emotion recognition method combining feature decomposition and transfer learning. Background Technique

[0002] In recent years, electroencephalogram (EEG)-based emotion recognition technology has received extensive attention because it can objectively reflect an individual's emotional state and shows great potential in the fields of mental health, brain-computer interfaces, and intelligent human-computer interaction. In human-computer interaction, emotional interaction is particularly important, so it is very urgent to help computers effectively recognize human emotions. In the field of affective computing, EEG-based emotion recognition technology has attracted much attention due to its characteristics of directly reflecting brain activities.

[0003] Traditional EEG emotion recognition methods usually adopt the method of combining manual feature extraction and classifiers. For example, researchers widely use features such as power spectral density (PSD) and combine classifiers such as support vector machines (SVMs) and random forests (RFs) for emotion recognition. Such methods can improve the interpretability of emotion recognition to a certain extent, but their feature extraction highly depends on manual experience, which limits the generality of the model. Moreover, feature extraction and classification are separated, and the model cannot fully utilize the distribution information of the data for adaptive learning.

[0004] In recent years, more and more deep learning has been applied in the field of EEG emotion recognition to achieve an end-to-end framework and reduce the dependence on manual experience. Among them, convolutional neural networks (CNNs), recurrent neural networks (RNNs), graph neural networks (GNNs), etc. have been widely used for automatic feature extraction, providing the possibility for decoding emotion tendencies. Although deep learning has made great progress in EEG emotion recognition, due to the individual differences in EEG data, models based on deep learning often have difficulty generalizing between different subjects. To solve the problem of individual differences in EEG signals, various transfer learning methods have emerged in recent years in order to achieve effective transfer of emotional knowledge from source subjects to target subjects, as Figure 1 shown.

[0005] Currently, various methods have emerged for the study of multi-domain differences. For example, existing technologies transfer full-channel EEG knowledge to few-channel EEG through cross-device transfer learning, improving the performance of few-channel emotion recognition. Existing technologies, based on the innovative design of a wide feature extractor (WFE), random matching operation (RMO), and ranking domain classifier selection (TOP), improve cross-domain EEG emotion recognition performance by effectively aligning the distributions of multi-source domains and target domains. In addition, existing technologies have also proposed a multi-modal coarse-fine grained distribution alignment method to achieve inter-domain feature alignment and efficient emotion modeling within the modality. At the same time, existing technologies show excellent performance in semi-supervised cross-session emotion recognition by combining label commonalities with specific feature exploration and graph regularization. And individual differences and noisy annotations in EEG emotion data are addressed through prototype learning and pairwise learning. These methods often match the overall features while ignoring the local alignment across channels, resulting in poor transfer effects of fine-grained features. Traditional domain adaptation methods usually match based on vectorized features, but EEG signals inherently have spatial topological relationships, so their spatial information cannot be fully utilized. Finally, existing transfer learning methods mainly focus on the overall alignment of features while ignoring the individual differences between the source domain and the target domain, resulting in the loss of personalized features and reducing the adaptability of emotion recognition. Therefore, although existing methods have alleviated the challenges brought by different domain differences to a certain extent, how to efficiently unify the source domain and the target domain to further improve the model generalization ability remains a core problem to be solved urgently. Summary of the Invention

[0006] The purpose of the present invention is to address the above deficiencies in the prior art and provide a cross-session-subject emotion recognition method that combines feature decomposition and transfer learning to solve the problem that it is difficult for the prior art to efficiently unify the source domain and the target domain, and thus difficult to improve the model generalization ability.

[0007] To achieve the above objective, the technical solution adopted by the present invention is:

[0008] A cross-session-subject emotion recognition method that combines feature decomposition and transfer learning, comprising the following steps:

[0009] S1. Preprocess the original EEG signal and extract EEG signal features;

[0010] S2. Decompose the EEG signal features to obtain brain common features and individual private features between the source domain and the target domain;

[0011] S3. Perform cross-domain fine-grained alignment and intra-domain optimization processing on the brain common features between the source domain and the target domain;

[0012] S4. Decode the brain common features of the source domain to achieve emotion recognition and classification;

[0013] S5. Construct a joint optimization objective function based on cross - domain fine - grained alignment loss, intra - domain optimization loss, and balanced classification loss to optimize model training and emotion recognition classification.

[0014] Furthermore, in S1, the EEG signal features are extracted as:

[0015]

[0016] where is the EEG signal feature; represents the pre - processed EEG signal of the \(i\) - th subject and the \(j\) - th sample with \(C\) channels and \(T\) time sampling points; \(B\) represents the frequency band; \(DE\) represents differential entropy.

[0017] Furthermore, in S2, a co - private graph separation module is used to decompose the EEG signal features, which specifically includes the following sub - steps:

[0018] S21. Adaptive adjacency matrix generation: Input the EEG signal features into a one - dimensional convolutional layer respectively for feature mapping to obtain the query value and the key value

[0019] S22. Cognition - driven graph learning: Calculate the similarity between features based on the query value and the key value ;

[0020] S23. Introduce a graph learning operator and decompose to obtain the brain common features and individual private features between the source domain and the target domain based on the similarity between features.

[0021] Furthermore, in S21, inputting the EEG signal features into a one - dimensional convolutional layer respectively for feature mapping includes:

[0022]

[0023] where \(m\) and \(n\) are the number of subjects and the number of samples of the subjects respectively; \(f1\) and \(f2\) are one - dimensional convolutional layers respectively.

[0024] Furthermore, in S22, calculating the similarity between features based on the query value and the key value includes:

[0025]

[0026] where is the source - domain data \(X\) s and the target - domain data \(X\) t generate the reference of the cognition - driven structure, representing the cognition - driven structure and The construction method; Softmax is the activation function; d is the dimension of the feature.

[0027] Further, in S23, the brain common features and individual private features between the source domain and the target domain are decomposed, including:

[0028]

[0029] In the formula, and are the brain common features between the source domain and the target domain; and are the individual private features between the source domain and the target domain; K is the number of layers of graph convolution adopted; k is the k-th layer of graph convolution; are the shared parameters of the common graph, the specific parameters of the source domain private graph, and the specific parameters of the target domain private graph respectively; T g is the graph learning operator, representing graph convolution; X s , X t are the source domain data and the target domain data respectively.

[0030] Further, in S3, the Earth Mover's Distance is used to measure the fine-grained difference between the source domain and the target domain, and its expression is:

[0031]

[0032] In the formula, L F is the cross-domain fine-grained alignment loss; j and j' represent the source domain sample and the target domain sample respectively; and are the iterative multipliers used to standardize the feature distributions on the c-th channel of the source domain and the target domain; is the brain common feature of the source domain sample and the brain common feature of the target domain sample is the Euclidean distance.

[0033] Further, in S3, the Margin function is used to perform in-domain optimization on the brain common features between the source domain and the target domain, and its expression is:

[0034]

[0035] In the formula, L M is the in-domain optimization loss; n is the number of samples used during training; η is the threshold.

[0036] Further, in S5, the joint optimization objective function is:

[0037] L = λ1L C +λ2L F +λ3L M

[0038] In the formula, L is the total loss; λ1, λ2, and λ3 are weight coefficients; L C is the balanced classification loss.

[0039] Furthermore, the balanced classification loss L C is:

[0040]

[0041] In the formula, p n (y K ) and respectively represent the predicted label and the true label of the K-th type of emotion.

[0042] The cross-session-subject emotion recognition method combining feature decomposition and transfer learning provided by the present invention has the following beneficial effects:

[0043] The present invention embeds a common-private graph separation module to optimize the accuracy of emotion decoding, aiming to dynamically extract the spatial brain common features and differential features between the source domain and the target domain so as to capture the spatial dependence characteristics, and realize cross-domain adaptation while retaining individual characteristics. On the other hand, the proposed cross-domain fine-grained alignment and intra-domain optimization strategy aims to better calibrate the feature distributions of the source domain and the target domain and accurately highlight the brain map patterns highly related to emotions. Based on the proposed framework, this method can achieve the comprehensive alignment of the feature space distribution and the effective optimization of the intra-domain structure, thus realizing the efficient decoding of the emotion recognition mode.

[0044] Through the common-private feature decomposition, cross-domain fine-grained alignment and intra-domain optimization strategy, the present invention can effectively improve the generalization ability and classification accuracy of EEG emotion recognition; experiments show that the present invention has achieved significant improvements in both cross-session and cross-subject recognition tasks on the SEED-IV and FACED data sets, proving the effectiveness of this method in dealing with individual differences and cross-domain adaptation. Description of the Drawings

[0045] Figure 1 is the emotion EEG feature transfer learning process between the source domain and the target domain.

[0046] Figure 2 is the entire learning framework of a cross-session-subject emotion recognition method (CPD-GTO) combining feature decomposition and transfer learning of the present invention; the present invention mainly consists of two parts, namely a common-private graph separation module and a cross-domain fine-grained alignment and intra-domain optimization strategy.

[0047] Figure 3 is the ablation experiment result of a cross-session experiment of a cross-session-subject emotion recognition method (CPD-GTO) combining feature decomposition and transfer learning of the present invention on the SEED_IV data.

[0048] Figure 4 This is the visualization result of the brain network connection for the cross-session experiment of the cross-session and subject emotion recognition method (CPD-GTO) combining feature decomposition and transfer learning in the SEED_IV dataset of the present invention.

[0049] Figure 5 This is the visualization result of the feature distribution for the cross-subject experiment of the cross-session and subject emotion recognition method (CPD-GTO) combining feature decomposition and transfer learning in the FACED dataset of the present invention.

[0050] Figure 6 This is the flowchart of the cross-session and subject emotion recognition method combining feature decomposition and transfer learning of the present invention. Detailed implementation manners

[0051] The following describes the detailed implementation manners of the present invention for the understanding of those skilled in the art of the present technology. However, it should be clear that the present invention is not limited to the scope of the detailed implementation manners. For those of ordinary skill in the art of the present technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions created using the concept of the present invention are within the scope of protection.

[0052] Example 1

[0053] The cross-session and subject emotion recognition method combining feature decomposition and transfer learning in this example extracts spatial brain common features and individual difference features through a co-private graph separation module, and combines cross-domain fine-grained alignment and in-domain optimization strategies to improve the consistency of feature distribution and the accuracy of emotion decoding. It can solve the problem of inter-domain distribution shift caused by individual differences and improve the generalization ability of emotion features. The specific content is as follows:

[0054] S1. Preprocess the original EEG signal and extract EEG signal features;

[0055] Specifically, after obtaining the original EEG signal, first preprocess it, including operations such as band-pass filtering and artifact removal, to remove noise and extract effective information, and obtain a specific dataset S with m subjects:

[0056]

[0057] In the formula, is a 2D matrix representing the preprocessed EEG signal data of the i-th subject and the j-th sample with C channels and T time sampling points. n represents the number of samples, represents the corresponding true label.

[0058] After completing the EEG signal preprocessing, the differential entropy (DE) features are used as the input features of the EEG data, where the differential entropy is expressed as:

[0059]

[0060] In the formula, σ represents the standard deviation of the signal in a specific frequency band. The EEG features related to emotions are captured in five standard frequency bands: δ (1 - 4 Hz), θ (4 - 8 Hz), α (8 - 14 Hz), β (14 - 30 Hz), and γ (30 - 45 Hz), to improve the robustness of emotion classification.

[0061] Based on this, the obtained differential entropy features are denoted as:

[0062]

[0063] In the formula, are the EEG signal features; B represents the frequency band.

[0064] S2. Decompose the EEG signal features to obtain the brain common features and individual private features between the source domain and the target domain;

[0065] Specifically, in this embodiment, a co - private graph separation module is used to decompose the EEG signal features. The topological structure of the brain is crucial for emotion recognition. Usually, Phase Locking Value (PLV) and Pearson's Correlation Coefficient (PCC) etc. are used to construct the brain network, but the connections they obtain are static through prior knowledge. However, the brain topological structure reflected by static functional connections is relatively fixed, which may make it difficult to distinguish some implicit features with emotion discriminability. Therefore, the core goal of this module is to construct a cognition - driven topological structure to extract the brain common features and individual private features between the source domain and the target domain, which specifically includes the following sub - steps:

[0066] S21. Adaptive adjacency matrix generation: Generate a cognition - driven structure through convolution operations to describe the connection relationship between different feature channels; specifically, the EEG signal features are respectively input into one - dimensional convolutional layers f1 and f2 for feature mapping, including:

[0067]

[0068] In the formula, and are respectively the query value and the key value calculated according to the source domain data X s and the target domain data X t ; m and n are the number of subjects and the number of samples of the subjects respectively; f1 and f2 are one - dimensional convolutional layers.

[0069] S22. Cognitive-driven graph learning: Further obtain the similarity between features, thereby generating a cognitive-driven structure; specifically, based on and calculate the similarity between features, including:

[0070]

[0071] In the formula, is the source domain data X s and the target domain data X t The reference for generating the cognitive-driven structure, indicating the cognitive-driven structure and The construction method; Softmax is the activation function, which normalizes the vector; d is the dimension of the feature (i.e., the number of channels of the convolutional output, 62).

[0072] S23. Introduce graph learning operators, aiming to cover local and global spatial structure information to effectively capture the spatial dependence relationship of brain representations, thereby further extracting high-order spatial features in the data. To improve the computational efficiency of the model, the Chebyshev polynomial approximation method is used to stack cognitive-driven graph information of different orders, further enhancing the feature expression ability of the model; based on this, the brain common features and individual private features between the source domain and the target domain are decomposed:

[0073]

[0074] In the formula, and are the brain common features between the source domain and the target domain; and are the individual private features between the source domain and the target domain; K is the number of layers of graph convolution adopted, with a value of 2; k is the kth layer of graph convolution; are the shared parameters of the common graph, the specific parameters of the source domain private graph, and the specific parameters of the target domain private graph respectively; T g is the graph learning operator, representing graph convolution; X s , X t are the source domain data and the target domain data respectively.

[0075] S3. Perform cross-domain fine-grained alignment and intra-domain optimization processing on the brain common features between the source domain and the target domain;

[0076] For cross-domain fine-grained alignment;

[0077] The fine-grained alignment strategy in this embodiment aims to fully adapt the channel information by calculating the feature distribution difference between the source domain and the target domain channel by channel.

[0078] This embodiment introduces the Earth Mover's Distance (EMD) to measure the fine-grained difference between the source domain and the target domain, so as to independently optimize the feature distribution on each channel. Its expression is:

[0079]

[0080] In the formula, L F is the cross-domain fine-grained alignment loss; j and j' represent the source domain sample and the target domain sample respectively; is the brain common feature of the source domain sample and the brain common feature of the target domain sample 's Euclidean distance;

[0081] and are the iterative multipliers used to standardize the feature distributions on the c-th channels of the source domain and the target domain; the multipliers are updated iteratively to try the pairing between each source domain sample and target domain sample, so that the cost is gradually reduced. Further, this embodiment uses the Sinkhorn-Knopp algorithm to update the current multiplier according to the distance between the source domain and the target domain in each iteration, so as to achieve that the distributions of each channel can be independently aligned. The update process is as follows:

[0082]

[0083] In the formula, ε represents the adjustment factor, which is used to grasp the smoothness of the iteration; Z1 and Z2 are used to ensure and Through multiple iterations, the distributions of the source domain and the target domain are gradually aligned, and then the EMD loss between them is minimized. In this way, the model can gradually reduce the distribution difference between the source domain and the target domain, so as to achieve a more refined alignment.

[0084] Intra-domain optimization

[0085] This step is used to strengthen the distinguishability between the brain common feature and the private feature, and uses the Margin function to maximize the interval between the common feature and the private feature. Essentially, if the distance is less than a certain threshold η, there will be a penalty to force the network to separate them. On the contrary, if the distance is already large enough (greater than η), the optimal goal is achieved, so that the network will not continue to punish. Its specific expression is:

[0086]

[0087] In the formula, L M is the in-domain optimization loss; n is the number of samples used during training (or the number of samples in a batch); η is the threshold.

[0088] S4. Decode the brain common features of the source domain, and then realize emotion recognition and classification;

[0089] Specifically, in this embodiment, the brain common features from the source domain are decoded through a fully connected layer and Softmax to obtain an emotion recognition pattern; and the balanced classification loss L C is:

[0090]

[0091] In the formula, p n (y K ) and respectively represent the predicted label and the true label of the K-th type of emotion.

[0092] S5. Train and optimize the model;

[0093] Construct a joint optimization objective function based on the cross-domain fine-grained alignment loss, the in-domain optimization loss, and the balanced classification loss to optimize the model training and emotion recognition and classification;

[0094] Among them, the joint optimization objective function is:

[0095] L = λ1L C + λ2L F + λ3L M

[0096] In the formula, L is the total loss; λ1, λ2, and λ3 are weight coefficients, which are set to 0.6, 0.2, and 0.2 respectively, for balancing the classification loss, the cross-domain fine-grained alignment loss, and the in-domain optimization loss; L C is the balanced classification loss. The model is trained through the Adam optimizer to achieve the optimal solution for cross-domain emotion recognition.

[0097] Embodiment 2

[0098] To verify the effect of the present invention, a specific case is used for illustration, which specifically includes the following content:

[0099] 1. Dataset;

[0100] The FACED dataset is a fine-grained emotion category affective computing EEG dataset released by the research group of Zhang Dan from the Department of Psychology, Tsinghua University. A total of 123 subjects were recruited in this dataset (75 females, average age = 23.2, ranging from 17 to 38 years old). This dataset includes 32-channel electroencephalogram data and self-reported emotion scores of the subjects when watching 28 emotion-evoking video clips, which are targeted at nine emotions (anger, fear, disgust, sadness, amusement, inspiration, joy, tenderness, and neutral emotion), including four negative emotions, four positive emotions, and three neutral emotions, so it can be further divided into three classifications. DE features are calculated from 5 frequency bands, following the practice of the SEED dataset. In this embodiment, the first 30 subjects are selected for evaluating the proposed method.

[0101] The SEED-Ⅳ dataset is an emotion EEG dataset of four emotions from Shanghai Jiao Tong University. This dataset includes 15 subjects, and each subject watches 24 video clips and records EEG data. These video clips are divided into 4 emotions, namely happiness, sadness, fear, and neutral. The experiment records 62-channel EEG data generated by each subject and watches the video clips for about 2 minutes. First, the EEG signal is downsampled to 200hz, and the signal is filtered into 5 frequency bands, namely (1 - 4hz), (4 - 8hz), (8 - 14hz), (14 - 31hz), and (31 - 50hz). Then, DE features are extracted with a 4-second time window, non-overlapping in 5 frequency bands.

[0102] 2. Experimental Setup

[0103] For the SEED_IV dataset, in this embodiment, a leave-one-session cross-validation protocol is adopted. The first and second sessions of each subject are used for training, and the third session is used for testing. In addition, for the FACED dataset, a leave-one-subject cross-validation protocol is adopted. Each time, the data of 29 subjects are selected for training, and the data of one subject are left for testing, and the validation process is repeated.

[0104] Both datasets use differential entropy (DE) features as input. The model in this embodiment is trained by the Adam optimizer with learning rates of 8e-5 and 8e-3 respectively. The batch size is 128, and the regularization parameter is set to 0.001. And the proposed method is compared with 7 state-of-the-art models to verify the superiority of the present invention.

[0105] These seven state-of-the-art models are respectively:

[0106] DGCNN: Dynamic Graph Convolutional Neural Network, a static graph baseline. By dynamically learning the internal relationships between different electroencephalogram (EEG) channels represented by the adjacency matrix, more discriminative EEG features are extracted.

[0107] GMSS: Graph-based Multi-task Self-Supervised Learning, which integrates multiple self-supervised tasks to learn more general representations, including spatial and frequency jigsaw tasks and contrastive learning tasks.

[0108] SimSiam: A contrastive learning algorithm based on Siamese networks.

[0109] MS-FRAN: Multi-Source Feature Representation and Alignment Network, which reduces the distribution differences between multiple source domains by aligning the distributions of each pair of source and target domains.

[0110] MS-MDA: Multi-Source Marginal Distribution Adaptation Network, which adopts one-to-one domain adaptation and extracts domain-specific features.

[0111] MDNet: Multi-Layer Disentanglement Network, which combines a modality-level disentanglement module and an individual-level disentanglement module to extract modality-invariant features and modality-specific features respectively, thus simultaneously modeling the consistency and heterogeneity of multi-modal signals.

[0112] HDGCN: Hierarchical Dynamic Graph Convolutional Network with interpretability, which uses the discriminative features of EEG signals as auxiliary information to explore the dynamic multi-level spatial information between EEG signal channels.

[0113] To verify the effectiveness of the model of the present invention, cross-session and cross-subject experiments were respectively carried out on the SEED_IV dataset and the FACED dataset, and compared with 7 state-of-the-art methods, as shown in Table 1.

[0114] Table 1

[0115]

[0116]

[0117] Table 1 Cross-session experiments on the SEED_IV dataset and cross-subject experiments on the FACED dataset, comparison of different methods in terms of accuracy and f1 score (in %), the best results are shown in bold in Table 1.

[0118] In comparison with other methods, the performance of the present invention is particularly prominent. Specifically, the performances of DGCNN and DANN are relatively weak, with accuracies of only 52.82% and 47.59% respectively, which are 19.61% and 24.84% lower than that of the present invention. This may be due to the deficiencies of DGCNN and DANN in fine-grained feature adaptation and cross-domain difference modeling. Although GMSS and SimSiam adopt self-supervised and contrastive learning strategies, they fail to effectively combine the domain characteristics of EEG data, and their accuracies are 6.3% and 21.19% lower than that of the present invention respectively. Other methods, such as CFDA-CSF and MS-FRAN, although they contribute to multimodal adaptation, still perform worse than the present invention in single-modal EEG. The accuracies of CFDA-CSF and MS-FRAN are 1.57% and 2.78% lower than that of the present invention respectively. Moreover, the standard deviation of the results of the proposed CFDA-CSF is also smaller, indicating better stability in different experiments. Compared with other methods, the present invention can more effectively capture the common brain features and differential features, and align the features from a finer-grained perspective, thus improving the accuracy and consistency of the cross-domain emotion recognition task, and fully verifying its advantages in dealing with individual differences and cross-domain adaptation.

[0119] To verify the effectiveness of the module of the present invention, an ablation study was conducted here using the SEED_IV dataset as an example.

[0120] To verify the effectiveness of the generated cognitive-driven structure, an ablation experiment was conducted by replacing this module with the fixed connection PLV calculated through prior knowledge, as shown in Figure 3 (a). The experimental results show that the classification performance has been significantly improved by the adaptively generated cognitive-driven structure. Specifically, although the static adjacency matrix based on PLV can reflect some basic connection relationships between brain regions, it lacks the dynamic adaptation ability to emotional-related brain activities. In contrast, the adaptively generated cognitive-driven structure can dynamically adjust the connection weights between brain regions according to the characteristics of emotional tasks, so as to better capture the topological structure changes of the brain in emotional states.

[0121] In the multi-source domain emotion recognition task, effectively extracting the general features across subjects and sessions while retaining the private differences of each individual is a key challenge. Figure 3The experimental results of (a) show that the separation of common features and individual private features has a non-negligible impact on emotion recognition. Without feature separation, the performance of the model significantly decreases in both accuracy and F1-score. Specifically, capturing multi-domain brain common features enables cross-domain adaptation in different emotion recognition tasks. At the same time, multi-domain brain private features are responsible for retaining the unique differences of each domain, helping the model better handle the differences in emotion expression across different domains. Thus, the construction of this module plays a crucial role in dealing with complex emotion recognition tasks.

[0122] To verify the effectiveness of the optimization strategy proposed in the present invention, the contribution of each optimization function to the model performance was evaluated one by one.

[0123] As Figure 3 shown in (b), it can be seen from the figure that each optimization function plays a crucial role in improving the model performance. Among them, cross-domain fine-grained alignment makes the most significant contribution to the improvement of the model performance, indicating that accurately aligning the source domain and target domain features is extremely helpful for improving the emotion recognition effect. In addition, intra-domain optimization also plays an important role in the model, because optimizing intra-domain features can better capture the common features across individuals while retaining the unique emotion patterns of each domain to avoid the interference of differences.

[0124] Brain connectivity visualization:

[0125] To explore the spatial dependence of the EEG channels captured by the present invention, the SEED-IV dataset is taken as an example here. The learned channel connection relationships are visualized, and the top 10% of the partial dependence relationships are shown to improve readability. From Figure 4It can be seen that the central region nodes (such as FCZ, CZ, CPZ, PZ) are core nodes in all emotions, highlighting their key role in emotion processing. At the same time, the frontal lobe nodes (such as FZ), as another highly active region, further demonstrate their important position in the brain functional network. In addition, the connections between the central region and the parietal lobe (such as FCZ-PZ and CPZ-PZ) are generally present in various emotions, which may be the basic connection structure relied on for emotion processing. Among different emotions, the connection characteristics show significant differences. For example, in the happy state, the frontal lobe anterior part (such as FP1, FP2, AF3, AF4) is particularly active, showing a globally distributed and dense connection and a significantly enhanced cross-hemisphere collaboration, which reflects the high collaboration of the whole brain. On the contrary, in the sad state, the central region (such as CZ, FCZ) dominates, which is manifested as the connection being concentrated in the central region and less cross-hemisphere collaboration, and the connection pattern being more localized. At the same time, in the fear state, the prefrontal cortex (such as FP1, F2) and the temporal lobe region (such as T8) show enhanced connectivity because these regions are involved in emotion regulation and threat assessment. It shows that the fear emotion is related to danger perception and physical reactions, and the connection pattern in the fear state shows strong signal transmission between the prefrontal lobe and the temporal lobe, reflecting a high level of alertness and rapid response to potential threats. In contrast, in the neutral state, the activity of all channels is generally balanced and stable, showing a uniform network structure distribution and cross-hemisphere collaboration that is neither highly concentrated nor overly dispersed, which reflects the basic activity level of the brain in this emotional state. Overall, the brain networks under different emotions show high heterogeneity in key regions, connection distribution, and collaboration patterns, which provides an important reference for the research on emotion-related brain functional networks.

[0126] Feature distribution visualization:

[0127] To verify the change in feature distribution before and after model learning, the FACED dataset is taken as an example here. The T-SNE technique is used to visualize the original features, brain common features, and private features, as Figure 5As shown, it can be seen that the proposed strategy plays a significant role in feature decomposition and alignment. Specifically, the initial features in (a) show obvious aliasing of samples in different categories in the feature space and are difficult to distinguish; the common features in (b) show better category aggregation effects, and samples of the same category are more compact in the feature space, indicating that this module has significant effects in extracting domain-invariant brain common features; the private features in (c) are more dispersed, reflecting the differential features of each domain. This is consistent with the goal of module design, that is, to extract common features and private features among multiple domains through feature decomposition. This result shows that the proposed strategy can effectively capture domain-invariant common features, thereby enhancing the distinguishability of categories. At the same time, it can retain the private features of each domain, further supporting its ability to model multi-domain differences in the emotion recognition task. This visualization verifies the effectiveness of the module in feature decomposition and provides theoretical support for improving the performance of cross-domain emotion recognition.

[0128] The development of the co-private map separation module of the present invention realizes the extraction of spatial brain common features and differential features among multiple domains, thereby capturing spatial dependence characteristics. The proposed cross-domain fine-grained alignment and intra-domain optimization strategy enable the model to better calibrate the multi-domain feature distribution and accurately highlight the brain map patterns highly related to emotions, thereby realizing efficient individual difference decoding. The experimental results prove the impressive decoding performance of the present invention across sessions and across subjects, and highlight its effectiveness in cross-domain emotion recognition. To further explore the contribution of the constructed module, ablation studies provide the necessity of each module in the present invention. In addition, the interpretation of the brain functional connections and feature distributions learned by the present invention reveals patterns highly related to emotions, enhances the generalization ability among subjects, and promotes the in-depth development of the learning of interaction patterns between individuals.

[0129] Although the specific implementation manners of the invention have been described in detail with reference to the accompanying drawings, it should not be construed as a limitation on the protection scope of this patent. Within the scope described in the claims, various modifications and deformations that can be made by those skilled in the art without creative efforts still fall within the protection scope of this patent.

Claims

1. A cross-session and subject emotion recognition method combining feature decomposition and transfer learning, characterized in that Including the following steps: S1. Preprocess the original EEG signal and extract EEG signal features; S2. Decompose the EEG signal features to obtain the brain common features and individual private features between the source domain and the target domain; S3. Perform cross-domain fine-grained alignment and intra-domain optimization on the brain common features between the source domain and the target domain; S4. Decode the brain common features of the source domain to achieve emotion recognition and classification; S5. Construct a joint optimization objective function based on the cross-domain fine-grained alignment loss, intra-domain optimization loss, and balanced classification loss to optimize model training and emotion recognition and classification.

2. The cross-session-subject emotion recognition method combining feature decomposition and transfer learning according to claim 1, characterized in that In S1, the extracted EEG signal features are: wherein, is the EEG signal feature; represents the preprocessed EEG signal of the j-th sample of the i-th subject with C channels and T time sampling points; B represents the frequency band; DE represents the differential entropy.

3. The cross-session and subject emotion recognition method combining feature decomposition and transfer learning according to claim 1, characterized in that In S2, a co-private graph separation module is used to decompose the EEG signal features, and specifically including the following sub-steps: S21. Adaptive adjacency matrix generation: The EEG signal features are respectively input into a one-dimensional convolutional layer for feature mapping to obtain query values and key values S22. Cognitive drive graph learning: Based on the query value and the key value calculate the similarity between features; S23. Introduce a graph learning operator and decompose to obtain the brain common features and individual private features between the source domain and the target domain based on the similarity between features.

4. The cross-session and subject emotion recognition method combining feature decomposition and transfer learning according to claim 3, characterized in that In S21, the EEG signal features are respectively input into a one-dimensional convolutional layer for feature mapping, including: In the formula, m and n are respectively the number of subjects and the number of samples of the subjects; f1 and f2 are respectively one-dimensional convolutional layers.

5. The cross-session and subject emotion recognition method combining feature decomposition and transfer learning according to claim 4, characterized in that In S22, based on the query value and the key value calculate the similarity between features, including: In the formula, is the source domain data X s and the target domain data X t generate the reference of the cognitive drive structure, representing the cognitive drive structure and the construction method; Softmax is the activation function; d is the dimension of the feature.

6. The cross-session-subject emotion recognition method combining feature decomposition and transfer learning according to claim 5, characterized in that In S23, the decomposition to obtain the brain common features and individual private features between the source domain and the target domain includes: Wherein, and are the brain common features between the source domain and the target domain; and are the individual private features between the source domain and the target domain; K is the number of layers of graph convolution adopted; k is the k-th layer of graph convolution; are the shared parameters of the common graph, the specific parameters of the source domain private graph, and the specific parameters of the target domain private graph respectively; T g is the graph learning operator, representing graph convolution; X s , X t are the source domain data and the target domain data respectively.

7. The cross-session and subject emotion recognition method combining feature decomposition and transfer learning according to claim 1, characterized in that In S3, the Earth Mover's Distance is used to measure the fine-grained difference between the source domain and the target domain, and its expression is: where L F is the cross - domain fine - grained alignment loss; j and j' represent source - domain samples and target - domain samples respectively; and are iterative multipliers used to regularize the feature distributions on the c - th channel of the source domain and the target domain; is the brain common feature of the source - domain sample and is the Euclidean distance of the brain common feature of the target - domain sample.

8. The cross-session and subject emotion recognition method combining feature decomposition and transfer learning according to claim 7, characterized in that In S3, a Margin function is used to perform intra-domain optimization on the brain common features between the source domain and the target domain, and its expression is: where L M is the in-domain optimization loss; n is the number of samples used during training; and η is the threshold value.

9. The cross-session-subject emotion recognition method combining feature decomposition and transfer learning according to claim 8, characterized in that In S5, the joint optimization objective function is: L = λ1L C + λ2L F + λ3L M where \(L\) is the total loss; \(\lambda_1\), \(\lambda_2\) and \(\lambda_3\) are weight coefficients; \(L\) C is the balanced classification loss.

10. The cross-session-subject emotion recognition method combining feature decomposition and transfer learning according to claim 9, characterized in that The balanced classification loss L C is as follows: where p n (y K ) and respectively represent the predicted label and the true label of the K-th type of emotion.

Images