Cross-subject electroencephalogram emotion recognition method and system based on spatio-temporal adaptive graph coding learning

Abstract

The application belongs to the field of deep learning, and provides a cross-subject electroencephalogram emotion recognition method and system based on spatiotemporal adaptive graph coding learning, comprising: extracting features from different frequency bands by using a sliding window technology to construct a feature matrix covering channel and time dimensions; fusing spatiotemporal hybrid embedding, time embedding and space embedding to convert the feature matrix into a high-dimensional semantic vector; calculating channel feature differences and time trend changes to generate a dynamic topology matrix that adapts to cross-subject individual differences; combining the dynamic topology matrix to calculate spatial cross-channel and time cross-step attention in parallel, fusing and strengthening key spatiotemporal features; and a decoder mapping the encoded features by means of a domain adversarial decoding module, reducing the distribution difference between the source domain and the target domain through cross-domain adversarial training, and outputting an emotion classification result. The above method improves the cross-subject generalization ability and recognition accuracy of electroencephalogram signal emotion recognition.

Description

Technical Field

[0001] This invention belongs to the field of deep learning technology, specifically relating to a spatiotemporal adaptive graph coding learning method and system for cross-subject EEG emotion recognition. Background Technology

[0002] With the deep integration of affective computing and artificial intelligence, emotion recognition based on electroencephalogram (EEG) signals has become a research hotspot in fields such as human-computer interaction and mental health monitoring because it can directly reflect brain neural activity. EEG signals contain rich spatiotemporal features, and their emotional association patterns (such as negative, neutral, and positive) can be automatically identified through feature extraction and modeling. However, the universality across subject scenarios remains a core challenge—individual physiological differences (such as brain structure and neural response intensity) and interference from the experimental environment often lead to a significant decline in the model's recognition performance on new subject data.

[0003] Traditional methods often rely on manually extracting time-domain and frequency-domain features (such as differential entropy and power spectral density) and combining them with shallow models like support vector machines for classification. This approach suffers from limited feature representation capabilities and sensitivity to individual differences. While deep learning has improved performance through models like convolutional neural networks (CNNs), recurrent neural networks (RNNs / LSTMs), and Transformers, it still faces many limitations. For example, most methods based on graph neural networks (GNNs) employ predefined EEG channel topologies (such as adjacency matrices based on electrode physical locations). For instance, the Chinese invention patent CN119279611A, "Emotion Recognition Method Based on EEG Spatiotemporal-Frequency Feature Fusion," fully considers the spatiotemporal, temporal, and frequency-based features of EEG signals and outputs emotion classification results by fusing the extracted emotion features. However, this method struggles to dynamically adapt to the differences in EEG signal distribution among different subjects, leading to a mismatch between the topology and the actual spatiotemporal correlation, thus limiting cross-subject generalization ability.

[0004] Furthermore, traditional self-attention mechanisms, when processing EEG signals, do not separate and optimize for their spatiotemporal heterogeneity. Simultaneously processing the correlation between spatial channels and time steps makes them susceptible to interference from redundant information, making it difficult to focus on key emotion-related features (such as channel coordination patterns in specific frequency bands). Existing models are mostly trained on single or mixed subject data, without considering the data distribution shift between the source domain (training subjects) and the target domain (new subjects), leading to a sharp drop in performance due to inter-domain differences in cross-subject scenarios. In addition, temporal embeddings often rely on simple location encoding, failing to fully integrate the multi-scale temporal characteristics of EEG signals (such as instantaneous reactions and long-term trends), and spatial embeddings do not effectively combine dynamic functional connectivity information, making it difficult to capture brain region coordination patterns in emotional states. Summary of the Invention

[0005] The technical problem to be solved by this invention is how to improve the cross-subject generalization ability and recognition accuracy of EEG signal emotion recognition.

[0006] The present invention solves the above-mentioned technical problems through the following technical means:

[0007] This invention provides a cross-subject EEG emotion recognition method based on spatiotemporal adaptive graph coding learning, comprising the following steps:

[0008] S1. The preprocessed and feature-extracted EEG data samples are divided using the sliding window technique, and features are then extracted from different frequency bands to construct a feature matrix covering channel and time dimensions.

[0009] S2. The feature matrix is ​​transformed into a high-dimensional semantic vector by using a feature preprocessing and mapper through a multi-dimensional data embedding module, which integrates spatiotemporal hybrid embedding, temporal embedding extracted by BiLSTM and dilated convolution.

[0010] S3. Based on the high-dimensional semantic vector obtained in step S2, the adaptive spatiotemporal topology construction module of the graph structure generator calculates the channel feature differences and time trend changes to generate a dynamic topology matrix that adapts to the differences across individual subjects.

[0011] S4. The encoder is based on a dynamic spatiotemporal hybrid attention coding module. It combines the dynamic topology matrix to perform parallel computation of spatial cross-channel and temporal cross-step attention, and integrates and strengthens key spatiotemporal features through a gating mechanism.

[0012] S5. The decoder uses the domain adversarial decoding module to map the encoded features, and reduces the distribution difference between the source domain and the target domain through cross-domain adversarial training, and outputs the sentiment classification result.

[0013] Further, step S1 includes the following steps:

[0014] S11. The preprocessed EEG feature data is divided into samples using the sliding window technique. The number of samples is increased by overlapping windows. Each window contains several consecutive 1-second time point data. Features of 5 frequency bands are extracted from the data of each window after division. The 5 frequency bands include delta, theta, alpha, beta, and gamma frequency bands, which correspond to the core dimensions of differential entropy features.

[0015] S12. Organize the extracted multi-band features into a three-dimensional feature matrix, where the 0th dimension is 62 EEG channels, the 1st dimension is the number of time points included in the window, and the 2nd dimension is 5 frequency bands, forming a feature matrix covering both channel and time dimensions.

[0016] Furthermore, step S2 includes the following steps:

[0017] S21, Based on historical data coding features and current data encoding characteristics By fusing information from both sources through a gating mechanism, a spatiotemporal hybrid embedding is obtained. As shown in the following formula:

[0018]

[0019] in , For learnable parameter matrix, a , b For bias terms, The tanh activation function is used. It is the sigmoid activation function. For Hadama accumulation, ,in N For the number of channels, T For time points, D For the hidden layer dimension;

[0020] S22. BiLSTM is used to model the temporal dimension of the feature matrix to capture bidirectional temporal dependencies, with the output dimension being... Temporal features are extracted using dilated convolutional layers with a dilation rate of 2. The receptive field increases exponentially with the number of layers, and the output dimension is... The outputs of both are concatenated and linearly transformed, then combined with sine and cosine position coding to obtain the time embedding. ,and ;

[0021] S23. Constructing an adjacency matrix based on the EEG channel topology. A Calculate the normalized graph Laplacian matrix as follows:

[0022]

[0023] in, I It is the identity matrix. D For degree matrix, It is the eigenvalue matrix;

[0024] Take its eigenvector As spatial embedding ;

[0025] S24. Embedding spatiotemporal mixing Time embedding Spatial embedding Element-wise addition yields a high-dimensional semantic vector. As shown in the following formula:

[0026]

[0027] in, , B For batch size.

[0028] Furthermore, step S3 includes the following steps:

[0029] S31, Regarding high-dimensional semantic vectors Joint modeling of nodes in spatial and temporal dimensions is performed, and the high-dimensional semantic vector containing 62 channels and T time points is flattened into a spatiotemporal node sequence composed of channel-time pairs. Each node uniquely corresponds to the feature of a specific channel at a specific time step.

[0030] S32. In the spatial dimension, for each time step, the feature associations between 62 channels are calculated using cosine similarity to generate a channel association matrix.

[0031] S33. In the time dimension, for each channel, the trend correlation over T time steps is calculated using the Pearson correlation coefficient to generate a time correlation matrix. ;

[0032] S34. The two matrices are fused using the Kronecker product to generate the initial global topology matrix. The matrix has dimensions of The diagonal submatrix represents the channel association at the same time point, while the off-diagonal submatrix represents the channel interaction across time points, thus realizing the explicit fusion of spatiotemporal associations.

[0033] S35. Introduce a learnable adjacency matrix. The feature projection layer maps node features to a dimension matching the number of nodes, and matrix multiplication is used to transform the initial global topology matrix. The result is fused with the feature projection result and then normalized using the Softmax function to obtain the dynamic topology matrix. ;

[0034] Furthermore, step S4 includes the following steps:

[0035] S41, Based on dynamic topology matrix Construct a global spatiotemporal graph to transform high-dimensional semantic vectors into a global association matrix containing channel and time dimensions. The diagonal submatrix represents the spatial association between all channels at the same time point, while the off-diagonal submatrix represents the cross-temporal association between all channels at different time points.

[0036] S42. Calculate the spatial cross-channel attention score. Attention score and time stride ;

[0037] S43. Obtaining the fusion features of two types of attention through gating mechanisms. As shown in the following formula:

[0038]

[0039]

[0040] in For gating parameters, For splicing operations;

[0041] S44, Integrating Features Multi-subspace feature aggregation is performed using a multi-head attention module, as shown in the following equation:

[0042]

[0043] in, For projection parameters, Let h be the feature representation learned by the h-th attention head in its dedicated subspace;

[0044] S45. Enhance feature stability through residual connections and layer normalization, as shown in the following formula:

[0045]

[0046]

[0047] S46. By reducing the sequence length and retaining key features through a distillation mechanism, the key spatiotemporal features are enhanced to obtain the encoded features, as shown in the following formula:

[0048]

[0049] in, For max pooling operations, ELU stands for exponential linear unit. This is a one-dimensional convolution operation.

[0050] Further, step S42 involves calculating the spatial cross-channel attention score. The specific process is as follows:

[0051] Through dynamic topology matrix Generate the space matrix Q, K, V as follows:

[0052]

[0053]

[0054]

[0055] in For dynamic topology matrix, As learnable parameters, the inter-channel attention scores are calculated using softmax, as shown in the following formula:

[0056]

[0057] Where T denotes matrix transpose. d k The feature dimension of the key.

[0058] Furthermore, step S42 involves calculating the attention score over a time span. The specific process is as follows:

[0059] A sliding window mechanism is used to capture long-range time dependencies, generating time Q, K, and V matrices as follows:

[0060]

[0061]

[0062]

[0063] in, The time step matrix, As a learnable parameter, the temporal attention score is calculated as follows:

[0064]

[0065] Where T denotes matrix transpose. d k The feature dimension of the key.

[0066] Further, step S5 includes the following steps:

[0067] S51. The encoded features output by the encoder are processed through a fully connected layer. Mapping to the sentiment classification space yields preliminary classification features, as shown in the following formula:

[0068]

[0069] in, and The weight matrix and bias terms are learnable. For classification feature vectors;

[0070] S52. Introduce a domain discrimination submodule for domain adversarial training. This module contains a domain classifier, whose input is encoded features. The output is the prediction result of the region to which the feature belongs;

[0071] S53. The decoder maps the encoded features of the input, obtains the three-class probability distribution through the softmax function, takes the class with the highest probability, and outputs the sentiment classification result.

[0072] Furthermore, the adversarial training described in step S52 is specifically as follows:

[0073] (1) Adversarial training is conducted through minimax games. The feature mapping submodule of the decoder is dedicated to learning domain-invariant features, making the domain discriminator unable to distinguish the source of features; the domain discriminator is dedicated to accurately distinguishing the source of the domain; the domain loss function is as follows:

[0074]

[0075] in, The domain probability output by the domain discriminator. Represents source domain samples, (2) The classification loss is calculated using the cross-entropy loss function, as shown in the following formula:

[0076]

[0077] Where M is the number of samples, For indicator functions, For the true labels of the samples, =-1, 1, and 1 correspond to negative, neutral, and positive, respectively. For the sample j The probability of being predicted as category c;

[0078] (3) Perform joint optimization by simultaneously optimizing the parameters of the feature mapping submodule and the domain discrimination submodule through backpropagation until the model converges. The total loss function is as follows:

[0079]

[0080] in, This is the balance coefficient.

[0081] This invention also provides a cross-subject EEG emotion recognition system based on spatiotemporal adaptive graph coding learning. The system operates using the above-described method and includes the following modules:

[0082] The data preprocessing module is used to divide the preprocessed and feature-extracted EEG data samples using the sliding window technique, and then extract features from different frequency bands to construct a feature matrix covering channel and time dimensions.

[0083] The high-dimensional semantic vector construction module is used to transform the feature matrix into a high-dimensional semantic vector by using feature preprocessing and mapper through the multi-dimensional data embedding module, which integrates spatiotemporal hybrid embedding, temporal embedding extracted by BiLSTM and dilated convolution;

[0084] The dynamic topology matrix construction module is used to construct high-dimensional semantic vectors based on the high-dimensional semantic vector construction module. The adaptive spatiotemporal topology construction module of the graph structure generator calculates channel feature differences and time trend changes to generate a dynamic topology matrix that adapts to the differences across individual subjects.

[0085] The spatiotemporal feature extraction module is used by the encoder, which takes the dynamic spatiotemporal hybrid attention coding module as the core, combines the dynamic topology matrix to perform parallel computation of spatial cross-channel and temporal cross-step attention, and integrates and strengthens key spatiotemporal features through a gating mechanism.

[0086] The output module is used by the decoder to map encoded features with the help of the domain adversarial decoding module. Through cross-domain adversarial training, the difference in distribution between the source domain and the target domain is reduced, and the sentiment classification result is output.

[0087] The advantages of this invention are:

[0088] (1) By dynamically calculating the channel feature differences and time trend changes, a topological matrix adapted to the individual differences of the subjects is generated, replacing the traditional predefined fixed adjacency matrix. This effectively alleviates the performance degradation problem caused by individual physiological differences and significantly improves the robustness of the model in recognizing new subject data.

[0089] (2) By splitting the traditional self-attention mechanism into parallel computation of spatial cross-channel and temporal cross-step attention, and combining it with the gating mechanism to dynamically fuse key features, it not only strengthens the capture of local channel collaboration patterns related to emotion, but also enhances the modeling ability of long-term temporal dependence, reduces redundant information interference, and improves the pertinence and effectiveness of feature representation.

[0090] (3) The domain adversarial module introduced by the decoder optimizes through minimax game, forcing the model to learn the shared features of the source domain and the target domain, which significantly reduces the negative impact of the data distribution shift between subjects and solves the pain point of the traditional method's performance drop sharply on new subject data.

[0091] (4) Temporal embedding integrates the fine-grained temporal features of BiLSTM with the long-term trend features of dilated convolution, while spatial embedding preserves the topological information of the graph Laplacian matrix. Combined with the gating fusion mechanism of spatiotemporal hybrid embedding, it comprehensively captures the multi-scale spatiotemporal characteristics of EEG signals, providing richer semantic representations for emotion recognition and helping the model distinguish subtle emotional differences such as negative, neutral, and positive. Attached Figure Description

[0092] Figure 1 This is a schematic diagram of the cross-subject EEG emotion recognition method based on spatiotemporal adaptive graph coding learning according to an embodiment of the present invention;

[0093] Figure 2 This is a schematic diagram of sample division with a sliding window of T=6 and an overlap of 4 seconds in an embodiment of the present invention;

[0094] Figure 3 This diagram illustrates the overall performance comparison between the method of this invention and other models on the SEED dataset. Detailed Implementation

[0095] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0096] Example 1

[0097] This embodiment provides a cross-subject EEG emotion recognition method based on spatiotemporal adaptive graphical encoding learning. The specific implementation process is as follows: Figure 1 As shown, it includes the following steps:

[0098] S1. The preprocessed and feature-extracted EEG data samples are divided using the sliding window technique, and features are then extracted from different frequency bands to construct a feature matrix covering both channel and time dimensions. The specific implementation includes the following steps:

[0099] S11. The preprocessed EEG feature data is divided into samples using a sliding window technique. Overlapping windows are used to increase the number of samples, with each window containing several consecutive 1-second time points. Features in five frequency bands are extracted from each window's data. These five frequency bands include delta, theta, alpha, beta, and gamma bands, corresponding to the core dimensions of the differential entropy feature. In this embodiment, as... Figure 2 As shown, T=6 indicates that it contains 6 consecutive 1-second time points, and the overlap time of the sliding window is 4 seconds.

[0100] S12. The extracted multi-band features are organized into a three-dimensional feature matrix, where the 0th dimension represents 62 EEG channels (conforming to the international 10-20 system electrode layout), the 1st dimension represents the number of time points included in the window, and the 2nd dimension represents 5 frequency bands, forming a feature matrix covering both channel and time dimensions. This matrix can be directly used in subsequent multi-dimensional data embedding steps.

[0101] S2. A feature preprocessing and mapping module is used to fuse temporal and spatial embeddings extracted by BiLSTM and dilated convolution through a multi-dimensional data embedding module, transforming the feature matrix into a high-dimensional semantic vector. The specific implementation includes the following steps:

[0102] S21, Based on historical data coding features and current data encoding characteristics By fusing information from both sources through a gating mechanism, a spatiotemporal hybrid embedding is obtained. As shown in the following formula:

[0103]

[0104] in , For learnable parameter matrix, a , b For bias terms, The tanh activation function is used. It is the sigmoid activation function. For Hadama accumulation, ,in N For the number of channels, T For time points, D In this embodiment, D=64, which is the dimension of the hidden layer.

[0105] S22. BiLSTM is used to model the temporal dimension of the feature matrix to capture bidirectional temporal dependencies, with the output dimension being... Temporal features are extracted using dilated convolutional layers with a dilation rate of 2. The receptive field increases exponentially with the number of layers, and the output dimension is... The outputs of both are concatenated and linearly transformed, then combined with sine and cosine position coding to obtain the time embedding. ,and The sine and cosine position encoding is as follows:

[0106]

[0107]

[0108] S23. Constructing an adjacency matrix based on the EEG channel topology. A Calculate the normalized graph Laplacian matrix as follows:

[0109]

[0110] in, I It is the identity matrix. D For degree matrix, It is the eigenvalue matrix;

[0111] Take its eigenvector As spatial embedding ;

[0112] S24. Embedding spatiotemporal mixing Time embedding Spatial embedding Element-wise addition yields a high-dimensional semantic vector. As shown in the following formula:

[0113]

[0114] in, , B For batch size.

[0115] S3. Based on the high-dimensional semantic vector obtained in step S2, the adaptive spatiotemporal topology construction module of the graph structure generator calculates the channel feature differences and temporal trend changes to generate a dynamic topology matrix adapted to the differences across individual subjects; the specific implementation includes the following steps:

[0116] S31, Regarding high-dimensional semantic vectors Joint modeling of nodes in spatial and temporal dimensions is performed, and the high-dimensional semantic vector containing 62 channels and T time points is flattened into a spatiotemporal node sequence composed of channel-time pairs. Each node uniquely corresponds to the feature of a specific channel at a specific time step.

[0117] S32. In the spatial dimension, for each time step, the feature associations between 62 channels are calculated using cosine similarity to generate a channel association matrix. Quantify the functional synergy between different brain regions, such as the frontal and parietal regions, at the same time point;

[0118] S33. In the time dimension, for each channel, the trend correlation over T time steps is calculated using the Pearson correlation coefficient to generate a time correlation matrix. Capture the temporal evolution of emotional states within the same channel, such as the trend of emotional arousal under video stimulation;

[0119] S34. The two matrices are fused using the Kronecker product to generate the initial global topology matrix. The matrix has dimensions of The diagonal submatrix represents the channel association at the same time point, while the off-diagonal submatrix represents the channel interaction across time points, thus realizing the explicit fusion of spatiotemporal associations.

[0120] S35. Introduce a learnable adjacency matrix. The feature projection layer maps node features to a dimension matching the number of nodes, and matrix multiplication is used to transform the initial global topology matrix. The result is fused with the feature projection result and then normalized using the Softmax function to obtain the dynamic topology matrix. The feature projection result is an intermediate matrix obtained by mapping high-dimensional semantic vectors through a feature projection layer, serving as a "bridge" connecting high-dimensional semantic vectors and dynamic topological matrices. During this process, learnable parameters... Through backpropagation optimization, the topology matrix is ​​made adaptable to the differences in EEG signal distribution among different subjects, such as compensating for individual differences in channel response intensity and emotional temporal patterns among subjects. The final generated topology matrix... It retains the physiological topological priors of EEG channels, such as the channel layout of the international 10-20 system, and dynamically adapts to cross-subject differences through data-driven parameter learning, providing precise structural support for the subsequent spatiotemporal attention calculation of the encoder, thereby improving the model's ability to generalize to different subjects' emotional characteristics.

[0121] S4. The encoder uses a dynamic spatiotemporal hybrid attention coding module as its core, combining parallel computation of spatial cross-channel and temporal cross-step attention with a dynamic topology matrix. It then fuses and enhances key spatiotemporal features through a gating mechanism. The specific implementation includes the following steps:

[0122] S41, Based on dynamic topology matrix Construct a global spatiotemporal graph to transform high-dimensional semantic vectors into a global association matrix containing channel and time dimensions. The diagonal submatrix represents the spatial association between all channels at the same time point, while the off-diagonal submatrix represents the cross-temporal association between all channels at different time points.

[0123] S42. Calculate the spatial cross-channel attention score. Attention score and time stride ;

[0124] The computational space cross-channel attention score The specific process is as follows:

[0125] Through dynamic topology matrix Generate the space matrix Q, K, V as follows:

[0126]

[0127]

[0128]

[0129] in For dynamic topology matrix, As learnable parameters, the inter-channel attention scores are calculated using softmax, as shown in the following formula:

[0130]

[0131] Where T denotes matrix transpose. d k The feature dimension of the key.

[0132] The calculation of the time span attention score The specific process is as follows:

[0133] A sliding window mechanism is used to capture long-range time dependencies, generating time Q, K, and V matrices as follows:

[0134]

[0135]

[0136]

[0137] in, The time step matrix, As a learnable parameter, the temporal attention score is calculated as follows:

[0138]

[0139] Where T denotes matrix transpose. d k The feature dimension of the key.

[0140] S43. Obtaining the fusion features of two types of attention through gating mechanisms. As shown in the following formula:

[0141]

[0142]

[0143] in For gating parameters, For splicing operations;

[0144] S44, Integrating Features Multi-subspace feature aggregation is performed using a multi-head attention module, as shown in the following equation:

[0145]

[0146] in, For projection parameters, Let h be the feature representation learned by the h-th attention head in its dedicated subspace;

[0147] S45. Enhance feature stability through residual connections and layer normalization, as shown in the following formula:

[0148]

[0149]

[0150] S46. By reducing the sequence length and retaining key features through a distillation mechanism, the key spatiotemporal features are enhanced to obtain the encoded features, as shown in the following formula:

[0151]

[0152] in, For max pooling operations, ELU stands for exponential linear unit. This is a one-dimensional convolution operation.

[0153] S5. The decoder maps the encoded features using a domain adversarial decoding module, reduces the distribution difference between the source and target domains through cross-domain adversarial training, and outputs the sentiment classification result. The specific implementation includes the following steps:

[0154] S51. The encoded features output by the encoder are processed through a fully connected layer. Mapping to the sentiment classification space yields preliminary classification features, as shown in the following formula:

[0155]

[0156] in, and The weight matrix and bias terms are learnable. For classification feature vectors;

[0157] S52. Introduce a domain discrimination submodule for domain adversarial training. This module contains a domain classifier, whose input is encoded features. The output is the prediction result of the feature's domain (source domain or target domain); the adversarial training process is as follows:

[0158] (1) Adversarial training is conducted through minimax games. The feature mapping submodule of the decoder is dedicated to learning domain-invariant features, making the domain discriminator unable to distinguish the source of features (the objective function is...). The domain discriminator, on the other hand, strives to accurately distinguish the origin of the domain (the objective function is...). The domain loss function is as follows:

[0159]

[0160] in, The domain probability output by the domain discriminator. Represents source domain samples, (2) The classification loss is calculated using the cross-entropy loss function, as shown in the following formula:

[0161]

[0162] Where M is the number of samples, For indicator functions, For the true labels of the samples, =-1, 1, and 1 correspond to negative, neutral, and positive, respectively. For the sample j The probability of being predicted as category c;

[0163] (3) Perform joint optimization by simultaneously optimizing the parameters of the feature mapping submodule and the domain discrimination submodule through backpropagation until the model converges. The total loss function is as follows:

[0164]

[0165] in, This is the balance coefficient.

[0166] S53. The decoder maps the encoded features of the input, obtains the three-class probability distribution through the softmax function, takes the class with the highest probability, and outputs the sentiment classification result, namely negative (-1), neutral (0) or positive (1).

[0167] This embodiment also provides experimental simulation results of the above method, as follows:

[0168] Accuracy was used to evaluate the performance of the model in this invention. The accuracy was calculated as follows:

[0169]

[0170] in, This represents the number of samples where the c-th sentiment was correctly predicted as c. This represents the number of samples where non-c-th sentiment was incorrectly predicted as c. This represents the number of samples where the c-th sentiment was incorrectly predicted as a non-c-th sentiment. (c can take values ​​of 0, 1, and 2, representing negative, neutral, and positive sentiments, respectively.)

[0171] The experimental data were obtained from the public dataset SEED and were validated.

[0172] For the SEED dataset, the 15 subjects were divided into a training set consisting of 14 subjects and a test set consisting of one subject, as shown in the table below. Figure 3 As shown, the overall performance comparison of different models and the same model after parameter adjustment on the SEED dataset is displayed. It can be seen that the method described in this embodiment is significantly better than the SVM algorithm and the random forest algorithm when different parameters (length of the sliding window, number of seconds of sliding window overlap) are adjusted, which confirms its significant advantage in EEG emotion recognition.

[0173]

[0174] Example 2

[0175] It should be further explained that, based on the same inventive concept, this embodiment also provides a cross-subject EEG emotion recognition system based on spatiotemporal adaptive graph coding learning. The system operates using the method described in Embodiment 1, and includes the following modules:

[0176] The data preprocessing module is used to divide the preprocessed and feature-extracted EEG data samples using the sliding window technique, and then extract features from different frequency bands to construct a feature matrix covering channel and time dimensions.

[0177] The high-dimensional semantic vector construction module is used to transform the feature matrix into a high-dimensional semantic vector by using feature preprocessing and mapper through the multi-dimensional data embedding module, which integrates spatiotemporal hybrid embedding, temporal embedding extracted by BiLSTM and dilated convolution;

[0178] The dynamic topology matrix construction module is used to construct high-dimensional semantic vectors based on the high-dimensional semantic vector construction module. The adaptive spatiotemporal topology construction module of the graph structure generator calculates channel feature differences and time trend changes to generate a dynamic topology matrix that adapts to the differences across individual subjects.

[0179] The spatiotemporal feature extraction module is used by the encoder, which takes the dynamic spatiotemporal hybrid attention coding module as the core, combines the dynamic topology matrix to perform parallel computation of spatial cross-channel and temporal cross-step attention, and integrates and strengthens key spatiotemporal features through a gating mechanism.

[0180] The output module is used by the decoder to map encoded features with the help of the domain adversarial decoding module. Through cross-domain adversarial training, the difference in distribution between the source domain and the target domain is reduced, and the sentiment classification result is output.

[0181] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-subject EEG emotion recognition method based on spatiotemporal adaptive graphical encoding learning, characterized in that, Includes the following steps: S1. The preprocessed and feature-extracted EEG data samples are divided using the sliding window technique, and features are then extracted from different frequency bands to construct a feature matrix covering channel and time dimensions. S2. The feature matrix is ​​transformed into a high-dimensional semantic vector by using a feature preprocessing and mapper through a multi-dimensional data embedding module, which integrates spatiotemporal hybrid embedding, temporal embedding extracted by BiLSTM and dilated convolution. Includes the following steps: S21, Based on historical data coding features and current data encoding characteristics By fusing information from both sources through a gating mechanism, a spatiotemporal hybrid embedding can be obtained. As shown in the following formula: in , For learnable parameter matrix, a , b For bias terms, The tanh activation function is used. It is the sigmoid activation function. For Hadama accumulation, ,in N For the number of channels, T For time points, D For the hidden layer dimension; S22. BiLSTM is used to model the temporal dimension of the feature matrix to capture bidirectional temporal dependencies, with the output dimension being... Temporal features are extracted using dilated convolutional layers with a dilation rate of 2. The receptive field increases exponentially with the number of layers, and the output dimension is... The outputs of both are concatenated and linearly transformed, then combined with sine and cosine position coding to obtain the time embedding. ,and ; S23. Constructing an adjacency matrix based on the EEG channel topology. A Calculate the normalized graph Laplacian matrix as follows: in, I It is the identity matrix. D For degree matrix, It is the eigenvalue matrix; Take its eigenvector As spatial embedding ; S24. Embedding spatiotemporal mixing Time embedding Spatial embedding Element-wise addition yields a high-dimensional semantic vector. As shown in the following formula: in, , B Batch size; S3. Based on the high-dimensional semantic vector obtained in step S2, the adaptive spatiotemporal topology construction module of the graph structure generator calculates the channel feature differences and time trend changes to generate a dynamic topology matrix that adapts to the differences across individual subjects. S4. The encoder is based on a dynamic spatiotemporal hybrid attention coding module. It combines the dynamic topology matrix to perform parallel computation of spatial cross-channel and temporal cross-step attention, and integrates and strengthens key spatiotemporal features through a gating mechanism. S5. The decoder uses the domain adversarial decoding module to map the encoded features, reduces the distribution difference between the source domain and the target domain through cross-domain adversarial training, and outputs the sentiment classification result.

2. The method for cross-subject EEG emotion recognition based on spatiotemporal adaptive graph coding learning according to claim 1, characterized in that, Step S1 includes the following steps: S11. The preprocessed EEG feature data is divided into samples using the sliding window technique. The number of samples is increased by overlapping windows. Each window contains several consecutive 1-second time point data. Features of 5 frequency bands are extracted from the data of each window after division. The 5 frequency bands include delta, theta, alpha, beta, and gamma frequency bands, which correspond to the core dimensions of differential entropy features. S12. Organize the extracted multi-band features into a three-dimensional feature matrix, where the 0th dimension is 62 EEG channels, the 1st dimension is the number of time points included in the window, and the 2nd dimension is 5 frequency bands, forming a feature matrix covering both channel and time dimensions.

3. The spatiotemporal adaptive graph coding learning method for cross-subject EEG emotion recognition according to claim 2, characterized in that, Step S3 includes the following steps: S31, Regarding high-dimensional semantic vectors Joint modeling of nodes in spatial and temporal dimensions is performed, and the high-dimensional semantic vector containing 62 channels and T time points is flattened into a spatiotemporal node sequence composed of channel-time pairs. Each node uniquely corresponds to the feature of a specific channel at a specific time step. S32. In the spatial dimension, for each time step, the feature associations between 62 channels are calculated using cosine similarity to generate a channel association matrix. S33. In the time dimension, for each channel, the trend correlation over T time steps is calculated using the Pearson correlation coefficient to generate a time correlation matrix. ; S34. The two matrices are fused using the Kronecker product to generate the initial global topology matrix. The matrix has dimensions of The diagonal submatrix represents the channel association at the same time point, while the off-diagonal submatrix represents the channel interaction across time points, thus realizing the explicit fusion of spatiotemporal associations. S35. Introduce a learnable adjacency matrix. The feature projection layer maps node features to a dimension matching the number of nodes, and matrix multiplication is used to transform the initial global topology matrix. The result is fused with the feature projection result and then normalized using the Softmax function to obtain the dynamic topology matrix. .

4. The cross-subject EEG emotion recognition method based on spatiotemporal adaptive graph coding learning according to claim 3, characterized in that, Step S4 includes the following steps: S41, Based on dynamic topology matrix Construct a global spatiotemporal graph to transform high-dimensional semantic vectors into a global association matrix containing channel and time dimensions. The diagonal submatrix represents the spatial association between all channels at the same time point, while the off-diagonal submatrix represents the cross-temporal association between all channels at different time points. S42. Calculate the spatial cross-channel attention score. Attention score and time stride ; S43. Obtaining the fusion features of two types of attention through gating mechanisms. As shown in the following formula: in For gating parameters, For splicing operations; S44, Integrating Features Multi-subspace feature aggregation is performed using a multi-head attention module, as shown in the following equation: in, For projection parameters, Let h be the feature representation learned by the h-th attention head in its dedicated subspace; S45. Enhance feature stability through residual connections and layer normalization, as shown in the following formula: S46. By reducing the sequence length and retaining key features through a distillation mechanism, the key spatiotemporal features are enhanced to obtain the encoded features, as shown in the following formula: in, For max pooling operations, ELU stands for exponential linear unit. This is a one-dimensional convolution operation.

5. The cross-subject EEG emotion recognition method based on spatiotemporal adaptive graph coding learning according to claim 4, characterized in that, Step S42 describes the calculation of spatial cross-channel attention scores. The specific process is as follows: Through dynamic topology matrix Generate the space matrix Q, K, V as follows: in For dynamic topology matrix, As learnable parameters, the inter-channel attention scores are calculated using softmax, as shown in the following formula: Where T denotes matrix transpose. d k The feature dimension of the key.

6. The spatiotemporal adaptive graph coding learning method for cross-subject EEG emotion recognition according to claim 4, characterized in that, The step S42 described above involves calculating the attention score over the time span. The specific process is as follows: A sliding window mechanism is used to capture long-range time dependencies, generating time Q, K, and V matrices as follows: in, The time step matrix, As a learnable parameter, the temporal attention score is calculated as follows: Where T denotes matrix transpose. d k The feature dimension of the key.

7. The spatiotemporal adaptive graph coding learning method for cross-subject EEG emotion recognition according to claim 4, characterized in that, Step S5 includes the following steps: S51. The encoded features output by the encoder are processed through a fully connected layer. Mapping to the sentiment classification space yields preliminary classification features, as shown in the following formula: in, and The weight matrix and bias terms are learnable. For classification feature vectors; S52. Introduce a domain discrimination submodule for domain adversarial training. This module contains a domain classifier, whose input is encoded features. The output is the prediction result of the region to which the feature belongs; S53. The decoder maps the encoded features of the input, obtains the three-class probability distribution through the softmax function, takes the class with the highest probability, and outputs the sentiment classification result.

8. The method for cross-subject EEG emotion recognition based on spatiotemporal adaptive graph coding learning according to claim 7, characterized in that, The adversarial training described in step S52 is as follows: (1) Adversarial training is conducted through minimax games. The feature mapping submodule of the decoder is dedicated to learning domain-invariant features, making the domain discriminator unable to distinguish the source of features; the domain discriminator is dedicated to accurately distinguishing the source of the domain; the domain loss function is as follows: in, The domain probability output by the domain discriminator. Represents source domain samples, (2) The classification loss is calculated using the cross-entropy loss function, as shown in the following formula: Where M is the number of samples, For indicator functions, For the true labels of the samples, =-1, 1, and 1 correspond to negative, neutral, and positive, respectively. For the sample j The probability of being predicted as category c; (3) Perform joint optimization by simultaneously optimizing the parameters of the feature mapping submodule and the domain discrimination submodule through backpropagation until the model converges. The total loss function is as follows: in, This is the balance coefficient.

9. A cross-subject EEG emotion recognition system based on spatiotemporal adaptive graphical encoding learning, characterized in that, When the system is running, it executes the method according to any one of claims 1-8, comprising the following modules: The data preprocessing module is used to divide the preprocessed and feature-extracted EEG data samples using the sliding window technique, and then extract features from different frequency bands to construct a feature matrix covering channel and time dimensions. The high-dimensional semantic vector construction module is used to transform the feature matrix into a high-dimensional semantic vector by using feature preprocessing and mapper through the multi-dimensional data embedding module, which integrates spatiotemporal hybrid embedding, temporal embedding extracted by BiLSTM and dilated convolution; The dynamic topology matrix construction module is used to construct high-dimensional semantic vectors based on the high-dimensional semantic vector construction module. The adaptive spatiotemporal topology construction module of the graph structure generator calculates channel feature differences and time trend changes to generate a dynamic topology matrix that adapts to the differences across individual subjects. The spatiotemporal feature extraction module is used by the encoder, which takes the dynamic spatiotemporal hybrid attention coding module as the core, combines the dynamic topology matrix to perform parallel computation of spatial cross-channel and temporal cross-step attention, and integrates and strengthens key spatiotemporal features through a gating mechanism. The output module is used by the decoder to map encoded features with the help of the domain adversarial decoding module. Through cross-domain adversarial training, the difference in distribution between the source domain and the target domain is reduced, and the sentiment classification result is output.

Images