Cross-subject electroencephalogram emotion recognition method and system based on space-time adaptive graph coding learning

Abstract

The invention belongs to the field of deep learning, and provides a cross-subject brain electrical emotion recognition method and system for space-time adaptive graph coding learning, and the method comprises the steps: extracting features from different frequency bands through a sliding window technology, and constructing a feature matrix covering channels and time dimensions; fusing space-time hybrid embedding, time embedding and space embedding, and converting the feature matrix into a high-dimensional semantic vector; calculating channel characteristic difference and time trend change, and generating a dynamic topology matrix adaptive to cross-tested individual difference; combining the dynamic topological matrix to calculate spatial cross-channel and time stride length attention in parallel, and fusing and strengthening key spatial-temporal features; and the decoder maps the coding features by means of a domain adversarial decoding module, reduces the distribution difference between a source domain and a target domain through cross-domain adversarial training, and outputs an emotion classification result. Through the method, the cross-subject generalization ability and the recognition accuracy of electroencephalogram emotion recognition are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning, and particularly relates to a cross-subject electroencephalogram emotion recognition method and system based on spatio-temporal adaptive graph encoding learning. BACKGROUND

[0002] With the deep integration of affective computing and artificial intelligence, emotion recognition based on electroencephalogram (EEG) signals has become a research hotspot in the fields of human-computer interaction and mental health monitoring, because it can directly reflect brain neural activity. EEG signals contain rich spatio-temporal features, and the emotion-related patterns (such as negative, neutral, and positive) can be automatically recognized through feature extraction and modeling. However, the universality in the cross-subject scenario is still a core challenge. Individual physiological differences (such as brain structure and neural response intensity) and experimental environment interference often lead to a significant decrease in the recognition performance of the model on new subject data.

[0003] Traditional methods mostly rely on manually extracting time-domain and frequency-domain features (such as differential entropy and power spectral density) and combining them with shallow model classification such as support vector machine. These methods have limited feature representation ability and are sensitive to individual differences. Although deep learning improves performance through convolutional neural networks (CNN), recurrent neural networks (RNN / LSTM), and Transformer models, it still faces many limitations. For example, most graph neural network (GNN)-based methods use a predefined EEG channel topology (such as an adjacency matrix based on the physical location of the electrodes). For example, the Chinese invention patent with the publication number CN119279611A, “Emotion recognition method based on EEG spatio-temporal frequency feature fusion”, fully considers the feature information in the time, frequency, and space dimensions of EEG signals. By fusing the extracted emotion features, the emotion classification result is output. However, this method cannot dynamically adapt to the distribution differences of EEG signals of different subjects, leading to a mismatch between the topology structure and the real spatio-temporal correlation, and thus limiting the cross-subject generalization ability.

[0004] In addition, traditional self-attention mechanisms do not separate and optimize the spatio-temporal heterogeneity of EEG signals when processing them. They simultaneously process the spatial channels and time steps, which are easily disturbed by redundant information and difficult to focus on key emotion-related features (such as channel coordination patterns in specific frequency bands). Existing models are mostly trained on single or mixed subject data, without considering the data distribution shift between the source domain (training subjects) and the target domain (new subjects), leading to a sharp performance drop in the cross-subject scenario due to domain differences. Furthermore, time embedding mostly relies on simple position encoding, without fully integrating the multi-scale time characteristics (such as instantaneous response and long-term trend) of EEG signals. Spatial embedding also does not effectively incorporate dynamic functional connectivity information, making it difficult to capture the brain region coordination patterns in the emotional state. SUMMARY

[0005] The technical problem to be solved by the present application is how to improve the cross-subject generalization ability and recognition accuracy of electroencephalogram signal emotion recognition.

[0006] The present application solves the above technical problems by the following technical means: The present application provides a cross-subject electroencephalogram emotion recognition method based on spatio-temporal adaptive graph coding learning, comprising the following steps: S1, using a sliding window technique to divide the electroencephalogram data samples that have been preprocessed and feature extracted, and then extracting features from different frequency bands to construct a feature matrix covering channel and time dimensions; S2, using a feature preprocessing and mapper through a multi-dimensional data embedding module, fusing time-space hybrid embedding, BiLSTM and dilated convolution extracted time embedding and space embedding, and converting the feature matrix into a high-dimensional semantic vector; S3, based on the high-dimensional semantic vector obtained in step S2, the adaptive spatio-temporal topology construction module of the graph structure generator calculates the channel feature difference and the time trend change, and generates a dynamic topology matrix that adapts to the individual differences of cross-subjects; S4, the encoder takes the dynamic spatio-temporal hybrid attention coding module as the core, combines the dynamic topology matrix to calculate the spatial cross-channel and time cross-step attention in parallel, and fuses and strengthens the key spatio-temporal features through the gating mechanism; S5, the decoder maps the encoded features with the help of the domain adversarial decoding module, reduces the distribution difference between the source domain and the target domain through cross-domain adversarial training, and outputs the emotion classification result.

[0007] Further, the step S1 comprises the following steps: S11, using a sliding window technique to divide the preprocessed electroencephalogram feature data into samples, increasing the number of samples by overlapping windows, and each window contains a plurality of continuous 1-second time point data; extract features of 5 frequency bands from each window data after division, the 5 frequency bands include delta, theta, alpha, beta and gamma frequency bands, which correspond to the core dimensions of differential entropy features; S12, organize the extracted multi-frequency band features into a three-dimensional feature matrix, where the 0th dimension is 62 EEG channels, the 1st dimension is the number of time points contained in the window, and the 2nd dimension is 5 frequency bands, forming a feature matrix covering channel and time dimensions.

[0008] Further, the step S2 comprises the following steps: S21, based on historical data encoding features and current data encoding features , fuse the information of the two through the gating mechanism to obtain the spatio-temporal hybrid embedding , as follows:

[0009] wherein 、 is a learnable parameter matrix, a 、 b is a bias term, is a tanh activation function, is a sigmoid activation function, is a Hadamard product, wherein N is a number of channels, T is a number of time points, D is a hidden layer dimension; S22, modeling the time dimension of the feature matrix using BiLSTM to capture bidirectional temporal dependencies, the output dimension is extracting time scale features through a convolutional layer with an expansion rate of 2, the receptive field exponentially increases with the number of layers, and the output dimension is ; concatenating the outputs of the two and performing linear transformation, combining the sine-cosine position encoding, obtaining the time embedding , and ; S23, constructing an adjacency matrix based on the topological structure of the electroencephalogram channel A , calculating the normalized graph Laplacian matrix as follows:

[0010] wherein, I is an identity matrix, D is a degree matrix, is an eigenvalue matrix; taking its eigenvector as a spatial embedding ; S24, element-wise adding the spatio-temporal hybrid embedding , the time embedding , and the spatial embedding , obtaining a high-dimensional semantic vector , as follows:

[0011] wherein, , B is a batch size.

[0012] Further, the step S3 comprises the following steps: S31, performing joint modeling of the spatial and temporal dimensions of the high-dimensional semantic vector , flattening the high-dimensional semantic vector containing 62 channels and T time points into a spatio-temporal node sequence composed of channel-time binary tuples, each node uniquely corresponding to the features of a specific channel at a specific time step; S32, in the spatial dimension, for each time step, the feature correlation between 62 channels is calculated by cosine similarity to generate a channel correlation matrix

[0013] S33, in the time dimension, for each channel, the trend correlation of T time steps is calculated by Pearson correlation coefficient to generate a time correlation matrix ; S34, the above two matrices are fused by Kronecker product to generate an initial global topology matrix , the dimension of which is , wherein the diagonal submatrix represents the channel correlation at the same time point, and the non-diagonal submatrix represents the channel interaction across time points, realizing the explicit fusion of space-time correlation; S35, a learnable adjacency matrix is introduced, which combines with the feature projection layer to map the node features to a dimension matching the number of nodes, and through matrix multiplication, the initial global topology matrix is fused with the feature projection result, and then normalized by the Softmax function to obtain a dynamic topology matrix ; Further, the step S4 comprises the following steps: S41, based on the dynamic topology matrix , a global space-time graph is constructed to convert the high-dimensional semantic vector into a global correlation matrix containing channel and time dimensions, wherein the diagonal submatrix represents the spatial correlation between all channels at the same time point, and the non-diagonal submatrix represents the cross-time correlation between all channels at different time points; S42, calculate the spatial cross-channel attention score and the time cross-step attention score ; S43, obtain the fusion features of the two types of attention through the gating mechanism , as follows:

[0014]

[0015] , wherein is a gating parameter, is a concatenation operation; S44, the fusion features are aggregated by a multi-head attention module to obtain multi-subspace features, as follows:

[0016] , wherein is a projection parameter, The feature representation learned by the hth attention head in its exclusive subspace; S45, the feature stability is strengthened by residual connection and layer normalization, as follows:

[0017]

[0018] S46, the sequence length is reduced and the key features are preserved through the distillation mechanism, the spatiotemporal key features are strengthened, and the encoded features are obtained, as follows:

[0019] wherein, is the max-pooling operation, and ELU is the exponential linear unit, is the one-dimensional convolution operation.

[0020] Further, the spatial cross-channel attention score calculated in step S42 is The specific process is as follows: The spatial Q, K, and V matrices are generated through the dynamic topology matrix

[0021]

[0022]

[0023] wherein, is the dynamic topology matrix, is a learnable parameter, and the inter-channel attention score is calculated through softmax, as follows:

[0024] wherein, T represents matrix transposition, d k represents the feature dimension of the key.

[0025] Further, the temporal cross-step length attention score calculated in step S42 is The specific process is as follows: The time Q, K, and V matrices are generated through the sliding window mechanism to capture long-range temporal dependencies, as follows:

[0026]

[0027]

[0028] wherein, ​is a time step length matrix, is a learnable parameter, the time attention score is calculated as follows:

[0029] where T represents the matrix transpose, d k represents the feature dimension of the key.

[0030] Further, the step S5 comprises the following steps: S51, mapping the encoded features output by the encoder to the emotion classification space through a fully connected layer to obtain preliminary classification features, as follows:

[0031] wherein, and are learnable weight matrices and bias terms, is a classification feature vector; S52, introducing a domain discriminator sub-module for domain adversarial training, which includes a domain classifier, the input of which is the encoded features , and the output is the predicted result of the domain to which the features belong; S53, the decoder maps the input encoded features, and obtains a three-class probability distribution through a softmax function, takes the class with the highest probability, and outputs the emotion classification result.

[0032] Further, the adversarial training of step S52 has the following specific process: (1) through the minimax game for adversarial training, the feature mapping sub-module of the decoder is committed to learning domain-invariant features, so that the domain discriminator cannot distinguish the source of the features; the domain discriminator is committed to accurately distinguishing the domain source; the domain loss function is as follows:

[0033] wherein, is the domain probability output by the domain discriminator, represents the source domain sample, represents the target domain sample; (2) the classification loss is calculated using the cross-entropy loss function, as follows:

[0034] wherein, M is the number of samples, is an indicator function, is the true label of the sample, =-1, 1, 1 correspond to negative, neutral, and positive, respectively, is the probability that the sample j is predicted to be class c; ​(3) Joint optimization is performed, parameters of the feature mapping sub-module and the domain discrimination sub-module are simultaneously optimized through back propagation until the model converges, and the total loss function is as follows:

[0035] wherein, is a balance coefficient.

[0036] The application further provides a cross-subject electroencephalogram emotion recognition system based on spatio-temporal adaptive graph coding learning, which adopts the above method during system operation and comprises the following modules: A data preprocessing module is configured to divide the electroencephalogram data samples that have been preprocessed and extracted features by using a sliding window technology, and then extract features from different frequency bands to construct a feature matrix covering the channel and time dimensions. A high-dimensional semantic vector construction module is configured to convert the feature matrix into a high-dimensional semantic vector by fusing the time embedding and space embedding extracted by the spatio-temporal hybrid embedding, BiLSTM and dilated convolution through the multi-dimensional data embedding module of the feature preprocessing and mapper. A dynamic topology matrix construction module is configured to calculate the channel feature difference and time trend change by the adaptive spatio-temporal topology construction module of the graph structure generator based on the high-dimensional semantic vector obtained by the high-dimensional semantic vector construction module, and generate a dynamic topology matrix that adapts to the cross-subject individual difference. A spatio-temporal feature extraction module is configured to calculate the spatial cross-channel and time cross-step attention in parallel by the dynamic spatio-temporal hybrid attention coding module as the core of the encoder in combination with the dynamic topology matrix, and fuse and strengthen the key spatio-temporal features through the gating mechanism. An output module is configured to map the encoded features by the decoder with the aid of the domain adversarial decoding module, reduce the distribution difference between the source domain and the target domain through cross-domain adversarial training, and output the emotion classification result.

[0037] The application has the following advantages: (1) The dynamic topology matrix that adapts to the cross-subject individual difference is generated by dynamically calculating the channel feature difference and time trend change, replacing the traditional pre-defined fixed adjacency matrix, effectively alleviating the performance decline problem caused by individual physiological differences, and significantly improving the recognition robustness of the model to new subject data. (2) The traditional self-attention mechanism is split into spatial cross-channel and time cross-step attention parallel calculation, the key features are dynamically fused by the gating mechanism, the capture of the emotion-related local channel collaborative mode is strengthened, the modeling ability of long-range time dependence is enhanced, the redundant information interference is reduced, and the pertinence and effectiveness of the feature representation are improved. (3) The domain adversarial module introduced by the decoder is forced to learn the shared features of the source domain and the target domain through minimax game optimization, significantly reduces the negative impact of the inter-subject data distribution offset, and solves the pain point of the performance of the traditional method on the new subject data. (4) The time embedding fuses the fine-grained time sequence features of the BiLSTM and the long-term trend features of the dilated convolution, the spatial embedding retains the topological information of the graph Laplacian matrix, and the gate fusion mechanism of the time-space hybrid embedding is combined, so that the multi-scale space-time characteristics of the electroencephalogram signal are comprehensively captured, and more rich semantic representations are provided for emotion recognition, and the model is helped to distinguish subtle emotional differences such as negative, neutral and positive. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 A flowchart of the cross-subject electroencephalogram emotion recognition method of the spatio-temporal adaptive graph coding learning of the embodiment of the present application is shown. Figure 2 A sample division schematic diagram of the sliding window with T=6 and an overlap of 4 seconds in the embodiment of the present application is shown. Figure 3 A schematic diagram of the overall performance comparison of the method of the embodiment of the present application and other models on the SEED dataset is shown. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be described clearly and completely below in combination with the embodiment of the present application. Obviously, the described embodiment is a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0040] Embodiment 1 The embodiment provides a cross-subject electroencephalogram emotion recognition method of spatio-temporal adaptive graph coding learning, and the specific implementation process is as shown in Figure 1 The embodiment provides a cross-subject electroencephalogram emotion recognition method of spatio-temporal adaptive graph coding learning, and the specific implementation process is as shown in S1, the sliding window technology is used to divide the electroencephalogram data sample after preprocessing and feature extraction, and then features are extracted from different frequency bands to construct a feature matrix covering channel and time dimensions; the specific implementation includes the following steps: S11, the sliding window technology is used to divide the electroencephalogram feature data after preprocessing, the number of samples is increased by overlapping windows, and each window contains a plurality of continuous 1-second time point data; features of 5 frequency bands are extracted from each window data after division, the 5 frequency bands include delta, theta, alpha, beta and gamma frequency bands, which correspond to the core dimensions of differential entropy features; in the embodiment, as Figure 2As shown, T=6, indicating that it contains 6 consecutive 1-second time point data, and the overlap time of the sliding window is 4 seconds.

[0041] S12, organize the extracted multi-band features into a three-dimensional feature matrix, where the 0th dimension is 62 EEG channels (consistent with the international 10-20 system electrode layout), the 1st dimension is the number of time points contained in the window, and the 2nd dimension is 5 frequency bands, forming a feature matrix covering the channel and time dimensions. The matrix can be directly used in the subsequent multi-dimensional data embedding step.

[0042] S2, through the multi-dimensional data embedding module, the feature preprocessing and mapper fuse the time embedding and spatial embedding extracted by the spatio-temporal hybrid embedding, BiLSTM and dilated convolution to convert the feature matrix into a high-dimensional semantic vector; the specific implementation includes the following steps: S21, encode the features based on historical data and current data encoding features fuse the information of the two through a gating mechanism to obtain a spatio-temporal hybrid embedding as follows:

[0043] wherein , is a learnable parameter matrix, a , b is a bias term, is a tanh activation function, is a sigmoid activation function, is a Hadamard product, wherein N is the number of channels, T is the number of time points, D is the hidden layer dimension, in this embodiment, D=64; S22, model the time dimension of the feature matrix using BiLSTM to capture bidirectional time sequence dependence, with an output dimension of extract time scale features through a dilated convolution layer with a dilation rate of 2, with a receptive field exponentially increasing with the number of layers, and an output dimension of ; after concatenating the outputs and linear transformation, combine the sine-cosine position encoding to obtain the time embedding , and ; the sine-cosine position encoding is as follows:

[0044]

[0045] S23, construct an adjacency matrix based on the electroencephalogram channel topology A, compute the normalized graph Laplacian matrix as follows:

[0046] where, I is the identity matrix, D is the degree matrix, is the eigenvalue matrix; take its eigenvector as the spatial embedding ; S24, add the spatio-temporal hybrid embedding , the temporal embedding , and the spatial embedding element by element to obtain a high-dimensional semantic vector as follows:

[0047] where, , B is the batch size.

[0048] S3, based on the high-dimensional semantic vector obtained in step S2, the adaptive spatio-temporal topology construction module of the graph structure generator calculates the channel feature difference and the temporal trend change to generate a dynamic topology matrix that adapts to the individual differences between subjects; the specific implementation includes the following steps: S31, perform joint modeling of nodes in the spatial and temporal dimensions on the high-dimensional semantic vector , and flatten the high-dimensional semantic vector containing 62 channels and T time points into a spatio-temporal node sequence composed of channel-time binary tuples, each node uniquely corresponding to the features of a specific channel at a specific time step; S32, in the spatial dimension, for each time step, calculate the feature correlation between the 62 channels through cosine similarity to generate a channel correlation matrix , which quantifies the functional synergy between different brain regions at the same time point, such as the frontal region and the parietal region; S33, in the temporal dimension, for each channel, calculate the trend correlation of T time steps through the Pearson correlation coefficient to generate a time correlation matrix ; capture the temporal evolution law of the emotional state within the same channel, such as the emotional arousal trend under video stimulation; S34, fuse the above two matrices through the Kronecker product to generate an initial global topology matrix , which has a dimension of , where the diagonal submatrix represents the channel correlation at the same time point, and the non-diagonal submatrix represents the channel interaction across time points, realizing the explicit fusion of spatio-temporal correlation; S35, introduce a learnable adjacency matrix , the node features are mapped to dimensions matching the number of nodes by a feature projection layer, and the initial global topology matrix is fused with the feature projection result, and then normalized by a Softmax function to obtain a dynamic topology matrix ; the feature projection result is an intermediate matrix after the high-dimensional semantic vector is mapped by the feature projection layer, which is a "bridge" connecting the high-dimensional semantic vector and the dynamic topology matrix. In this process, the learnable parameters are optimized through backpropagation, so that the topology matrix can adapt to the differences in EEG signal distribution of different subjects, such as compensating for individual differences in channel response intensity and emotional timing patterns between subjects. The final generated topology matrix not only retains the physiological topology prior of the EEG channel, such as the channel layout of the international 10-20 system, but also dynamically adapts to cross-subject differences through data-driven parameter learning, providing precise structural support for the subsequent encoder's spatio-temporal attention calculation, thereby improving the model's generalization ability for different subjects' emotional features.

[0049] S4, the encoder takes a dynamic spatio-temporal hybrid attention encoding module as the core, combines the dynamic topology matrix to calculate spatial cross-channel and temporal cross-step attention in parallel, and fuses and strengthens key spatio-temporal features through a gating mechanism; the specific implementation includes the following steps: S41, based on the dynamic topology matrix , a global spatio-temporal graph is constructed to convert the high-dimensional semantic vector into a global correlation matrix containing channel and time dimensions, where the diagonal submatrix represents the spatial correlation between all channels at the same time point, and the non-diagonal submatrix represents the cross-time correlation between all channels at different time points; S42, calculate the spatial cross-channel attention score and the temporal cross-step attention score ; The spatial cross-channel attention score is calculated as follows: Through the dynamic topology matrix , the spatial Q, K, and V matrices are generated as follows:

[0050]

[0051]

[0052] where is the dynamic topology matrix, is a learnable parameter, and the channel attention score is calculated by softmax as follows:

[0053] wherein T represents matrix transposition, d k denotes the feature dimension of the key.

[0054] The calculation of the time step length attention score The specific process is as follows: The sliding window mechanism is adopted to capture long-range time dependence, and time Q, K and V matrices are generated as follows:

[0055]

[0056]

[0057] wherein, is a time step length matrix, is a learnable parameter, and the time attention score is calculated as follows:

[0058] wherein T represents matrix transposition, d k denotes the feature dimension of the key.

[0059] S43, obtain the fusion features of the two types of attentions through a gating mechanism as follows:

[0060]

[0061] wherein is a gating parameter, is a concatenation operation; S44, fuse the fusion features through a multi-head attention module to aggregate multi-subspace features, as follows:

[0062] wherein, is a projection parameter, is a feature representation learned by the hth attention head in its exclusive subspace; S45, strengthen the feature stability through residual connection and layer normalization, as follows:

[0063]

[0064] S46, reduce the sequence length and retain key features through a distillation mechanism to strengthen the spatiotemporal key features, and obtain the encoded features, as follows:

[0065] wherein, is max-pooling operation, ELU is exponential linear unit, is one-dimensional convolution operation.

[0066] S5, the decoder maps the encoded features by means of the domain adversarial decoding module, reduces the distribution difference between the source domain and the target domain through cross-domain adversarial training, and outputs the emotion classification result. The specific implementation includes the following steps: S51, the encoded features output by the encoder are mapped to the emotion classification space through a fully connected layer to obtain preliminary classification features, as follows:

[0067] wherein, and are learnable weight matrix and bias term, is a classification feature vector; S52, a domain discriminator submodule is introduced for domain adversarial training, which includes a domain classifier, the input of which is the encoded features , and the output is the predicted result of the domain to which the features belong (source domain or target domain); the specific process of the adversarial training is as follows: (1) the adversarial training is carried out through minimax game, the feature mapping submodule of the decoder is committed to learning domain-invariant features, so that the domain discriminator cannot distinguish the source of the features (the target function is ); the domain discriminator is committed to accurately distinguishing the domain source (the target function is ); the domain loss function is as follows:

[0068] wherein, is the domain probability output by the domain discriminator, denotes the source domain sample, denotes the target domain sample; (2) the cross-entropy loss function is used to calculate the classification loss, as follows:

[0069] wherein, M is the number of samples, is an indicator function, is the true label of the sample, =-1, 1, 1 correspond to negative, neutral, and positive, is the probability that the sample j is predicted to be of class c; (3) Joint optimization is performed, parameters of the feature mapping sub-module and the domain discrimination sub-module are simultaneously optimized through back propagation until the model converges, and the total loss function is as follows:

[0070] wherein, is a balance coefficient.

[0071] S53, the decoder maps the input encoded features, obtains a three-class probability distribution through a softmax function, takes the class with the maximum probability, and outputs a sentiment classification result, i.e., negative (-1), neutral (0), or positive (1).

[0072] The embodiment also provides experimental simulation results of the above method, and the specific results are as follows: Accuracy (accuracy) is used to evaluate the performance of the model of the embodiment, and the calculation of Accuracy is as follows:

[0073] wherein, represents the number of samples of the c-th sentiment correctly predicted as c, represents the number of samples of non-c-th sentiment incorrectly predicted as c, represents the number of samples of the c-th sentiment incorrectly predicted as non-c. (c can take values 0, 1, 2, representing negative, neutral and positive respectively) The experimental data is derived from a public data set SEED and is verified.

[0074] For the SEED data set, 15 subjects are divided into a training set consisting of 14 subjects and a test set consisting of one subject, as shown in the following table and Figure 3 It can be seen that the method described in the embodiment is significantly better than the SVM algorithm and the random forest algorithm under the condition of adjusting different parameters (length of sliding window, number of seconds of sliding window overlap), which confirms its significant advantage in EEG emotion recognition.

[0075]

[0076] Embodiment 2 It should be further pointed out that based on the same inventive concept, the embodiment also provides a cross-subject EEG emotion recognition system based on spatiotemporal adaptive graph encoding learning, which uses the method described in embodiment 1 when running, including the following modules: The data preprocessing module is configured to divide the EEG data sample which has been preprocessed and extracted features by using a sliding window technique, and then extract features from different frequency bands to construct a feature matrix covering channel and time dimensions. The high-dimensional semantic vector construction module is configured to convert the feature matrix into a high-dimensional semantic vector by using a feature preprocessing and mapper, a multi-dimensional data embedding module, a time embedding extracted by a BiLSTM, and a space embedding extracted by an expansion convolution, and a space-time hybrid embedding. The dynamic topology matrix construction module is configured to calculate channel feature differences and time trend changes by an adaptive space-time topology construction module of a graph structure generator based on the high-dimensional semantic vector obtained by the high-dimensional semantic vector construction module, and generate a dynamic topology matrix adaptive to cross-subject individual differences. The space-time feature extraction module is configured to calculate spatial cross-channel and time cross-step attention in parallel by an encoder with a dynamic space-time hybrid attention encoding module as the core, combine a dynamic topology matrix, and fuse and strengthen key space-time features by a gating mechanism. The output module is configured to map the encoded features by a decoder with a domain adversarial decoding module, reduce the distribution difference between the source domain and the target domain by cross-domain adversarial training, and output a sentiment classification result.

[0077] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit the same; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A cross-subject electroencephalogram emotion recognition method based on spatio-temporal adaptive graph coding learning, characterized in that, The method comprises the following steps: S1, using a sliding window technique to divide the EEG data samples which have been preprocessed and feature extracted, and then extracting features from different frequency bands to construct a feature matrix covering channel and time dimensions; S2, using a feature preprocessing and mapper to fuse time-space hybrid embedding, BiLSTM and dilated convolution extracted time embedding and space embedding through a multi-dimensional data embedding module, and convert the feature matrix into a high-dimensional semantic vector; S3, based on the high-dimensional semantic vector obtained in step S2, calculating channel feature differences and time trend changes by an adaptive space-time topology construction module of a graph structure generator, and generating a dynamic topology matrix adapted to individual differences across subjects; S4, using an encoder with a dynamic space-time hybrid attention encoding module as the core to calculate spatial cross-channel and time cross-step attention in parallel in combination with the dynamic topology matrix, and fusing and strengthening key space-time features through a gating mechanism; S5, using a decoder with a domain adversarial decoding module to map the encoded features, reducing the distribution difference between the source domain and the target domain through cross-domain adversarial training, and outputting a sentiment classification result.

2. The cross-subject electroencephalogram emotion recognition method of spatio-temporal adaptive graph encoding learning according to claim 1, characterized in that, The step S1 comprises the following steps: S11, using a sliding window technique to divide the preprocessed EEG feature data, increasing the number of samples through overlapping windows, and each window containing a plurality of continuous 1-second time point data; extracting features of 5 frequency bands from each window data after division, the 5 frequency bands including delta, theta, alpha, beta and gamma bands, corresponding to the core dimensions of differential entropy features; S12, organizing the extracted multi-frequency band features into a three-dimensional feature matrix, wherein the 0th dimension is 62 EEG channels, the 1st dimension is the number of time points contained in the window, and the 2nd dimension is 5 frequency bands, forming a feature matrix covering channel and time dimensions.

3. The cross-subject electroencephalogram emotion recognition method of spatio-temporal adaptive graph encoding learning according to claim 2, characterized in that, The step S2 comprises the following steps: S21, encode features based on historical data and current data fuse both information through a gating mechanism, obtaining spatio-temporal hybrid embedding as follows: wherein , is a learnable parameter matrix, a , b is a bias term, is a tanh activation function, is a sigmoid activation function, is a Hadamard product, wherein N is a number of channels, T is a number of time points, D is a hidden layer dimension; S22, the time dimension of the feature matrix is modeled by BiLSTM to capture bidirectional time sequence dependence, and the output dimension is , the time scale feature is extracted by the dilated convolution layer with the expansion rate of 2, the receptive field increases exponentially with the layer number, and the output dimension is ; the outputs of the two are spliced and linearly transformed, combined with the sine and cosine position encoding, to obtain the time embedding , and ; S23. Constructing an adjacency matrix based on the electroencephalogram channel topology A , calculating a normalized graph Laplacian matrix as follows: wherein I is the identity matrix, D is the degree matrix, is the eigenvalue matrix; taking its eigenvectors as spatial embeddings ; S24, spatiotemporal hybrid embedding temporal embedding spatial embedding element-wise addition to obtain high-dimensional semantic vector as follows: wherein, , B is the batch size.

4. The cross-subject electroencephalogram emotion recognition method of spatio-temporal adaptive graph encoding learning according to claim 3, characterized in that, The step S3 comprises the following steps: S31, high-dimensional semantic vector The nodes of space and time dimensions are jointly modeled, and the high-dimensional semantic vector containing 62 channels and T time points is flattened into a space-time node sequence composed of channel-time binary tuples, each node uniquely corresponding to the characteristics of a specific channel at a specific time step; S32、Space dimension, for each time step, the cosine similarity is used to calculate the feature correlation between 62 channels to generate a channel correlation matrix S33、In the time dimension, for each channel, the trend correlation of T time steps is calculated by the Pearson correlation coefficient to generate a time correlation matrix ; S34, generating an initial global topology matrix by Kronecker product fusion of the above two matrices , the dimension of the matrix is , wherein the diagonal sub-matrix represents the channel association at the same time point, and the non-diagonal sub-matrix represents the channel interaction across time points, realizing explicit fusion of space-time association; S35、introducing a learnable adjacency matrix , the node features are mapped to dimensions matching the number of nodes by combining the feature projection layer, the initial global topology matrix is fused with the feature projection result, and then normalized by the Softmax function to obtain the dynamic topology matrix .

5. The cross-subject electroencephalogram emotion recognition method of claim 4, wherein, The step S4 comprises the following steps: S41、based on the dynamic topology matrix A global spatio-temporal graph is constructed to convert the high-dimensional semantic vector into a global correlation matrix containing channel and time dimensions, wherein a diagonal submatrix represents spatial correlation among all channels at the same time point, and a non-diagonal submatrix represents cross-time correlation among all channels at different time points. S42, compute spatial cross-channel attention scores and temporal cross-step attention scores ; S43, obtain the fusion feature of the two attentions through the gating mechanism as follows: wherein is a gating parameter, is a stitching operation; S44, fusing features Multi-subspace feature aggregation is performed by the multi-head attention module as follows: wherein, is a projection parameter, is the feature representation learned by the h-th attention head in its dedicated subspace; S45, strengthening feature stability through residual connection and layer normalization, as follows: S46, reducing sequence length and retaining key features through a distillation mechanism to strengthen key space-time features, and obtaining encoded features, as follows: wherein, is a max pooling operation, ELU is an exponential linear unit, is a one-dimensional convolution operation.

6. The cross-subject electroencephalogram emotion recognition method of learning spatio-temporal adaptive graph coding according to claim 5, characterized in that, calculating the spatial cross-channel attention score The specific process is as follows: By dynamic topology matrix Generate space Q, K, V matrix, as follows: wherein is a dynamic topology matrix, is a learnable parameter, and the inter-channel attention score is calculated by softmax as follows: where T denotes matrix transpose, d k denotes the feature dimension of the key.

7. The cross-subject electroencephalogram emotion recognition method of claim 5, wherein, The step S42 calculates the time step length attention score The specific process is as follows: Using a sliding window mechanism to capture long-range temporal dependencies, generating time Q, K and V matrices, as follows: wherein, is a time step length matrix, are learnable parameters, the time attention score is calculated as follows: where T denotes matrix transpose, d k denotes the feature dimension of the key.

8. The cross-subject electroencephalogram emotion recognition method of spatio-temporal adaptive graph encoding learning according to claim 5, characterized in that, The step S5 comprises the following steps: S51. The encoded features output by the encoder are processed through a fully connected layer. Mapping to the sentiment classification space yields preliminary classification features, as shown in the following formula: wherein, and are learnable weight matrices and bias terms, is a classification feature vector; S52, introduce the domain discrimination sub-module to perform domain adversarial training, the module includes a domain classifier, the input of which is the encoded features , and the output is the predicted result of the domain to which the features belong; S53, the decoder maps the input encoded features, obtains a three-class probability distribution through a softmax function, takes the class with the maximum probability, and outputs a sentiment classification result.

9. The cross-subject electroencephalogram emotion recognition method of spatio-temporal adaptive graph encoding learning according to claim 8, characterized in that, The adversarial training in step S52 has the following specific process: (1) performing adversarial training through a minimax game, the feature mapping submodule of the decoder is committed to learning domain-invariant features so that the domain discriminator cannot distinguish the feature source; the domain discriminator is committed to accurately distinguishing the domain source; the domain loss function is as follows: wherein, a domain probability output by the domain discriminator, denotes a source domain sample, denotes a target domain sample; (2) a classification loss is calculated using a cross-entropy loss function, as follows: where M is the number of samples, is an indicator function, is the true label of a sample, = -1, 1, 1 for negative, neutral, positive, respectively, is the predicted label of a sample j is the probability that a sample is of class c. (3) joint optimization, simultaneously optimizing the parameters of the feature mapping submodule and the domain discrimination submodule through back propagation until the model converges, and the total loss function is as follows: wherein is a balancing factor.

10. A cross-subject electroencephalogram emotion recognition system based on spatio-temporal adaptive graph encoding learning, characterized in that, The method comprises the following modules: The data preprocessing module is configured to divide the EEG data sample which has been preprocessed and extracted features by using a sliding window technique, and then extract features from different frequency bands to construct a feature matrix covering channel and time dimensions. The high-dimensional semantic vector construction module is configured to convert the feature matrix into a high-dimensional semantic vector by using a feature preprocessing and mapper, fusing time and space mixed embedding, time embedding and space embedding extracted by BiLSTM and dilated convolution through a multi-dimensional data embedding module. The dynamic topology matrix construction module is configured to calculate channel feature differences and time trend changes by an adaptive space-time topology construction module of a graph structure generator based on the high-dimensional semantic vector obtained by the high-dimensional semantic vector construction module, and generate a dynamic topology matrix adaptive to cross-subject individual differences. The space-time feature extraction module is configured to encode the key space-time features by a dynamic space-time hybrid attention encoding module as a core of an encoder, combining a dynamic topology matrix to calculate spatial cross-channel and time cross-step attention in parallel, and fusing and strengthening the key space-time features by a gating mechanism. The output module is configured to map the encoded features by a decoder with the aid of a domain adversarial decoding module, reduce the distribution difference between the source domain and the target domain by cross-domain adversarial training, and output a sentiment classification result.

Images