Multi-feature electroencephalogram emotion recognition method based on space-time attention mechanism

Abstract

The invention discloses a multi-feature electroencephalogram emotion recognition method based on a space-time attention mechanism, and the method comprises the following steps: preprocessing electroencephalogram source data, extracting a time domain feature, a frequency domain feature and a space domain feature, and generating a plurality of artificial electroencephalogram features; superposing and combining the extracted electroencephalogram features of the time domain, the frequency domain and the space domain in another dimension to form four-dimensional electroencephalogram feature data; the combined features are encoded through a space-time attention encoder to eliminate redundant information, and useful information is reserved; the method has the advantages that the electroencephalogram feature encoder based on the space-time attention is provided, input important information is paid attention to through multiple layers of space-time attention, and the purpose of eliminating redundant information is achieved; the encoder is connected with artificial features and deep learning features, so that a model can extract general emotion features through the artificial features, and then a deep similar emotion mode in electroencephalogram signals is obtained through a deep learning model.

Description

Technical Field

[0001] The present invention belongs to the technical fields of emotion recognition, artificial intelligence, and brain-computer interface, and specifically relates to a multi-feature EEG emotion recognition method based on a spatiotemporal attention mechanism. Background Art

[0002] Emotion recognition is an important part of computer perception of human emotions and human-computer interaction feedback. EEG can reflect the changes of various physiological electrical signals generated by brain nerve cells in the cerebral cortex by recording the changes of electrical signals of normal brain activities. Current research shows that EEG patterns will change with different physiological and psychological states. Therefore, it is feasible to use EEG for emotion recognition, which can truly reflect a person's emotional state.

[0003] Among various EEG-based emotion recognition methods, traditional machine learning methods include support vector machines, decision trees, and random forests. These methods are mainly based on manually extracted features, such as time domain features, frequency domain features, and time-frequency features, and then use shallow models for classification. The disadvantage of these traditional methods is that they have limited modeling capabilities for nonlinear relationships and are difficult to handle complex emotion recognition tasks. In recent years, deep learning models such as convolutional neural networks and recurrent neural networks have been mainly used for EEG emotion recognition. Convolutional neural networks can effectively learn local features and spatial relationships, and recurrent neural networks can capture contextual information on time series, but these deep learning models still have some challenges.

[0004] Traditional deep learning models face the problem of domain shift when dealing with EEG emotion recognition, that is, the model has poor generalization ability in emotion classification from one individual to another. This is because there are differences in EEG signals between different individuals, which leads to changes in data distribution; traditional models are difficult to adapt to the differences between different individuals, thus limiting the accuracy of emotion recognition; secondly, traditional models often ignore the impact of changes in local EEG signals on emotional states; EEG signals are non-stationary and time-varying, and local spatial features and time series features under different emotional states may have small but important changes. Traditional models cannot capture these small feature changes well, thus limiting the performance of emotion recognition; deep neural networks can automatically learn EEG classification end-to-end, but ignore the intrinsic relationship between EEG features and emotions, resulting in the interpretability of emotional features automatically extracted by deep learning models being inferior to manually designed emotional features; therefore, how to combine traditional EEG feature extraction methods with deep learning algorithms to improve the interpretability of the model and cross-subject emotion recognition performance is a current challenge. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism, which avoids complex feature engineering problems; it is conducive to discovering similar emotional patterns of different subjects in EEG signals, and can improve the accuracy of cross-subject emotion recognition.

[0006] To achieve the above object, the present invention provides the following technical solution: a multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism, comprising the following steps:

[0007] Preprocess EEG source data, extract time domain features, frequency domain features and space domain features, and generate multiple artificial EEG features;

[0008] The extracted EEG features in time domain, frequency domain and space domain are superimposed and combined in another dimension to form four-dimensional EEG feature data;

[0009] Encode the combined features through a spatiotemporal attention encoder to eliminate redundant information and retain useful information;

[0010] The encoded features are input into the emotion recognition model based on Swin Transformer for feature extraction and emotion classification;

[0011] Block partitioning, linear embedding, and block merging are used in the Swin Transformer model to process the input EEG features and output the sentiment classification results.

[0012] As a preferred technical solution of the present invention, the extracted time domain, frequency domain, and space domain EEG features are superimposed and combined in another dimension into four-dimensional EEG feature data. The four dimensions include single-channel features, multi-channel EEG, single feature dimensions, and multi-feature combinations.

[0013] As a preferred technical solution of the present invention, the time domain features include first-order differences, second-order differences, Hjorth mobility, Hjorth complexity and non-static index; the frequency domain features include differential entropy DE and power spectral density PSD; the spatial domain features include asymmetric difference DASM, asymmetric quotient RASM and asymmetric index AsI.

[0014] As a preferred technical solution of the present invention, when selecting artificial features representing emotions, an emotion recognition test is performed on each feature on the SEED dataset.

[0015] As a preferred technical solution of the present invention, the encoder connects artificial features and deep learning features, so that the model first extracts common emotional features through artificial features, and then obtains deep similar emotional patterns in EEG signals through the deep learning model.

[0016] As a preferred technical solution of the present invention, the Swin Transformer model uses a self-attention mechanism to capture tiny local features between EEG signals of different channels, and enhances the local receptive field of the model by reducing the size of image blocks.

[0017] As a preferred technical solution of the present invention, the channels and duration of the source data are selected and processed so that the EEG data conforms to the input size of the Swin Transformer model.

[0018] As a preferred technical solution of the present invention, the encoded features are input into the emotion recognition model based on SwinTransformer, including:

[0019] The input feature size is a set of 56 EEG channels, the single channel feature length is 56, and the feature depth is determined by the combined original feature dimension; after block partitioning into different blocks of size 28×28×4n, and a linear embedding layer, a data format suitable for input into the Transformer is obtained.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] A spatiotemporal attention-based EEG feature encoder is proposed. It focuses on important information of the input through multi-layer spatiotemporal attention, thereby eliminating redundant information. The encoder connects artificial features and deep learning features, so that the model can first extract common emotional features through artificial features, and then obtain deep similar emotional patterns in EEG signals through the deep learning model.

[0022] This method uses the Swin Transformer model as the backbone network and uses its self-attention mechanism to capture the tiny local features between EEG signals in different channels. By reducing the size of the image block, the local receptive field of the model is enhanced and the number of model parameters is reduced, which has high accuracy and real-time performance.

[0023] This method first extracts artificial EEG features, and then uses a deep learning model to extract deep EEG features for emotion classification. This is different from the previous artificial features + classifiers or end-to-end deep learning EEG emotion classification. Instead, it combines traditional machine learning artificially extracted features with deep learning feature classification in the emotion recognition task of the emotion brain-computer interface to improve the accuracy of multi-feature EEG emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a framework diagram of multi-feature EEG emotion recognition based on spatiotemporal attention mechanism of the present invention;

[0025] Figure 2The Swin Transformer model module structure and the spatial / temporal attention module structure diagram of the present invention;

[0026] Figure 3 This is a flow chart of the identification method of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Example 1

[0029] Figure 3 is a flow chart of the identification method of the present invention; Figure 1 The multi-feature EEG emotion recognition framework based on spatiotemporal attention mechanism of the present invention is presented, which consists of three parts: Artificial EEG Features, Feature Fusion and Emotion Recognition Model;

[0030] In the artificial EEG features, temporal features, frequency features and spatial features are extracted from the preprocessed EEG source data to obtain different artificial EEG features.

[0031] In feature fusion, the extracted EEG features in time, frequency and space domains have three dimensions (3-DEEGFeatures), which are combined into four-dimensional EEG feature data (4-D EEGFeatures) by superimposing them on another dimension; these four dimensions include single-channel features, multi-channel EEG, single feature dimensions and multi-feature combinations;

[0032] In the emotion recognition model, the combined features will be eliminated through the spatial-temporal attention encoder (Spatial Attention Encoder, Temporal Attention Encoder) to eliminate redundant information, and then input into the Swin Transformer emotion recognition model; the input feature size is a group of 56 EEG channels, the single channel feature length is 56, and the feature depth is determined by the original feature dimension of the combination; after the patch partitioning (Patch Partition) is divided into different blocks of 28×28×4n size, and the linear embedding layer (Linear Embedding) is used to obtain the data format suitable for input into the Transformer; with the depth of the network structure, the input size is reduced and the feature depth is increased by using the patch merging (Patch Merging), and then the deep learning features are extracted through the Swin Transformer basic structure block; finally, according to the number of categories of the emotion classification task, the fully connected layer (MLP Head) is changed to achieve emotion multi-classification;

[0033] EEG Artificial Features:

[0034] In order to improve the interpretability of emotion recognition and combine the traditional EEG feature extraction method with the deep learning algorithm, the following artificial features were experimented, including time domain, frequency domain and space domain; time domain features include first-order difference, second-order difference, Hjorth mobility, Hjorth complexity and non-static index (NSI); frequency domain features include differential entropy (DE) and power spectral density (PSD); spatial domain features include asymmetric difference (DASM), asymmetric quotient (RASM) and asymmetric index (AsI); the most suitable artificial features were selected through experiments. Representative features are used to represent EEG artificial emotion features. Experiments are conducted from the following artificial features, including frequency domain and space domain. Since sequence information also contains time information, artificial features essentially also have time domain information. Frequency domain features include DE and PSD, and space features include AsI. According to frequency domain features, EEG is divided into five frequency bands: δ (0.5-4Hz), θ (4-8Hz), α (8-13Hz), β (13-30Hz) and γ (30-47Hz). The calculation formulas for DE value, PSD value and AsI value are as follows:

[0035]

[0036] Among them, σ2 represents the variance of the signal x, μ represents the mean value of the signal, n represents the number of possible states, and pi represents the probability of the i-th state;

[0037]

[0038] Where x(n) represents the discrete sampling value of the input signal, j is the imaginary unit, k is the frequency domain sampling point index, N is the number of sampling points, and PSD is the result of normalizing the square of the Fourier transform amplitude;

[0039]

[0040] Among them, PR represents the power of the EEG signal from the electrode on the right side of the brain, PL represents the power of the EEG signal from the electrode on the left side of the brain, and AsI is an indicator to measure the degree of asymmetry of activity between the two hemispheres of the brain, with a value range of -1 to 1;

[0041] In order to make the EEG data conform to the input size of the Swin Transformer model, the channels and duration of the source data need to be selected and processed. The size of the model input is (56, 56). The 56 in the first dimension means that 56 EEG channels will be selected. Because the frontal lobe of the brain is the main part that controls people's emotions and is closely related to emotions, for example, among the 62 EEG devices, 6 channels of the non-frontal lobe part that have little effect on the EEG characteristics are removed: P7, P8, PO7, PO8, CB1 and CB2. The 56 in the second dimension means that the EEG channel under one time slice The channel feature length is 56. Each experiment collected 175 seconds of EEG data of the subjects induced by a single emotion, totaling 35,000 EEG sampling points. The data was divided into 25 groups according to 7-second time slices, and each group had 1,400 EEG sampling points. Taking differential entropy as an example, we calculated the differential entropy feature once every 25 sampling points, so the time slice differential entropy feature is a vector with a length of 56. After combining 56 channels and 5 frequency bands, the feature size of the 7-second time slice is (56,56,5). These features are superimposed in dimension according to the number of frequency bands or feature groups;

[0042] Spatiotemporal Attention Module:

[0043] The spatial and temporal attention are realized by replacing multiple layers of attention layers. Spatial attention is an operation that uses convolution and deconvolution to focus the input features on the important information of each feature layer, thereby eliminating redundant information in space. Temporal attention uses a multi-layer perceptron (MLP) to focus on the time series data of each channel of EEG data by changing the arrangement method. It focuses on the important data in the time series from the perspective of each EEG channel and eliminates redundant information in time. The data passing through the spatiotemporal attention module is not directly input into the model features, but is a secondary feature generated based on the artificial features. It concentrates more useful information into the spatiotemporal attention module, indirectly reducing the noise impact in the EEG data.

[0044] exist Figure 2In the figure, a is two consecutive Swin Transformer modules; Layer Normalization (LN) can standardize the hidden layer input of the deep neural network, which can effectively improve the training speed; W-MSA and SW-MSA are multi-head self-attention modules, which use conventional and sliding window attention mechanisms respectively; finally, the multi-layer perceptron (MLP) output is used to calculate the features after self-attention; b is the spatial / temporal attention block; spatial attention Attention is implemented through 7x7 convolution. The first 7x7 convolution block is used to scale the first dimension of the size C×T×F to C / r×T×F, and then another 7x7 convolution block is used to deconvolve the intermediate features back to the size of C×T×F. Temporal Attention is processed by a single-channel multi-layer perceptron (MLP). Permutation is first used to change the order of the dimensions of C×T×F to T×F×C, so that the MLP can directly process the information of the temporal dimension, and then the dimensions are changed back to C×T×F through reverse permutation.

[0045] Emotion Recognition Model:

[0046] To adapt to the input structure of the Swin Transformer model, the EEG feature size of the input model is set to 56×56. The size of the input feature is determined by the specific feature or feature combination. If it is assumed to be n-dimensional, the size is converted to 28×28×4n through a 2×2 patch. According to the structure of the EEG artificial feature, each 2×2 block contains feature data of adjacent time and adjacent channels. Block merging is similar to the merging layer of CNN, which enables the model to generate multi-scale features. As the model structure deepens, the field of view of each block will be doubled through block merging, making the representation ability stronger. The linear embedding layer increases the feature dimension to 8n, improving the ability to distinguish different features. In the basic module of Transformer, a conventionally configured multi-head self-attention module (W-MSA) is used to calculate the self-attention within a 2×2 block; then, a sliding window configuration (SW-MSA) multi-head self-attention module is used for interaction between windows and to link the features of different patches; the efficiency is improved by limiting the self-attention mechanism to non-overlapping local windows and allowing cross-window connections between sliding windows; this hierarchical structure has the flexibility of modeling at different scales and linear computational complexity relative to the image size, which can improve the running speed of the model; after using the linear embedding layer, the single-layer features are straightened into sequences and the channel direction is normalized using layer normalization (LN); finally, the output is sent to the next module through MLP; since the original feature input size is 56, this method modifies the module of the Swin Transformer model, deletes the last block merging and Swin Transformer structure blocks, and reconnects the final output to the fully connected layer for EEG emotion recognition;

[0047] Experimental dataset:

[0048] The model effect of this method is verified on the SEED and SEED-IV datasets; the SEED dataset contains EEG data collected from 15 subjects while watching movie clips; the movie clips evoke different types of emotions in the subjects: positive, neutral and negative; the dataset provides EEG data of 15 subjects, each subject undergoes three rounds, a total of 45 trials; a single EEG data point contains EEG recorded from 62 channels, which are downsampled from the original 1000Hz EEG data to 200Hz; the SEED-IV dataset contains four different emotion categories, including happiness, sadness, fear and neutral emotions; the experiment consists of 15 subjects; the EEG signals are cut into 4-second non-overlapping segments and downsampled to 128Hz;

[0049] Experimental parameter settings:

[0050] In order to study the emotion recognition ability of the multi-feature EEG emotion recognition method based on the spatiotemporal attention mechanism in single-subject and cross-subject, experiments were designed and compared with other models; in the single-subject emotion recognition experiment, the ratio of training set to test set was 4:1; the single-subject accuracy means that 15 subjects were tested separately, and the average accuracy of all subjects was taken as the single-subject accuracy of the feature; in the cross-subject experiment, the subjects were divided into 5 groups, each group was used as the target domain for model testing, and when one group was used as the target domain, the other groups were used as training sets for 10-fold cross-validation training; the average accuracy of the 5 models trained in this way is the cross-subject accuracy; the ablation experiment verified the contribution of the spatiotemporal attention module to improving the emotion recognition ability of the model; the model was implemented using the PyTorch framework and trained using the AdamW optimizer with a learning rate of 1e-5, and the learning rate decay adopted the cosine annealing method. The batch size used during training was 32, and the model was trained for 100 rounds;

[0051] When selecting artificial features to represent emotions, each feature was tested for emotion recognition on the SEED dataset. In the combination of multiple features, multiple features with high accuracy that can represent emotions were selected. The DE feature had the highest single-subject accuracy (90.24%), and the PSD feature had the highest cross-subject accuracy (82.16%). The emotion recognition accuracy of each feature is shown in Table 1. Combining the characteristics and test accuracy of different features, we selected three features with a single subject accuracy higher than 80%, namely DE and PSD in the frequency domain and AsI in the spatial domain, to form a representative fusion feature input into the spatiotemporal attention encoder.

[0052] Table 1. EEG emotion recognition accuracy of different artificial features on SEED dataset

[0053] feature Single subject accuracy / % Accuracy across subjects / % First Difference 45.82 38.76 Second order difference 44.46 38.12 HjorthMobility 43.63 36.78 Hjorth Complexity 66.71 52.33 NSI 41.87 33.46 DE 90.24 81.62 PSD 89.32 82.16 DASM 73.31 61.24 RASM 76.89 62.83 AsI 81.43 72.38

[0054] Effects of spatiotemporal attention:

[0055] Table 2 shows the comparison of EEG emotion recognition accuracy across subjects in the SEED dataset in the past five years; the EEG features constructed by multiple artificial features are used as the input of the model, and a high EEG emotion recognition accuracy is achieved in both single-subject and cross-subject models; the spatiotemporal attention mechanism is used to reduce the noise in the input artificial features, and the multi-layer spatiotemporal attention focuses on the important information of the input, which plays a role in eliminating redundant information; the input of the model integrates different features, so the emotion recognition accuracy is higher than that of a single modality; in single-subject emotion recognition, compared with the single-modality model inputs such as STNN, R2G-STNN, BiDANN, BiHDM and ATDD-LSTM, the model uses a 7-second time slice to calculate each feature, and improves the model's emotion recognition ability by integrating multiple effective features; the single-subject emotion recognition accuracy of the model is 94.13%, and the cross-subject emotion recognition accuracy is 89.33%, and it can achieve real-time processing speed;

[0056] Table 2 Comparison of EEG emotion recognition accuracy on the SEED dataset

[0057] Model Single subject accuracy / % Accuracy across subjects / % STNN 89.50 - BiDANN 92.38±7.04 83.28±9.60 R2G-STNN 93.38±5.96 84.16±7.63 BiHDM 93.12±6.06 85.40±7.53 ATDD-LSTM 91.08±5.34 90.92±1.05 DGCNN 90.40±8.49 79.95±9.02 CLISA - 86.40±6.40 RGNN 94.24±5.95 85.30±6.72 STST(Ours) 94.13±2.11 89.33±4.37

[0058] From the ablation experiment results in Table 3, we can see that the Swin Transformer model using spatiotemporal attention is more effective in EEG emotion recognition, and spatial attention has a greater impact on model performance than temporal attention. The reason is that for EEG data, it is more necessary to associate EEGs from different channels to improve the expressiveness of EEG features. We found that the number of spatiotemporal attention layers also affects the effect of the model. Without affecting the speed of the model, 8 layers of spatiotemporal attention modules are stacked to achieve a balance between accuracy and operating efficiency.

[0059] Table 3 Ablation experiment on the spatiotemporal attention mechanism on the SEED dataset

[0060] Spatial Attention Time Attention Single subject accuracy / % Accuracy across subjects / % - - 89.78±4.63 83.74±6.27 √ - 93.28±2.78 88.57±5.13 - √ 91.08±3.89 85.17±5.78 √ √ 94.13±2.11 89.33±4.37

[0061] Emotion recognition accuracy:

[0062] Table 4 shows the comparison of emotion recognition accuracy of all models on SEED and SEED-IV datasets; the regularized graph neural network RGNN takes into account the biological topological structure between different brain regions to capture the local and global relationships between different EEG channels; in the three-category classification task, the accuracy of RGNN reached 94.24%; the model result is very close to the accuracy of RGNN; the single-subject emotion recognition accuracy of this model on the SEED dataset reached 94.13%, and the standard deviation in repeated experiments was low; in the four-category classification task, the model achieved state-of-the-art performance; compared with other similar models, it has the highest accuracy on the SEED-IV dataset, reaching 82.34%;

[0063] In terms of model speed, the test speed of the model can reach the level of real-time processing; compared with the 29M model parameters and 4.5G FLOPs of the original Swin-T model, the improved model is reduced to 13M model parameters and 2.4G FLOPs while maintaining the accuracy of emotion recognition; in the EEG emotion recognition test under experimental training conditions, the average processing time of 7 seconds of EEG is 0.96 seconds, which shows that it is easier to achieve the level of real-time processing in some environments with weak computing power and future practical applications;

[0064] Table 4 Comparison of single-subject EEG emotion recognition accuracy on SEED and SEED-IV datasets

[0065]

[0066]

[0067] Although the embodiments of the present invention have been shown and described, as detailed above, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism, characterized by: The steps include: Preprocess EEG source data, extract time domain features, frequency domain features and space domain features, and generate multiple artificial EEG features; The extracted EEG features in time domain, frequency domain and space domain are superimposed and combined in another dimension to form four-dimensional EEG feature data; Encode the combined features through a spatiotemporal attention encoder to eliminate redundant information and retain useful information; The encoded features are input into the emotion recognition model based on Swin Transformer for feature extraction and emotion classification; Block partitioning, linear embedding, and block merging are used in the Swin Transformer model to process the input EEG features and output the sentiment classification results.

2. The multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism according to claim 1 is characterized by: The extracted EEG features in time domain, frequency domain and space domain are superimposed and combined in another dimension into four-dimensional EEG feature data. The four dimensions include single-channel features, multi-channel EEG, single feature dimension and multi-feature combination.

3. The multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism according to claim 1 is characterized in that: The time domain features include first-order difference, second-order difference, Hjorth mobility, Hjorth complexity and non-static index; the frequency domain features include differential entropy DE and power spectral density PSD; the space domain features include asymmetric difference DASM, asymmetric quotient RASM and asymmetric index AsI.

4. The multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism according to claim 1, characterized in that: According to the frequency domain characteristics, the EEG is divided into five frequency bands: δ (0.5-4 Hz), θ (4-8 Hz), α (8-13 Hz), β (13-30 Hz) and γ (30-47 Hz). The calculation formulas of DE value, PSD value and AsI value are as follows: Among them, σ2 represents the variance of the signal x, μ represents the mean value of the signal, n represents the number of possible states, and pi represents the probability of the i-th state; Where x(n) represents the discrete sampling value of the input signal, j is the imaginary unit, k is the frequency domain sampling point index, N is the number of sampling points, and PSD is the result of normalizing the square of the Fourier transform amplitude; Among them, PR represents the power of the EEG signal from the electrode on the right side of the brain, PL represents the power of the EEG signal from the electrode on the left side of the brain, and AsI is an indicator to measure the degree of asymmetry of activity between the two hemispheres of the brain, with a value range of -1 to 1.

5. The multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism according to claim 1, characterized in that: When selecting artificial features to represent emotions, each feature was tested for emotion recognition on the SEED dataset.

6. The multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism according to claim 1, characterized in that: The encoder connects artificial features and deep learning features, so that the model first extracts common emotional features through artificial features, and then obtains deep similar emotional patterns in EEG signals through the deep learning model.

7. The multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism according to claim 1, characterized in that: The Swin Transformer model uses a self-attention mechanism to capture tiny local features between EEG signals of different channels and enhances the local receptive field of the model by reducing the size of image blocks.

8. The multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism according to claim 1, characterized in that: The channels and duration of the source data were selected and processed so that the EEG data fit the input size of the SwinTransformer model.

9. The multi-feature EEG emotion recognition method based on spatiotemporal attention mechanism according to claim 1, characterized in that: The encoded features are input into the emotion recognition model based on Swin Transformer, including The input feature size is a set of 56 EEG channels, the single channel feature length is 56, and the feature depth is determined by the combined original feature dimension; after block partitioning into different blocks of size 28×28×4n, and a linear embedding layer, a data format suitable for input into the Transformer is obtained.

Images