An electroencephalogram signal emotion classification method, device, medium and equipment

Abstract

The application discloses an electroencephalogram signal emotion classification method and device, a medium and equipment, relates to the electroencephalogram signal recognition technical field, and includes the differential entropy information of the original electroencephalogram signal EEG of different channels according to acquisition, determines the differential entropy information of the electroencephalogram signal of each channel in five frequency bands, and takes it as the electroencephalogram signal feature;The input electroencephalogram signal feature is classified through the pre-trained enhanced electroencephalogram emotion recognition network PG3DAMT model, and the electroencephalogram signal emotion classification result is obtained;Wherein, the PG3DAMT model includes the parallel 3D graph convolution Para-G3D module, the attention-multiscale time convolution AT-MSTCN module, the Transformer module and the full connection layer connected in turn.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal recognition, and in particular to a method, apparatus, medium, and device for classifying EEG signals for emotion. Background Technology

[0002] As a core component of human psychological activity, the accurate identification of emotions is of great significance for fields such as human-computer interaction, mental health monitoring, and emotional education. Electroencephalography (EEG), a physiological signal that directly reflects the activity of neurons in the brain, has advantages such as high temporal resolution and convenient acquisition, and has become an important data source for emotion recognition.

[0003] With the continuous improvement of feature extraction capabilities of various neural networks, research on directly learning temporal and spatial information from EEG has received widespread attention. Among them, graph neural networks (GNNs), with their powerful modeling capabilities for non-Euclidean data, provide a new method for mining emotional features in EEG signals. Du et al. proposed a multidimensional graph convolutional network (MD-GCN), which extracts spatial features by constructing an adjacency matrix using the asymmetry of neuronal activity in the left and right hemispheres for emotion classification. Li et al. proposed a graph fusion model (EGFG) that extracts spatial features through graph convolution to achieve emotion classification. Yang et al. proposed the DC-ASTGCN model, which extracts and fuses temporal and spatial features of EEG signals through a dual-branch structure, effectively combining multi-dimensional features and improving recognition capabilities. Wang et al. proposed a spatiotemporal feature fusion neural network (STFFNN), which learns spatial and temporal features of brain regions through two independent sub-modules and achieves feature fusion through a fusion module. Feng et al. proposed the ST-GCLSTM model, which learns the intrinsic correlation of EEG channels through two layers of graph convolution, combines Bi-LSTM to capture temporal dependence, and introduces an attention mechanism to dynamically highlight key spatiotemporal features, enhancing sensitivity to important time segments.

[0004] However, the above methods only focus on spatial feature extraction or separate spatiotemporal features, making it difficult to deeply explore the cross-spatiotemporal correlation patterns of EEG signals, resulting in the loss of key emotional features and thus affecting the accuracy of EEG signal emotion classification. Summary of the Invention

[0005] This invention provides a method, apparatus, medium, and device for classifying emotions using electroencephalogram (EEG) signals, to address the aforementioned problems in the prior art, namely, how to improve the accuracy of EEG signal emotion classification in the prior art. This invention provides a method for classifying emotions using EEG signals, which includes:

[0006] Based on the acquired raw EEG signals from different channels, the differential entropy information of the EEG signal in each channel in five frequency bands is determined and used as the features of the EEG signal.

[0007] The PG3DAMT model, an enhanced EEG emotion recognition network, is pre-trained to classify the emotions of input EEG signal features, thus obtaining the emotion classification results. The PG3DAMT model includes a parallel 3D graph convolutional Para-G3D module and an attention-multi-scale temporal convolutional AT-MSTCN module, followed by a feature fusion layer, a Transformer module, and a fully connected layer. The Para-G3D module is used to construct the spatiotemporal graph. The system aggregates the input EEG signal features using a spatiotemporal graph to determine the corresponding spatiotemporal features. The Attention-Multi-Scale Temporal Convolution (AT-MSTCN) module is used to determine the multi-scale temporal information corresponding to the input EEG signal features based on an attention mechanism. The feature fusion layer performs a fusion operation on the spatiotemporal features and the multi-scale temporal information to determine the fused features. The Transformer module is used to obtain the fused features of the input through a multi-head attention mechanism and to concatenate and fuse the fused features to obtain global features. The fully connected layer performs a linear transformation on the global features to obtain the EEG signal emotion classification result.

[0008] Optionally, the Para-G3D parallel 3D graph convolution module specifically includes:

[0009] The spatiotemporal graph construction module TSGC and a 3D convolution are connected sequentially. The spatiotemporal graph construction module TSGC divides the sliding time window and adds cross-spatiotemporal edges to generate a spatiotemporal graph that integrates the time dimension and spatial layout. The spatiotemporal graph is used by the 3D convolution to aggregate the features of the input EEG signal and determine the corresponding spatiotemporal features. The spatiotemporal graph includes multiple nodes and an adjacency matrix to represent the adjacency relationship between nodes.

[0010] Optionally, the acquisition of the adjacency matrix used to represent the adjacency relationship between nodes in the spatiotemporal graph specifically includes:

[0011] Constructing a spatial topology matrix By using the spatial topology matrix Tiling W times yields a spatiotemporally fixed adjacency matrix for capturing spatiotemporal connectivity features of EEG signals. A w ;

[0012] Based on spatiotemporal fixed adjacency matrix A wCombined with a pre-set, self-learning dynamic adjacency matrix The two are combined linearly to form an adjacency matrix for representing the adjacency relationships between nodes. Specifically, it includes:

[0013] ;

[0014] in, w Size of the sliding time window R For the real number field, C This represents the number of electrode channels.

[0015] Optionally, the attention-multi-scale temporal convolution AT-MSTCN module specifically includes:

[0016] The system consists of a sequentially connected attention module and four parallel convolutions with different dilation coefficients; wherein, the attention module uses multiple two-dimensional convolutions to map spatiotemporal features to a high-dimensional feature space, and... The activation function generates attention weights corresponding to the spatiotemporal features. The mapped spatiotemporal features are weighted with the corresponding attention weights to determine the weighted features. Through convolutions with different dilation coefficients, features at different time scales in the weighted features are obtained to determine the corresponding multi-scale temporal information.

[0017] Optionally, determining the differential entropy information of the EEG signal in five frequency bands for each channel based on the acquired raw EEG signals from different channels specifically includes:

[0018] Based on the raw EEG signals acquired from different channels, the raw EEG signals were segmented into multiple segments using the sliding window technique.

[0019] Each segment is further divided into multiple sub-segments using the sliding window technique, and for each sub-segment, the differential entropy information of each frequency band is determined.

[0020] Optionally, the PG3DAMT model can be trained using the constructed SEED and DEAP datasets to determine a pre-trained PG3DAMT model.

[0021] This invention provides a brainwave signal emotion classification device, comprising:

[0022] The acquisition module is used to determine the differential entropy information of the EEG signal in five frequency bands for each channel based on the acquired raw EEG signals from different channels, and use it as the features of the EEG signal.

[0023] A classification module is used to classify the input EEG signal features into emotions using a pre-trained enhanced EEG emotion recognition network (PG3DAMT) model, obtaining the emotion classification result. The PG3DAMT model includes a parallel 3D graph convolutional (Para-G3D) module and an attention-multi-scale temporal convolutional (AT-MSTCN) module, followed by a feature fusion layer, a Transformer module, and a fully connected layer. The Para-G3D module is used to construct... A spatiotemporal graph is constructed, and the input EEG signal features are aggregated using the spatiotemporal graph to determine the corresponding spatiotemporal features. The Attention-Multi-Scale Temporal Convolution (AT-MSTCN) module is used to determine the multi-scale temporal information corresponding to the input EEG signal features based on the attention mechanism. The feature fusion layer performs a fusion operation on the spatiotemporal features and the multi-scale temporal information to determine the fused features. The Transformer module is used to obtain the fused features of the input through a multi-head attention mechanism, and then concatenates and fuses the fused features to obtain global features. The fully connected layer performs a linear transformation on the global features to obtain the EEG signal emotion classification result.

[0024] The present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described EEG signal emotion classification method.

[0025] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-mentioned EEG signal emotion classification method.

[0026] Compared to existing technologies, the beneficial effects of this invention are as follows: This invention provides a method for classifying emotions in electroencephalogram (EEG) signals. This method classifies the emotions of input EEG signal features using a constructed enhanced EEG emotion recognition network (PG3DAMT) model. Specifically, the Para-G3D module of the PG3DAMT model is used to construct a spatiotemporal graph that integrates temporal dimensions and spatial layout, capturing complementary spatiotemporal correlation patterns in EEG signals to enrich the model's ability to express complex EEG patterns, thereby enhancing the comprehensiveness of the spatiotemporal features of EEG signals. Simultaneously, the Attention-Multi-Scale Temporal Convolutional Network (AT-MSTCN) module in the PG3DAMT model utilizes an attention mechanism to dynamically learn emotion-related information and captures emotional features at different time scales through multi-scale temporal convolution. Furthermore, considering that emotional cognition is a continuous dynamic process, this invention introduces a Transformer module. This module, through a multi-head self-attention mechanism, can model the dependencies between features at different times in a long time series, thereby constructing a global temporal representation of emotional evolution and improving the accuracy of EEG signal emotion classification. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0028] Figure 1 A flowchart of an EEG signal emotion classification method provided in an embodiment of the present invention;

[0029] Figure 2 A framework diagram of the PG3DAMT model, an enhanced EEG emotion recognition network, provided in an embodiment of the present invention;

[0030] Figure 3 This is a Para-G3D module diagram provided in an embodiment of the present invention;

[0031] Figure 4 The AT-MSTCN module provided in this embodiment of the invention;

[0032] Figure 5 This is a t-SNE feature distribution diagram of the subjects before and after inputting the model in the SEED dataset provided in this embodiment of the invention;

[0033] Figure 6 The t-SNE feature distribution diagram of the subjects before and after inputting the model in the DEAP(V) dataset provided in this embodiment of the invention;

[0034] Figure 7 A schematic diagram of a computer device for the EEG signal emotion classification method provided in an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0036] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0037] Figure 1 This is a flowchart of an EEG signal emotion classification method provided in an embodiment of the present invention, such as... Figure 1 As shown in this embodiment, a method for classifying emotions using electroencephalogram (EEG) signals includes:

[0038] S1: Based on the acquired raw EEG signals from different channels, determine the differential entropy information of the EEG signal in five frequency bands for each corresponding channel, and use it as the EEG signal feature.

[0039] Generally, emotional cognition is a dynamic and evolving process. Emotional features in EEG signals are often embedded in temporal changes; therefore, modeling the temporal context is crucial for EEG emotion recognition. Recurrent Neural Networks (RNNs) and their variants (such as Long Short-Term Memory Networks (LSTM) and Gated Recurrent Units (GRUs)) capture temporal dependencies through memory units and are widely used for EEG temporal feature modeling. For example, existing techniques segment EEG signals into time windows and then use LSTM to model the feature sequences of each window to capture dynamic changes in emotion.

[0040] Optionally, the PG3DAMT model can be trained using the constructed SEED dataset and DEAP dataset to determine the pre-trained PG3DAMT model.

[0041] For example, this invention can train the PG3DAMT model using two public EEG datasets: the SEED dataset and the DEAP dataset. The SEED dataset is an EEG emotion dataset constructed by the Brain-Inspired Computing and Machine Intelligence Laboratory (BCMI) at Shanghai Jiao Tong University. This dataset consists of 15 Chinese subjects, each participating in three experiments. In these experiments, emotional responses were induced by watching 15 clips from Chinese films (containing positive, negative, and neutral emotions, each approximately 4 minutes long). After the experiments, subjects were asked to complete a questionnaire to obtain emotional feedback. Data acquisition used a 62-channel 10-20 system with an initial sampling rate of 1000Hz, which was then downsampled to 200Hz. The DEAP dataset recorded the EEG signals and peripheral physiological signals of 32 subjects while watching 40 one-minute music video clips. After each experiment, evaluation data was recorded from four dimensions: valence, arousal, dominance, and liking, with a rating range of 1 to 9. In this invention, the dimensions of valence and arousal were used for emotion classification. Using a 32-channel EEG device with a 10-20 system, EEG signals were recorded at a sampling rate of 512 Hz, and then downsampled to 128 Hz. In this invention, by using preprocessed data provided by the authors, baseline data from the first 3 seconds of the experiment was removed, and 60 seconds of video was selected as the experimental data.

[0042] For example, to enable the model to more fully learn the spatiotemporal details of EEG, this invention designs a time-map input representation through two time window divisions. First, this invention uses a sliding window to segment the original EEG signal into short segments. Then, these short segments are further divided into even shorter sub-segments. For an EEG signal... ,in, C This represents the number of electrode channels. T Represents the total number of time points, using a length of l 1 =20 s The step size is s1=4 s The sliding window divides X into Then, for each Use length l 2 = 2 s The sliding window with a step size of s2 = 0.5s is divided into... This invention relates to... Differential entropy (DE) is calculated as a feature of EEG signals. Specifically, the EEG signal of each channel is divided into five frequency bands: (1-4Hz) (4-8Hz) (8-13Hz) (13-30Hz) and (30-50Hz), and calculate the differential entropy information for each frequency band to obtain ;

[0043] in, f For frequency band characteristic numbers, These are the characteristics of raw electroencephalogram (EEG) signals. The characteristics of the EEG signal after X is segmented. To The segmented EEG signal characteristics To The characteristics of differential entropy obtained by calculating differential entropy N The number of samples in the first split. M This represents the number of segments in the second division. f For frequency band features.

[0044] Considering that the DEAP dataset has already been filtered Frequency band, therefore in the DEAP dataset f =4, in the SEED dataset f =5.

[0045] S2: The pre-trained Enhanced EEG Emotion Recognition Network (PG3DAMT) model is used to classify the input EEG signal features for emotion, obtaining the EEG signal emotion classification result. The PG3DAMT model includes a parallel 3D graph convolutional (Para-G3D) module and an attention-multi-scale temporal convolutional (AT-MSTCN) module, followed by a feature fusion layer, a Transformer module, and a fully connected layer. The Para-G3D module is used to construct the temporal... The system first generates an empty graph and then aggregates the input EEG signal features using a spatiotemporal graph to determine the corresponding spatiotemporal features. The Attention-Multi-Scale Temporal Convolution (AT-MSTCN) module is used to determine the multi-scale temporal information corresponding to the input EEG signal features based on an attention mechanism. The feature fusion layer performs a fusion operation on the spatiotemporal features and multi-scale temporal information to determine the fused features. The Transformer module is used to obtain the fused features of the input through a multi-head attention mechanism and then concatenates and fuses the fused features to obtain global features. Finally, the fully connected layer performs a linear transformation on the global features to obtain the EEG signal emotion classification result.

[0046] like Figure 2As shown, the PG3DAMT model mainly consists of three functional modules: the Parallel 3D Graph Convolution (Para-G3D) module, which is mainly used to learn the spatiotemporal information of brain signal EEG data, specifically using the Spatiotemporal Graph Construction (TSGC) module within the Para-G3D module to convert the temporal graph into a spatiotemporal graph; the Attention-Multi-Scale Temporal Convolution (AT-MSTCN) module, which dynamically adjusts the feature weights of different frequency bands and further captures multi-scale temporal features; and the Transformer module, which aims to capture the continuity of contextual information during emotional cognition, providing a global representation for the dynamic evolution of emotional states. Finally, the learned information is averaged and pooled to achieve emotion classification.

[0047] For example, such as Figure 3 As shown, the Para-G3D Para-3D Graph Convolution module divides the sliding time window and adds cross-spatiotemporal edges through the Spatiotemporal Graph Construction Module (TSGC) to generate a spatiotemporal graph that integrates the time dimension and spatial layout. It adopts a hybrid construction strategy of data-driven trainable adjacency matrix and spatial prior adjacency matrix based on electrode physical distance. Through the parallel 3D graph convolution structure, it performs cross-spatiotemporal information aggregation of EEG signal features, realizes complementary spatiotemporal feature extraction, and can solve the technical bottlenecks of existing technologies such as spatiotemporal feature separation processing and the inability of single graph convolution to deeply mine cross-spatiotemporal correlation patterns. The Attention-Multi-Scale Temporal Convolution (AT-MSTCN) module first dynamically learns the association weights between different frequency bands and emotions through an attention mechanism, adaptively strengthening key emotional frequency band information such as β and γ while suppressing redundant frequency band interference. Then, it captures the dynamic features of emotions at multiple temporal granularities through temporal convolution with multiple dilation coefficients, overcoming the deficiency of traditional multi-scale convolution in lacking the ability to adaptively select frequency bands. The Transformer module models the long-range dependencies of features at different times in long-term sequences through a multi-head self-attention mechanism, constructing a global temporal representation of emotional evolution, making up for the insufficiency of local temporal modeling in reflecting the continuous dynamic evolution of emotions.

[0048] Generally, existing methods extract spatial and temporal information from EEG data using spatial modules (such as graph convolutional networks like GCN) and temporal modules (such as temporal convolutional networks like TCN). However, due to the complexity of EEG signals, a single G3D model is insufficient to handle their dynamic characteristics. To address this issue, this invention proposes a parallel 3D graph convolutional module, Para-G3D. This invention adds spatiotemporal edges to Para-G3D, directly aggregating spatiotemporal information through graph convolution operations. The data processing of the Para-G3D module specifically includes:

[0049] First, a spatiotemporal graph is constructed using the Spatiotemporal Graph Construction Module (TSGC). A sliding time window of size w with a step size of 1 is used to control the partitioning of the time graph x, adding edges across time and space. Zero-padding is used to partition the graph to obtain M time windows. .

[0050] Then, using a two-dimensional convolution pair with a kernel size of 1*1... x w By performing feature upsizing, the final spatiotemporal graph representation is obtained: For the construction of the adjacency matrix, this invention designs an adjacency matrix construction strategy that combines data-driven approaches with spatial prior information to fully reflect the functional associations and physical layout characteristics between EEG channels. A spatial relationship modeling method based on physical distance is used to describe the spatial distribution of electrodes. The connection strength between channels follows an inverse square law with respect to their distance, thus constructing a spatial topology matrix. Its elements are defined as:

[0051] ;

[0052] in, d i，j For channel i With channel j The distance between them It is the distance calibration constant, let =5 to maintain approximately 20% effective connectivity. To simultaneously capture spatiotemporal features, this invention uses an adjacency matrix... Tiling w times yields a fixed spatiotemporal adjacency matrix A w , where A w The expression is as follows:

[0053] ;

[0054] Therefore, every node in each spatiotemporal graph is connected to itself and to its one-hop spatial neighbors in all w frames. Furthermore, this invention defines a self-learning dynamic adjacency matrix. The goal is to dynamically learn the potential connection patterns between different channels during training. Finally, through... A l and A w The adjacency matrix is ​​fused using a linear combination method. Its expression is:

[0055] ;

[0056] in, w Size of the sliding time window R For the real number field, C This represents the number of electrode channels.

[0057] Therefore, graph convolution can be achieved using the following formula:

[0058] ;

[0059] in, It adds the node's own characteristics. D It is an adjacency matrix A The degree matrix, k These are learnable parameters used to control the weights of each graph convolution. K The number of steps in each aggregation determines the range of information for the aggregation nodes. This can be obtained from the above formula. To aggregate temporal information, a convolution kernel with a size of (1, ...) can be used. w ,1) 3D convolution pairs Perform a convolution operation to obtain .

[0060] For example, to further learn the temporal information of EEG data, this invention uses an attention-based multi-scale temporal convolutional AT-MSTCN module to weight features in different frequency bands, dynamically learning key frequency band features related to sentiment, while simultaneously aggregating multi-scale temporal information, such as... Figure 4 As shown, this specifically includes: First, obtaining high-dimensional feature representations by spatially mapping the features. Specifically, an attention distribution is generated through multiple two-dimensional convolutions and softmax normalization.

[0061] ;

[0062] in, W 1 , W 2 and W 3 These are learnable parameters. soft max and ReLU There are two activation functions. .

[0063] Emotions are dynamic processes that change over time, and single-scale temporal modeling cannot fully capture this dynamism. Therefore, this invention uses four convolutions with different dilation coefficients to capture temporal features. The parameter d controls different receptive fields to capture information at different scales. Simultaneously, this invention employs an average pooling module to enhance the model's robustness, improve the stability of extracted features, and avoid overfitting. The Attention-Multi-Scale Temporal Convolution (AT-MSTCN) module is used to dynamically capture key frequency band features and aggregate multi-scale temporal features.

[0064] For example, emotional features in EEG signals are often embedded in complex temporal dynamic changes, and signal features at a single moment are insufficient to fully characterize the evolution of emotional states. Therefore, this invention introduces a Transformer module, which, through a self-attention mechanism, can adaptively focus on temporal contextual information closely related to emotional expression. This module consists of two core components: a multi-head self-attention mechanism and a multilayer perceptron (MLP), and ensures the stability of information transmission through residual connections and normalization layers. First, this invention will... and x d Perform fusion and determine the features after fusion. x f The specific formula is as follows:

[0065] ;

[0066] in, This refers to the output characteristics of the Para-G3D module. x d For the output characteristics of the AT-MSTCN module, To further integrate the characteristics between channels, this invention... x f Average pooling was used: ;in, As the second feature dimension, To x f Features after average pooling.

[0067] Therefore, for the input A Transformer can be represented as:

[0068] ;

[0069] in, This indicates the layer number of the Transformer module. The Transformer module contains two sub-layers. The first is the multi-head self-attention sub-layer, calculated as follows:

[0070] ;

[0071] ;

[0072] The multi-head attention mechanism is defined as follows:

[0073] ;

[0074] ;

[0075] ;

[0076] Where h is the number of attention heads, d k Let W be the dimension of the key vector, and W be the dimension of the key vector. , , This is the learnable parameter matrix. Next is the MLP layer, whose calculation process is as follows:

[0077] ;

[0078] in, This is the final output of the Transformer module. and It is a learnable matrix. It is a bias vector. Through this design, the Transformer module can effectively capture continuous features of contextual information during emotion cognition, provide a global representation of the dynamic evolution of emotional states, and provide richer feature representations for subsequent emotion classification tasks.

[0079] For example, a leave-one-out (LOSO) validation method can be used, where data from one subject is selected as the test set, and the data from the remaining subjects is used as the training set. On the SEED dataset, binary classification of positive and negative emotions can be performed. On the DEAP dataset, only the valence and arousal dimensions are used as the emotion rating criteria. During training, the initial learning rate was set to 1e-3, the batch size to 64, and after experimental optimization, the final epochs were set to 30. The parameter settings of the PG3DAMT model are shown in Table 1.

[0080] Table 1 Parameter settings for the PG3DAMT model

[0081]

[0082] For example, to verify the effectiveness of the PG3DAMT model, this invention was compared with existing models. In experiments on both datasets, the accuracy (ACC) and standard deviation (Std) of this invention were used to evaluate its model. Existing models were selected for comparison, as shown in Table 2. These include: classic machine learning models such as Support Vector Machine (SVM) and Deep Belief Network (DBN), and advanced models such as Dynamic Graph Convolutional Neural Network (DGCNN), Spatiotemporally Aware Convolutional Network (TSception), Pyramid Graph Convolutional Network (PGCN), and Emotion Transformer Network (EmT). The comparison results show that the model of this invention has excellent performance for emotion recognition.

[0083] Table 2. Model comparison results on the SEED and DEAP datasets.

[0084]

[0085] In emotion recognition tasks, the PG3DAMT model of this invention achieved the highest accuracy compared to other state-of-the-art methods on the SEED dataset. On the SEED dataset, PG3DAMT achieved an accuracy of 85.21% and a standard deviation of 10.70%, which is 5.01% higher than the current state-of-the-art model EmT (80.20%), demonstrating the superior emotion recognition capabilities of this invention. However, the standard deviation of this invention's model is slightly higher than that of the PGCN and DGCNN models. On the DEAP dataset, the valence dimension accuracy was 60.27% and the standard deviation was 6.27%, which is 1.81% higher than the DGCNN model, while the standard deviation was lower by 1.58%, showing significant advantages in accuracy and stability. The arousal dimension accuracy reached its highest level at 64.68%, with a standard deviation of 8.25%, lower than DGCNN. Overall, PG3DAMT performs better in cross-subject emotion recognition.

[0086] To evaluate the contribution of each module to model performance, this invention designs a progressive ablation implementation, observing the changing trend of classification accuracy by gradually introducing core modules. The specific experimental procedure is as follows: First, a baseline model with only the basic 3D graph convolution (G3D) module is constructed to verify the basic effect of spatiotemporal graph convolution; then, a parallel mechanism is introduced to form Para-G3D to evaluate the effect of multi-scale feature fusion; subsequently, a multi-scale temporal convolutional network (MSTCN) is added to capture temporal dynamics, and then an attention mechanism is integrated to upgrade to AT-MSTCN, enhancing the adaptive selection capability for key frequency bands; finally, the feature representations of the above modules are fused and input into the Transformer layer to model long-range dependencies. In addition, this invention also explores the impact of the number of Transformer layers on model performance, and the results are shown in Table 3.

[0087] Table 3 Ablation Experiment

[0088]

[0089] In Table 3, m represents the number of layers in the Transformer module; G3D uses only 3D graph convolution; Para-G3D uses only parallel 3D graph convolution. Ablation experiments demonstrate the contribution of each core module of the PG3DAMT model to emotion recognition performance and the impact of the number of Transformer layers. When using only the basic G3D ​​module, the accuracy on the SEED dataset is 82.99%, and the valence and arousal dimensions accuracy on the DEAP dataset are 56.66% and 60.54%, respectively, indicating that the spatiotemporal graph convolution architecture can capture some emotional features. After introducing a parallel mechanism to form Para-G3D, the accuracy on the SEED dataset increases to 84.28%, and the DEAP valence dimension accuracy increases to 57.01%. The effectiveness of the parallel structure in enhancing feature representation through multi-scale spatiotemporal information complementarity was verified. After adding the MSTCN module, the accuracy of the SEED dataset was further improved to 84.81%, and the DEAP wake-up dimension was improved to 59.59%, demonstrating the role of multi-scale temporal convolution in capturing temporal dynamic features. When the attention mechanism of this invention was upgraded to AT-MSTCN, the accuracy of the SEED dataset reached 85.78%, and the DEAP wake-up dimension was improved to 61.23%, proving that the attention mechanism can dynamically adjust the frequency band weights and highlight key information related to sentiment. After introducing the Transformer module, the model performance was significantly improved. The accuracy of the SEED dataset stabilized at 85.21% (m=5), and the DEAP valence and wake-up dimension reached 60.27% and 64.68%, respectively. Moreover, the standard deviation of all datasets was significantly reduced, indicating that Transformer successfully modeled long-range temporal dependencies, constructed a global representation of sentiment evolution, and improved the model stability and discriminative ability. From the perspective of the impact of the number of Transformer layers, different numbers of layers (m=5, 7, 9) exhibit differentiated characteristics. The DEAP valence dimension achieves the highest accuracy (61.37%) when m=7, while the arousal dimension is optimal (64.68%) when m=5. The SEED dataset shows a slight decrease with increasing layer count, indicating that the number of layers needs to be adjusted according to the temporal complexity of the dataset to balance model performance and complexity and avoid overfitting. In summary, the modules work synergistically in capturing spatiotemporal correlations, dynamic frequency band features, and global temporal dependencies, significantly improving the model's sentiment recognition performance and demonstrating the effectiveness of the PG3DAMT framework.

[0090] To verify the PG3DAMT model's ability to transform and distinguish sentiment features, this invention uses the t-SNE method to map the original features and the features extracted by the model to a two-dimensional space. The differences in feature distribution are then analyzed through visualization. The results are as follows: Figure 5 (SEED dataset) and Figure 6 As shown in the DEAP dataset.

[0091] Figure 5and Figure 6 The t-SNE visualization results for subject 2 in the SEED dataset and the DEAP dataset (DEAP valence dimension) are presented separately. In the SEED dataset, the visualization results of the raw features before inputting into the model are shown, such as... Figure 5 As shown in (a), the sample points of the two emotion categories exhibit a highly mixed state, with no obvious clustering trend in spatial distribution and blurred category boundaries, indicating that the emotional information contained in the original EEG features has not been effectively distinguished. After processing by the PG3DAMT model, as shown in (a), the emotional information is not effectively distinguished. Figure 5 As shown in (b), sentiment samples of the same category exhibit a significant clustering trend, forming clear and identifiable spatial boundaries between different categories. This significantly improves feature separation, intuitively demonstrating the model's powerful ability to extract and transform sentiment features from the SEED dataset, consistent with its high classification accuracy on this dataset. In the DEAP dataset (valence dimension), the visualization results of the original features are shown below. Figure 6 As shown in (a), the sample points of high and low valence states are generally intertwined. Although there is a slight clustering tendency, the overall separation is low. This is closely related to the low sampling rate of the DEAP dataset, which makes it difficult to fully capture the subtle emotional dynamics of high-frequency bands, thus limiting the recognizability of emotional information in the original features. After processing by the PG3DAMT model, as shown... Figure 6 As shown in (b), the clustering trend of the sample points is enhanced, and the spatial distribution of high and low valence samples shows some separation. However, the feature separation effect is still inferior to that of the SEED dataset. This phenomenon is consistent with the model's lower classification accuracy on the DEAP dataset, further verifying the consistency of the model's feature extraction ability across different datasets. It also reflects the impact of the dataset's characteristics on the model's performance. The t-SNE visualization results intuitively demonstrate that the PG3DAMT model can effectively mine emotional features in EEG signals. The synergistic effect of each module enhances the discriminative power of the features, providing visual support for the model's effectiveness in emotion recognition tasks.

[0092] The proposed PG3DAMT model integrates three core modules: Para-G3D, AT-MSTCN, and Transformer. Para-G3D aggregates spatiotemporal features of EEG signals through a spatiotemporal graph approach and complements these features in parallel. AT-MSTCN leverages an attention mechanism to highlight key frequency bands and capture multi-scale temporal information. The Transformer learns temporal context continuity to construct a global representation. The LOSO validation strategy was employed on the SEED and DEAP datasets, and the model was compared with traditional machine learning methods and existing state-of-the-art methods. Results show that the accuracy on the SEED dataset reaches 85.21%, while the valence and arousal dimension accuracies on the DEAP dataset are 60.27% and 64.68%, respectively, outperforming the compared methods. Ablation experiments confirm that each module significantly contributes to the model's performance. Visualization analysis also verifies PG3DAMT's ability to transform and distinguish emotional features, fully demonstrating its effectiveness in EEG emotion recognition.

[0093] The above describes one or more embodiments of the EEG signal emotion classification method provided in this specification. Based on the same idea, this specification also provides a corresponding EEG signal emotion classification device, including:

[0094] The acquisition module is used to determine the differential entropy information of the EEG signal in five frequency bands for each channel based on the acquired raw EEG signals from different channels, and use it as the features of the EEG signal.

[0095] A classification module is used to classify the input EEG signal features into emotions using a pre-trained enhanced EEG emotion recognition network (PG3DAMT) model, obtaining the emotion classification result. The PG3DAMT model includes a parallel 3D graph convolutional (Para-G3D) module and an attention-multi-scale temporal convolutional (AT-MSTCN) module, followed by a feature fusion layer, a Transformer module, and a fully connected layer. The Para-G3D module is used to construct... A spatiotemporal graph is constructed, and the input EEG signal features are aggregated using the spatiotemporal graph to determine the corresponding spatiotemporal features. The Attention-Multi-Scale Temporal Convolution (AT-MSTCN) module is used to determine the multi-scale temporal information corresponding to the input EEG signal features based on the attention mechanism. The feature fusion layer performs a fusion operation on the spatiotemporal features and the multi-scale temporal information to determine the fused features. The Transformer module is used to obtain the fused features of the input through a multi-head attention mechanism, and then concatenates and fuses the fused features to obtain global features. The fully connected layer performs a linear transformation on the global features to obtain the EEG signal emotion classification result.

[0096] Specific limitations regarding the EEG signal emotion classification device can be found in the limitations of the EEG signal emotion classification method above, and will not be repeated here. Each module in the aforementioned EEG signal emotion classification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0097] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described EEG signal emotion classification method.

[0098] The present invention also provides Figure 7 The schematic diagram of the computer device shown is as follows: Figure 7 As shown, at the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the EEG signal emotion classification method provided in the above embodiments.

[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this invention.

Claims

1. A method for classifying emotions using electroencephalogram (EEG) signals, characterized in that, include: Based on the acquired raw EEG signals from different channels, the differential entropy information of the EEG signal in each channel in five frequency bands is determined and used as the features of the EEG signal. The PG3DAMT model, an enhanced EEG emotion recognition network, is pre-trained to classify the emotions of input EEG signal features, thus obtaining the emotion classification results. The PG3DAMT model includes a parallel 3D graph convolutional Para-G3D module and an attention-multi-scale temporal convolutional AT-MSTCN module, followed by a feature fusion layer, a Transformer module, and a fully connected layer. The Para-G3D module is used to construct the spatiotemporal graph. The system aggregates the input EEG signal features using a spatiotemporal graph to determine the corresponding spatiotemporal features. The Attention-Multi-Scale Temporal Convolution (AT-MSTCN) module is used to determine the multi-scale temporal information corresponding to the input EEG signal features based on an attention mechanism. The feature fusion layer performs a fusion operation on the spatiotemporal features and the multi-scale temporal information to determine the fused features. The Transformer module is used to obtain the fused features of the input through a multi-head attention mechanism and to concatenate and fuse the fused features to obtain global features. The fully connected layer performs a linear transformation on the global features to obtain the EEG signal emotion classification result.

2. The EEG signal emotion classification method as described in claim 1, characterized in that, The Para-G3D parallel 3D graph convolution module specifically includes: The spatiotemporal graph construction module TSGC and a 3D convolution are connected sequentially. The spatiotemporal graph construction module TSGC divides the sliding time window and adds cross-spatiotemporal edges to generate a spatiotemporal graph that integrates the time dimension and spatial layout. The spatiotemporal graph is used by the 3D convolution to aggregate the features of the input EEG signal and determine the corresponding spatiotemporal features. The spatiotemporal graph includes multiple nodes and an adjacency matrix to represent the adjacency relationship between nodes.

3. The EEG signal emotion classification method as described in claim 2, characterized in that, The acquisition of the adjacency matrix used to represent the adjacency relationship between nodes in the spatiotemporal graph specifically includes: Constructing a spatial topology matrix By using the spatial topology matrix Tiling W times yields a spatiotemporally fixed adjacency matrix for capturing spatiotemporal connectivity features of EEG signals. A w ; Based on spatiotemporal fixed adjacency matrix A w Combined with a pre-set, self-learning dynamic adjacency matrix The two are combined linearly to form an adjacency matrix for representing the adjacency relationships between nodes. Specifically, it includes: ； in, w Size of the sliding time window R For the real number field, C This represents the number of electrode channels.

4. The EEG signal emotion classification method as described in claim 1, characterized in that, The attention-multi-scale temporal convolution AT-MSTCN module specifically includes: The system consists of a sequentially connected attention module and four parallel convolutions with different dilation coefficients; wherein, the attention module uses multiple two-dimensional convolutions to map spatiotemporal features to a high-dimensional feature space, and... The activation function generates attention weights corresponding to the spatiotemporal features. The mapped spatiotemporal features are weighted with the corresponding attention weights to determine the weighted features. Through convolutions with different dilation coefficients, features at different time scales in the weighted features are obtained to determine the corresponding multi-scale temporal information.

5. The EEG signal emotion classification method as described in claim 1, characterized in that, The step of determining the differential entropy information of the EEG signal in five frequency bands for each channel based on the acquired raw EEG signals from different channels specifically includes: Based on the raw EEG signals acquired from different channels, the raw EEG signals were segmented into multiple segments using the sliding window technique. Each segment is further divided into multiple sub-segments using the sliding window technique, and for each sub-segment, the differential entropy information of each frequency band is determined.

6. The EEG signal emotion classification method as described in claim 1, characterized in that, The PG3DAMT model was trained using the constructed SEED and DEAP datasets to determine the pre-trained PG3DAMT model.

7. A brainwave signal emotion classification device, characterized in that, include: The acquisition module is used to determine the differential entropy information of the EEG signal in five frequency bands for each channel based on the acquired raw EEG signals from different channels, and use it as the features of the EEG signal. A classification module is used to classify the input EEG signal features into emotions using a pre-trained enhanced EEG emotion recognition network (PG3DAMT) model, obtaining the emotion classification result. The PG3DAMT model includes a parallel 3D graph convolutional (Para-G3D) module and an attention-multi-scale temporal convolutional (AT-MSTCN) module, followed by a feature fusion layer, a Transformer module, and a fully connected layer. The Para-G3D module is used to construct... A spatiotemporal graph is constructed, and the input EEG signal features are aggregated using the spatiotemporal graph to determine the corresponding spatiotemporal features. The Attention-Multi-Scale Temporal Convolution (AT-MSTCN) module is used to determine the multi-scale temporal information corresponding to the input EEG signal features based on the attention mechanism. The feature fusion layer performs a fusion operation on the spatiotemporal features and the multi-scale temporal information to determine the fused features. The Transformer module is used to obtain the fused features of the input through a multi-head attention mechanism, and then concatenates and fuses the fused features to obtain global features. The fully connected layer performs a linear transformation on the global features to obtain the EEG signal emotion classification result.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the EEG signal emotion classification method according to any one of claims 1-6.

9. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the EEG signal emotion classification method according to any one of claims 1-6.

Images