Electroencephalogram emotion recognition method based on multi-scale convolution and attention mechanism

Abstract

The invention discloses an electroencephalogram emotion recognition method based on multi-scale convolution and an attention mechanism. The method comprises the steps that firstly, original electroencephalogram signals are processed; secondly, extracting space and frequency features of the electroencephalogram signals by utilizing a feature pyramid network, and capturing multi-level information of local and global brain regions in a multi-scale convolution structure; a multi-scale attention aggregation module is further introduced, and brain region feature self-adaptive weighting is achieved through a parallel space and channel attention mechanism; secondly, global modeling and long-range dependence capture of time domain features are achieved through a Transform coding structure; and finally, outputting an emotion recognition result through a full-connection classifier. According to the method, the time domain, frequency domain and space domain features of the electroencephalogram signals can be extracted at the same time, efficient and accurate emotion classification is achieved, the robustness and universality of electroencephalogram emotion recognition are remarkably improved, and the method can be widely applied to the fields of intelligent human-computer interaction, mental health monitoring, emotion regulation and control and the like.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and brain-computer interface technology, specifically relating to a brainwave emotion recognition method based on multi-scale convolution and attention mechanisms. Background Technology

[0002] With the convergence of artificial intelligence and neuroscience, brain-computer interface (BCI) technology has become an important research direction in the field of artificial neural information processing. BCIs enable direct interaction between the human brain and external devices by collecting neural signals from electroencephalography (EEG), magnetoencephalography (MEG), and functional magnetic resonance imaging (fMRI). Among the various branches of BCI technology, affective brain-computer interfaces (aBCIs) have gradually become a research hotspot. Their core objective is to automatically identify an individual's emotional state in different situations by analyzing physiological characteristics such as EEG signals, thereby achieving natural and intelligent human-computer interaction. Emotion recognition technology can not only enhance the human-computer interaction experience but also be widely applied in scenarios such as clinical diagnostic assistance, driver fatigue monitoring, psychological and behavioral assessment, and human-computer interaction.

[0003] However, emotion is a complex psychological and physiological state, characterized by strong subjectivity, high ambiguity, and significant individual differences. Its manifestation in electroencephalogram (EEG) signals often exhibits nonlinear and non-stationary characteristics, making emotion recognition a highly challenging task. Compared to external behavioral signals such as speech and facial expressions, EEG signals directly reflect central nervous system activity and possess advantages such as high temporal resolution, low cost, convenient acquisition, and resistance to spoofing. Therefore, they demonstrate strong reliability and anti-interference capabilities in emotion recognition (ER) tasks.

[0004] Despite some progress in EEG-based emotion recognition tasks, several challenges remain to be addressed, as follows:

[0005] First, convolutional neural networks (CNNs) have insufficient feature extraction capabilities. In existing EEG emotion recognition research, CNNs are widely used to extract spatial features from EEG signals. However, most methods use fixed-scale convolutional kernels for feature extraction, which is insufficient to fully adapt to the multi-scale characteristics of EEG signals across different brain regions and spatial resolutions. Because EEG signals exhibit significant spatial diffusion, different scales of convolutional kernels correspond to different receptive fields. If features are extracted only at a single scale, the model cannot simultaneously capture the detailed changes in local brain regions and the interaction patterns of global brain regions. This results in insufficient hierarchical representation of features and an inability to acquire comprehensive spatial frequency domain information, thus limiting further improvements in classification and recognition performance.

[0006] Second, the fusion effect of attention mechanisms is limited. Existing EEG emotion recognition methods still have significant shortcomings in the design of attention mechanisms. On the one hand, some models do not incorporate attention mechanisms, and therefore cannot actively highlight important emotion-related features. On the other hand, even when some studies incorporate attention models, they typically only perform weighted calculations in a single dimension, such as channel or spatial dimension, failing to simultaneously consider multi-dimensional features such as brain regions and frequency bands. Furthermore, existing attention modules often employ a serial structure, requiring sequential processing of spatial and channel features, resulting in low information exchange efficiency and delayed weight updates, thus reducing the overall response speed and fusion capability of the model. These factors contribute to the significant inadequacy of the model's ability to represent EEG signals in complex emotional states.

[0007] Third, the temporal relationship modeling is insufficient. Existing technologies mostly utilize Long Short-Term Memory (LSTM) networks for time-series modeling of emotional EEG signals, but this structure still has significant limitations when processing high-dimensional, long-sequence EEG data. Because LSTM relies on sequential iteration for computation, there are strong dependencies between its time steps, making parallel training difficult and resulting in low computational efficiency when processing signals with high sampling frequencies or long time spans. Furthermore, LSTM has limited ability to capture long-distance temporal dependencies; as the number of time steps increases, previous emotional association information is easily forgotten or weakened, causing the model to fail to fully represent the global temporal dynamics of the EEG signal. Therefore, LSTM-based temporal modeling is significantly insufficient in revealing the continuity, stages, and emotional evolution patterns of EEG signals, limiting the overall expressive power and stability of emotion recognition models.

[0008] Fourth, insufficient information fusion. Existing emotion recognition models still have limitations in the fusion of multi-domain features of EEG signals. Most methods tend to model spatial, frequency, and temporal features separately, lacking a unified expression of the intrinsic connections between the three, resulting in fragmented feature levels and restricted information flow. While spatial features can reflect activation patterns between brain regions, they lack perception of frequency band energy changes; frequency domain features can reveal differences in energy distribution under emotional states, but ignore temporal dynamics; and while temporal features capture the temporal evolution of signals, they easily lose information about the spatial structure of brain regions. Some existing studies have attempted feature fusion through simple splicing or concatenation, but this static fusion method is difficult to establish semantic correspondences across domains, easily causing redundant or conflicting information, thus affecting the model's overall understanding and representation of emotional patterns. Therefore, how to achieve efficient information coupling and collaborative modeling across multiple dimensions of space, frequency, and time remains one of the key problems facing the field of EEG emotion recognition. Summary of the Invention

[0009] The purpose of this invention is to provide an EEG emotion recognition method based on multi-scale convolution and attention mechanisms, which can efficiently extract features of EEG signals in the time, frequency, and spatial domains and achieve accurate classification of emotional states.

[0010] To achieve the above objectives, this invention designs an EEG emotion recognition method based on multi-scale convolution and attention mechanisms. The technical solution specifically includes the following steps:

[0011] S1: EEG signal processing. This step mainly includes EEG signal preprocessing, feature calculation, and 3D feature mapping. By cleaning and performing basic transformations on the raw EEG signals, a stable input is provided for subsequent models.

[0012] S2: Multi-scale Feature Extractor. This module utilizes a Feature Pyramid Network (FPN) to extract preprocessed EEG features, acquiring multi-scale features in both the frequency and spatial domains. The FPN structure includes bottom-up paths, top-down paths, and lateral connection modules. It achieves efficient capture of multi-scale spatial information from EEG signals through hierarchical aggregation of multi-scale convolutional kernels.

[0013] S3: Multi-scale Attention Aggregation (MSAA). This module uses a multi-scale attention aggregation module to weight and optimize the features output from the FPN. It includes two branches: spatial attention and channel attention, employing a parallel structure to fuse multi-scale information. It adaptively enhances emotion-related region features from both channel and spatial dimensions while suppressing interference from irrelevant signals.

[0014] S4: Transformer Temporal Modeling. Features optimized by the MSAA module are fed into the Transformer network to capture the long-range dependencies of EEG signals over time. The Transformer module learns the global correlation patterns of signals over time through multi-head self-attention mechanisms, achieving a complete model of emotional dynamics.

[0015] S5: Emotion Classification and Output. The output of the Transformer is input into a fully connected classifier. The Softmax function is used to obtain the predicted probability distribution of the emotion category, and the model is optimized using cross-entropy loss. Finally, the emotion classification result is output.

[0016] In a preferred embodiment of this invention, the preprocessing step S1 includes filtering, baseline correction, artifact removal, segmentation, and standardization. First, the EEG data is imported using the MNE toolkit, and a zero-phase filter is used for bandpass filtering within the 0.5–50 Hz frequency band to remove high-frequency electromyography interference and power line noise. Second, baseline correction is performed on the filtered signal to eliminate DC components and drift errors. Third, independent component analysis (ICA) is used to remove non-EEG artifacts such as eye movement and muscle movements. Finally, the EEG signal is segmented into multiple segments using a fixed-length non-overlapping time window, and each segment is standardized using the following formula:

[0017]

[0018] In the formula The standardized EEG signal EEG signals, The mean, The standard deviation is denoted as .

[0019] Feature calculation mainly refers to the quantitative analysis of the frequency domain features of EEG signals. This invention uses Differential Entropy (DE) as the main feature index to measure the complexity and distribution uncertainty of EEG signals. First, the EEG data within each time segment is decomposed into four frequency bands (θ, α, β, γ) using a Butterworth filter, and the DE feature value for each frequency band is calculated, defined as follows:

[0020]

[0021] Where p(x) represents the probability density function of x, and [a, b] represents the information range.

[0022] Three-dimensional feature mapping is used to map the frequency domain features of different channels onto a two-dimensional matrix according to the 10–20 international electrode distribution standard to construct a three-dimensional feature map of brain regions with spatial topology. The three dimensions of each three-dimensional feature map are length, width, and number of frequency bands (9×9×4). Each electrode node corresponds to a frequency domain feature location point, and the stacking of features from different frequency bands forms a three-dimensional tensor representation, which provides input for the subsequent spatial convolution module.

[0023] As a preferred technical solution in this invention, in step S2, the Feature Pyramid Network (FPN) structure consists of a bottom-up path, a top-down path, and a lateral connection path, used to jointly extract multi-scale features in the spatial-frequency domain of EEG signals. Before model input, the outer boundary of the EEG feature matrix is ​​first zero-padded, expanding the two-dimensional structure from 9×9 to 12×12 to ensure the integrity of spatial information during subsequent downsampling. The bottom-up path constructs multiple feature layers, using a convolution operator with a stride of 2 and a kernel size of 3×3 to progressively compress the spatial dimensions, forming feature pyramids of different resolutions. For given input data... The feature output of the k-th layer It can be represented as:

[0024]

[0025] in This represents a 3×3 convolution operation with the number of layers k = 1, 2, 3. This represents the number of channels in the k-th layer.

[0026] Next, the high-level feature maps obtained from the bottom-up path are upsampled using a top-down path. Then, the upsampled features are horizontally concatenated with the features from the previous layer to obtain the final predicted feature values. This aims to enhance the semantic representation of deep features. Specifically, the top-down path uses a 2x nearest neighbor upsampling method, and the horizontal concatenation uses a 1×1 convolution to adjust the number of channels. The calculation formula can be expressed as:

[0027]

[0028] in Indicates upsampling, This represents the output of the horizontal connection and upsampling. Specifically, To reduce aliasing during upsampling, we use a 3×3 convolution to obtain the prediction value for each layer, and then sum all the prediction values ​​to obtain the final prediction value. Its formula can be expressed as:

[0029]

[0030] in This represents the predicted value of the k-th layer. This represents a 1×1 convolution operation. This indicates a splicing operation.

[0031] As another preferred technical solution in this invention, in step S3, a multi-scale attention aggregation module (MSAA) is introduced to further enhance the spatial and frequency band features highly correlated with emotional states. This module first compresses the input features through a 1×1 convolution to obtain a channel count of... The compact representation is obtained (where α = 0.25). The spatial attention branch extracts multi-layer spatial responses using convolutional kernels of different scales and fuses the feature distribution through average pooling and max pooling. This is then followed by a 7×7 convolution and sigmoid activation to generate a spatial attention weight map. The channel attention branch performs global average pooling and max pooling on the same input, extracting global features, and then generates a channel attention map through a 1×1 convolution and ReLU activation. Finally, the outputs of the two branches are weighted and fused to obtain an attention-enhanced feature map that fuses spatial and channel information, achieving synergistic enhancement of global and local emotional features.

[0032] In a preferred embodiment of this invention, in step S4, the Transformer module is used to capture the temporal dependence and emotional evolution characteristics of EEG signals. This module consists of a multi-head self-attention layer and a feedforward neural network. Firstly, based on the time step position... Calculate the position code:

[0033]

[0034] In the formula , These represent the positional encoding values ​​in even and odd dimensions, respectively. Indicates the location of a sample point in a time series. Representing feature dimensions. Location encoding and input features. After addition, multi-head attention input is obtained. .

[0035] Next, a query vector is generated through a linear transformation. Key vector AND value vector Calculate single-head attention:

[0036]

[0037] in , , , , , These are linear transformation matrices.

[0038] Multi-head attention structures compute multiple attention heads in parallel and then concatenate the output:

[0039]

[0040] The self-attention output, after being processed by residual connections and layer normalization, is input into the feedforward neural network:

[0041]

[0042] After stacking multiple layers, the output features of the Transformer are obtained. This result comprehensively characterizes the global time dependence of EEG signals.

[0043] As a preferred technical solution in this invention, in step S5, the classification module adopts a two-layer fully connected neural network structure, and outputs the probability distribution of each emotion category through the Softmax function. During the model training phase, the cross-entropy loss function is used as the objective function, and the optimization formula is:

[0044]

[0045] in This represents the cross-entropy loss, where N represents the number of samples. and These represent the true label and the predicted possible value of the label, respectively. After training, the model can perform high-precision emotion recognition and classification on input EEG signals.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] This invention achieves unified representation and optimized learning of multi-scale, multi-domain features of EEG signals. By introducing a multi-scale convolutional structure into the feature pyramid network, different receptive field convolutional kernels are used to hierarchically extract and fuse the spatial frequency domain features of EEG signals. This allows for the simultaneous capture of subtle fluctuations in local brain regions and co-activation patterns in global brain regions, thus achieving a balance between detailed and overall information at the feature level. This multi-scale network structure significantly improves the model's generalization and robustness in handling complex spatiotemporal EEG patterns, providing more discriminative input features for subsequent temporal modeling and classification.

[0048] A multi-scale attention aggregation mechanism based on a parallel structure is proposed, which significantly improves feature fusion efficiency and sensitivity to emotion-related regions. Unlike traditional single-dimensional serial attention models, this invention employs a parallel attention mechanism in both spatial and channel dimensions, enabling the model to simultaneously focus on key brain regions and emotion-related frequency bands. This achieves dynamic weighting and focusing of important features at different scales and in different brain regions. This mechanism effectively improves the model's response speed to key emotional signals, suppresses interference from irrelevant features, and makes feature representations more sparse, concentrated, and semantically aggregated, thus establishing a stronger discriminative foundation for the recognition of complex emotional states.

[0049] By introducing the Transformer architecture, the model's ability to model the global temporal dependence of EEG signals is enhanced. This invention embeds the Transformer's multi-head self-attention mechanism and feedforward neural network into the EEG feature learning framework, enabling dependency modeling and contextual association capture of EEG signals over long time scales. This overcomes the problems of gradient vanishing and inefficiency of recurrent neural networks (RNN / LSTM) in long-sequence information transmission. This architecture can flexibly focus on key signal segments at any position in the time series, more accurately representing the emotional evolution process and EEG fluctuation characteristics, thereby significantly improving the model's temporal representation ability and classification accuracy.

[0050] An end-to-end multimodal fusion recognition framework was constructed, significantly improving the accuracy, stability, and practicality of EEG emotion recognition. This invention achieves collaborative learning of spatial, frequency, and temporal features within a unified framework. Through the organic combination of multi-scale convolutional feature extraction, attention aggregation optimization, and Transformer temporal modeling, it realizes deep semantic representation and efficient recognition of EEG signals. This method effectively reduces reliance on manual feature design, enabling the model to maintain good recognition performance and robustness under different individuals, different emotion induction conditions, and different experimental noise levels. It has high practical application value and promising prospects, and can provide reliable technical support for mental health monitoring, driving safety warnings, and intelligent human-computer interaction. Attached Figure Description

[0051] Figure 1 This is a flowchart of the present invention;

[0052] Figure 2 This is a flowchart of the EEG signal processing of the present invention;

[0053] Figure 3 This is a mapping diagram of the electrode channels of the present invention;

[0054] Figure 4 This is a schematic diagram of the multi-scale feature extractor processing flow of the present invention;

[0055] Figure 5This is a flowchart of the attention mechanism module of the present invention.

[0056] Figure 6 This is a flowchart of the Transformer of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Please see Figures 1 to 6 This invention provides a brainwave emotion recognition method based on multi-scale convolution and attention mechanisms, specifically including the following steps:

[0059] S1: EEG signal processing. The flowchart for this step is as follows: Figure 2 As shown, this aims to reduce noise and lay the foundation for subsequent feature extraction.

[0060] (1) Preprocessing of EEG signals

[0061] Preprocessing includes filtering, baseline correction, artifact removal, segmentation, and standardization of the raw EEG signal. First, a zero-phase-shift FIR filter is used to bandpass filter the EEG data in the 0.5–50 Hz band to eliminate electrical interference from 50 Hz line noise. Second, baseline correction is performed on the processed EEG data to eliminate DC offset and improve analytical accuracy. Third, ICA processing is performed to eliminate non-EEG-related artifacts, and the EEG signal is segmented into n equal-length time segments, each with a length of Ts, using non-overlapping time windows. Finally, the EEG signal is standardized.

[0062] Let the EEG signal be The mean is The standard deviation is The standardized formula is:

[0063]

[0064] In the formula This is the standardized EEG signal.

[0065] (2) Frequency domain feature calculation

[0066] The frequency domain features of the standardized EEG signal are calculated for each time segment. First, the EEG data for each time segment is decomposed into four frequency bands (θ, α, β, γ) using a Butterworth filter, and the differential entropy (DE) is calculated for each band. The DE feature is used to measure the complexity of the EEG signal, and its calculation formula can be expressed as:

[0067]

[0068] Where p(x) represents the probability density function of x, and [a, b] represents the information value interval. For an EEG signal of a specific length, its DE calculation formula can be expressed as:

[0069]

[0070] Where x follows a Gaussian distribution N(μ, σ) 2 ), where e and μ represent the natural constant and the standard deviation of the time series, respectively.

[0071] (3) Feature 3D mapping

[0072] We will use the DE vector calculated in the previous step as follows: Figure 3 The electrode channels shown are mapped sequentially to a two-dimensional matrix and stacked. In Figure 3, red represents the electrode corresponding to channel 32, red and black represent the electrode corresponding to channel 62, and the surrounding gray represents zero-filled areas. Therefore, the four-dimensional structure corresponding to each segment can be represented as follows: Where h represents the height of the matrix, w represents the width of the matrix, d represents the number of frequency bands, and 2T represents the length of the segment.

[0073] S2: Feature Extractor: Feature Pyramid Network (FPN). The processed EEG data obtained in the previous step is fed into the Feature Pyramid Network, such as... Figure 4 As shown, this structure includes bottom-up paths, top-down paths, and lateral connection paths to acquire multi-scale features in the spatial-frequency domain of EEG. To facilitate subsequent calculations, the EEG feature vectors are... The outer edge is filled with 0s, creating a new feature vector. It can be represented as:

[0074]

[0075] in The symbols represent the filling process, where h and w represent the height and width before filling, d represents the number of channels, and H and W represent the height and width after filling.

[0076] (1) Bottom-up path

[0077] The bottom-up path is the forward propagation process of a convolutional neural network, consisting of feature maps at multiple scales. It uses convolutional kernels with a stride of 2 and a size of 3×3 to progressively compress the spatial dimensions, forming a feature pyramid with decreasing resolution. For a given input... We obtain feature vectors with different scales through feature pyramids. The calculation formula can be expressed as:

[0078]

[0079] in This represents a 3×3 convolution operation with the number of layers k = 1, 2, 3. This represents the number of channels in the k-th layer.

[0080] (2) Top-down path and lateral connection path

[0081] The top-down path upsamples the high-level feature maps obtained from the bottom-up path, and then horizontally concatenates the upsampled features with the features from the previous layer to obtain the final predicted feature values. This aims to enhance the semantic representation of deep features. Specifically, the top-down path uses a 2x nearest neighbor upsampling method, and the horizontal concatenation uses a 1×1 convolution to adjust the number of channels. The calculation formula can be expressed as:

[0082]

[0083] in Indicates upsampling, This represents the output of the horizontal connection and upsampling. Specifically, To reduce aliasing during upsampling, we use a 3×3 convolution to obtain the prediction value for each layer, and then sum all the prediction values ​​to obtain the final prediction value. Its formula can be expressed as:

[0084]

[0085] in This represents the predicted value of the k-th layer. This represents a 1×1 convolution operation. This indicates a splicing operation.

[0086] S3: Attention Module: Multi-Scale Attention Convergence (MSAA) module. The MSAA module consists of spatial attention and temporal attention, such as... Figure 5 As shown. Before entering the MSAA module, we will use the output vector from the previous step. Let F be the channel number, and its number of channels C is compressed using a 1×1 convolution. ,in In this paper, α = 0.25. The spatial attention mechanism first sums the convolution results of different kernel sizes through a multi-scale fusion module to fuse features at different scales. Then, it aggregates spatial features using average pooling and max pooling, and performs nonlinear transformations using 7x7 convolutions and the Sigmoid activation function to obtain the spatial attention operation result. The channel attention mechanism compresses the spatial dimension of the feature map to 1x1 and performs global average pooling and max pooling to extract global information. It generates a channel attention map using 1x1 convolutions and the ReLU activation function, and enlarges its size to match the dimension of the input feature map. Finally, it adds the results of the spatial and channel dimensions to obtain an output feature map containing more details. .

[0087] S4: Time Series Modeling: Transformer. Since emotional representations in EEG signals are often contained within long time series exceeding seconds, traditional recurrent neural network (RNN) models struggle to model long-range dependencies due to the vanishing gradient problem. Furthermore, LSTMs used in some studies have limited ability to capture long-distance temporal dependencies; as time steps increase, previous emotional associations are easily forgotten or weakened, resulting in the model's inability to fully represent the global temporal dynamics of EEG signals. To extract long-range temporal dependency features from EEG signals, this embodiment introduces a Transformer encoding structure into the deep learning model. The Transformer consists of a multi-head self-attention module and a feedforward neural network module, as shown in the flowchart below. Figure 6 As shown. To enable the model to have time-aware capabilities, positional encoding is calculated for the input signal sequence:

[0088]

[0089] In the formula , These represent the positional encoding values ​​in even and odd dimensions, respectively. Indicates the location of a sample point in a time series. This represents the feature dimension. The positional encoding is added to the input feature matrix to obtain the input Z of the multi-head self-attention module. , where H is the temporal feature matrix output by the previous layer (multi-scale attention aggregation module).

[0090] In the multi-head self-attention module, the input Z is first linearly mapped to obtain the query vector Q, key vector K, and value vector V. , , , , , These are the linear transformation matrices, and then the single-head self-attention is calculated:

[0091]

[0092] In the formula Let be the dimension of the key vector, and Softmax be the softmax function. Let K be the number of columns in the matrix, i.e., the vector dimension. Multi-head self-attention involves setting up multiple attention heads, each independently calculating its self-attention, then concatenating and linearly transforming the results to obtain the multi-head self-attention output.

[0093]

[0094] in Let each represent a linear transformation matrix for the i-th attention head. This represents multi-head self-attention, and Concat(⋅) represents concatenation, representing the self-attention of each head. This is the output weight matrix.

[0095] The multi-head self-attention outputs are summed and normalized before being input into a feed-forward network (FFN), which consists of two fully connected layers:

[0096]

[0097] In the formula, 𝛔(⋅) is the ReLU activation function. , This is the weight matrix. , This is the bias term. Both the multi-head self-attention layer and the feedforward layer are equipped with residual connections and layer normalization, the calculation formula of which is:

[0098]

[0099] in, This represents the sub-layer output. After several stacked layers of computation, the final output of the Transformer is obtained:

[0100]

[0101] This output will serve as input for the subsequent classification module, used to generate emotion recognition results.

[0102] S5: Emotion Recognition Result Classification. Two fully connected layers are used as classifier modules, with cross-entropy loss as the model's loss function. A final classification result is obtained after a softmax function. The loss function calculation formula is as follows:

[0103]

[0104] in This represents the cross-entropy loss, where N represents the number of samples. and These represent the actual label and the predicted possible value of the label, respectively.

Claims

1. A brainwave emotion recognition method based on multi-scale convolution and attention mechanisms, characterized in that, Includes the following steps: S1, Processing of EEG signals; S2. Extract preprocessed EEG features using a multi-scale feature extractor to obtain multi-scale features in the frequency and spatial domains. S3. A multi-scale attention aggregation module is used to perform weighted optimization of multi-scale features in the frequency domain and spatial domain; S4. Input the optimized features into the Transformer module to capture the long-range dependency features of EEG signals in the time dimension. Furthermore, by learning the global correlation patterns of signals over time through a multi-head self-attention mechanism, a complete model of emotional dynamics can be achieved. S5. Input the output of the Transformer into the fully connected classifier module for classification, and finally output the emotion classification result.

2. The EEG emotion recognition method based on multi-scale convolution and attention mechanisms according to claim 1, characterized in that, The EEG signal processing in step S1 includes EEG signal preprocessing, frequency domain feature calculation, and three-dimensional feature mapping.

3. The EEG emotion recognition method based on multi-scale convolution and attention mechanisms according to claim 2, characterized in that, The EEG signal preprocessing includes filtering, baseline correction, artifact removal, segmentation, and standardization. First, a zero-phase-shift FIR filter is used to bandpass filter the EEG data in the 0.5-50 Hz band to eliminate electrical interference in the 50 Hz line noise. Second, the processed EEG data undergoes baseline correction. Third, ICA processing is performed to eliminate non-EEG-related artifacts, and the EEG signal is segmented into n equal-length time segments using non-overlapping time windows, each segment having a length of Ts. Finally, the EEG signal is standardized. Let the brainwave signal be The mean is The standard deviation is The standardized formula is: ; In the formula This is the standardized EEG signal.

4. The EEG emotion recognition method based on multi-scale convolution and attention mechanisms according to claim 2, characterized in that... The frequency domain feature calculation specifically involves: decomposing the EEG data within each time segment into four frequency bands (θ, α, β, γ) using a Butterworth filter, and calculating the differential entropy (DE) for each band. The DE feature is used to measure the complexity of the EEG signal, and its calculation formula can be expressed as: , Where p(x) represents the probability density function of x, and [a, b] represents the information range; For a specific length of EEG signal, the DE calculation formula can be expressed as: , Where x follows a Gaussian distribution N(μ, σ) 2 ), where e and μ represent the natural constant and the standard deviation of the time series, respectively.

5. The EEG emotion recognition method based on multi-scale convolution and attention mechanisms according to claim 2, characterized in that... The three-dimensional mapping specifically involves mapping the differential entropy (DE) vectors of each frequency band into two-dimensional matrices according to the corresponding electrode channels and stacking them. The four-dimensional structure corresponding to each segment can be represented as follows: Where h represents the height of the matrix, w represents the width of the matrix, d represents the number of frequency bands, and 2T represents the length of the segment.

6. The EEG emotion recognition method based on multi-scale convolution and attention mechanisms according to claim 1, characterized in that, The multi-scale feature extractor in step S2 uses a feature pyramid network, which consists of bottom-up paths, top-down paths, and lateral connection paths to achieve joint extraction of multi-scale features in the spatial-frequency domain of EEG signals.

7. The EEG emotion recognition method based on multi-scale convolution and attention mechanisms according to claim 1, characterized in that, Before inputting the model, the outer boundary of the EEG feature matrix is ​​first zero-padded, expanding the two-dimensional structure from 9×9 to 12×12 to ensure the integrity of spatial information during subsequent downsampling. A multi-layered feature hierarchy is constructed from the bottom up, using a convolution operator with a stride of 2 and a kernel size of 3×3 to progressively compress the spatial dimensions, forming feature pyramids of different resolutions. For the given input data... The feature output of the k-th layer It can be represented as: , in This represents a 3×3 convolution operation with the number of layers k = 1, 2, 3. This represents the number of channels in the k-th layer; Next, the high-level feature maps obtained from the bottom-up path are upsampled through the top-down path, and then the upsampled features are horizontally connected with the features of the previous layer to obtain the final predicted feature values. Specifically, the top-down path uses a 2x nearest neighbor upsampling method, and the lateral connections use 1×1 convolutions to adjust the number of channels. The calculation formula can be expressed as: ; in Indicates upsampling, This represents the output of the horizontal connection and the upsampling sum, specifically, To reduce aliasing during upsampling, we use a 3×3 convolution to obtain the prediction value for each layer, and then sum all the prediction values ​​to obtain the final prediction value. Its formula can be expressed as: , in This represents the predicted value of the k-th layer. This represents a 1×1 convolution operation. This indicates a splicing operation.

8. The EEG emotion recognition method based on multi-scale convolution and attention mechanisms according to claim 1, characterized in that, In step S3, the multi-scale attention aggregation module consists of spatial attention and temporal attention. First, the input features are compressed in dimension using a 1×1 convolution to obtain a channel count of... The compact representation is achieved by: a spatial attention branch extracting multi-layer spatial responses using convolutional kernels of different scales and fusing the feature distribution through average pooling and max pooling, followed by 7×7 convolution and Sigmoid activation to generate a spatial attention weight map; a channel attention branch performing global average pooling and max pooling on the same input to extract global features, followed by 1×1 convolution and ReLU activation to generate a channel attention map; finally, the outputs of the two branches are weighted and fused to obtain an attention-enhanced feature map that fuses spatial and channel information, achieving synergistic enhancement of global and local emotional features.

9. The EEG emotion recognition method based on multi-scale convolution and attention mechanisms according to claim 1, characterized in that, In step S4, the Transformer module consists of two parts: a multi-head self-attention layer and a feedforward neural network. It is used to capture the temporal dependence and emotional evolution characteristics of EEG signals. Specifically, it first focuses on the time step location. Calculate the position code: ; In the formula, i is the index of the encoding dimension, and... , These represent the positional encoding values ​​in even and odd dimensions, respectively. Indicates the location of a sample point in a time series. Representing feature dimensions; location encoding and input features After addition, multi-head attention input is obtained. ; Then, a query vector is generated through linear transformation. Key vector AND value vector Calculate single-head attention: ; in , , , , , These are linear transformation matrices; Multi-head attention structures compute multiple attention heads in parallel and then concatenate the output: , in Let each represent a linear transformation matrix for the i-th attention head. h represents the total number of attention heads. This represents multi-head self-attention, and Concat(⋅) represents concatenation, representing the self-attention of each head. To output the weight matrix; The self-attention output, after being processed by residual connections and layer normalization, is input into the feedforward neural network: ; In the formula, 𝛔(⋅) is the ReLU activation function. , This is the weight matrix. , As a bias term, both the multi-head self-attention layer and the feedforward layer are equipped with residual connections and layer normalization, and their calculation formula is as follows: ; in, This represents the output of a sub-layer, which, after being processed through several stacked layers, yields the final output of the Transformer. ; Output This will be used as input for the subsequent classification module to generate emotion recognition results.

10. The EEG emotion recognition method based on multi-scale convolution and attention mechanisms according to claim 1, characterized in that, The fully connected classifier module employs a two-layer fully connected neural network structure, outputting the probability distribution of each emotion category through the Softmax function. During model training, the cross-entropy loss function is used as the objective function, and its formula is: ; in This represents the cross-entropy loss, where N represents the number of samples. and These represent the actual label and the predicted possible value of the label, respectively.

Images