Electroencephalogram signal emotion recognition method and system

Abstract

The present application relates to the technical field of electroencephalogram signal processing, and provides an electroencephalogram signal emotion recognition method and system, which takes preprocessed EEG time domain signals as input, extracts local space-time features and global connection features in the EEG time domain signals through a multi-scale 3D convolution module and a graph convolution module in parallel, then fuses through a feature fusion modulation module based on an attention mechanism, effectively fuses two heterogeneous but complementary feature representations, avoids dilution or conflict caused by simple splicing or averaging, and through an asymmetric modulation mechanism of 'global guiding local', selectively enhances high discriminative local features while effectively integrating and utilizing low contribution global features, so as to obtain more comprehensive and discriminative emotion representation. The present application can effectively solve the problems existing in feature extraction, information representation and fusion decision of the prior art.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal processing technology, and more specifically, to a method and system for emotion recognition based on deep learning EEG signals. Background Technology

[0002] Emotion recognition based on electroencephalogram (EEG) signals is an important research direction in human-computer interaction and neuroscience. Its core lies in how to more effectively extract discriminative spatiotemporal features from multi-channel EEG signals. In recent years, with the development of machine learning technology, EEG signal emotion recognition methods have generally evolved from traditional machine learning methods to deep learning methods, and from manual feature extraction to end-to-end deep learning.

[0003] Early research primarily relied on signal processing techniques and traditional machine learning classifiers. The extracted features were typically categorized into three types:

[0004] (1) Time-domain characteristics: These include event-related potentials (ERPs), Hjorth parameters (activity, mobility, complexity), higher-order zero-crossings (HOCs), and statistical characteristics (mean, variance, first / second-order differences). These time-domain characteristics can directly reflect the amplitude changes of the signal, but are sensitive to noise;

[0005] (2) Frequency domain characteristics: Power spectral density (PSD) is the most commonly used characteristic. The energy of rhythms such as δ (1-3Hz), θ (4-7Hz), α (8-13Hz), β (14-30Hz), and γ (31-50Hz) is extracted by Fourier transform. Differential entropy (DE) is widely used because it approximates PSD in a specific frequency band.

[0006] (3) Time-frequency domain characteristics: such as Hilbert-Huang transform (HHT), wavelet transform, etc., can simultaneously capture the time-frequency characteristics of non-stationary signals.

[0007] In early studies, the aforementioned handcrafted features were often combined with classifiers such as support vector machines (SVM) and random forests (RF). For example, Soroush et al. [1] used phase space reconstruction to extract angle variation features and combined them with DS evidence theory for a four-class classification task; Gupta et al. [2] used flexible analytical wavelet transform (FAWT) to extract time-frequency features and combined them with random forests for classification. However, handcrafted features rely heavily on prior knowledge of the domain, the design process is cumbersome and the generalization ability is limited, making it difficult to fully exploit the high-dimensional nonlinear characteristics of EEG.

[0008] With the rise of deep learning technology, the following are some of the more prominent existing deep learning-based methods:

[0009] (1) Convolutional Neural Network (CNN)

[0010] CNN-based methods typically treat EEG signals as image formats and use CNNs to extract joint spatial-temporal features. For example, Zhao et al. [3] proposed a lightweight 3D-CNN model whose input is a temporal signal. By reconstructing the electrode positions into a two-dimensional grid, a three-dimensional tensor (width × height × time point) is formed. CNN-based methods use two layers of 3×3×4 convolutional kernels to directly process 9×9×128 inputs. They apply a baseline correction preprocessing method, which significantly improves the robustness of the model to individual differences by calculating the mean from the baseline signal 3 seconds before the experiment and subtracting it [4].

[0011] (2) Graph Neural Network (GNN)

[0012] To explicitly model the functional connections between channels, GNN treats EEG electrodes as graph nodes and channel relationships as edges, aggregating neighborhood information through graph convolution. GNNs are categorized into two types based on their graph structure definition:

[0013] Fixed graph: Static adjacency matrix is ​​constructed based on spatial distance or predefined brain region division. RGNN[5] defines connections based on electrode Euclidean distance and introduces regularization constraints to improve generalization; GraphNet[6] uses distance thresholds to construct binary adjacency matrices. Although such methods introduce prior knowledge and rules of neuroanatomy, they are difficult to adapt to individual differences and task-specific functional coupling.

[0014] Dynamic graph: The adjacency matrix is ​​dynamically optimized through learnable parameters. The Dynamic Graph Convolutional Network (DGCNN) proposed by Song et al. [7] is a representative work in this field. It uses differential entropy (DE) features as node attributes to extract energy information of five frequency bands (δ, θ, α, β, γ). The adjacency matrix is ​​dynamically updated through backpropagation to capture functional connectivity patterns related to sentiment. The performance of DGCNN is significantly better than that of fixed graph models (such as GCNN) and traditional methods (such as SVM, DBN), which verifies the advantages of dynamically learning functional connectivity.

[0015] (3) Neuroscience-inspired hybrid architecture

[0016] In recent years, there has been a trend of incorporating the prior knowledge of the hierarchical organization of brain networks into deep learning architectures. The Local-Global Graph Network (LGGNet) proposed by Ding et al. [8] is a typical example. It takes temporal signals as input and uses triple 1D convolutional kernels (size proportional to sampling rate) to extract multi-scale temporal features, and adaptively weights them through kernel-level attention fusion layers. The multiple local graphs it contains are defined according to neural functional regions (such as frontal lobe and parietal lobe), focusing on modeling dense connections between channels within specific brain regions; the global graph models global connections between all channels and focuses on cross-regional connections. Local features and global features complement each other, providing the model with a more comprehensive and hierarchical spatial information representation.

[0017] However, the existing brainwave signal emotion recognition methods mentioned above still have the following drawbacks:

[0018] 1) It generally relies on manually designed, task-related feature engineering, which may introduce additional bias and information loss. For example, DGCNN[7] takes the differential entropy (DE), power spectral density (PSD), differential asymmetry (DASM), rational asymmetry (RASM), and differential causality features (DCAU) of EEG signals as inputs. These features are all extracted manually, which introduces additional bias and learning loss.

[0019] 2) Most of them focus on a single feature and cannot simultaneously extract and fuse multi-dimensional heterogeneous features, thus having the inherent limitation of incomplete emotion representation. For example, 3DCNN focuses on short-term features of local brain regions extracted through 3D convolution, ignoring global functional connectivity; DGCNN extracts global connectivity by dynamically learning the adjacency matrix, but the inherent volumetric transmission effect of EEG signals leads to spatial information ambiguity, making the information content of simple global graph structure representation insufficient to reflect the completeness in emotion recognition tasks;

[0020] 3) The completeness of spatiotemporal information representation is still insufficient, lacking fusion decisions based on heterogeneous features. For example, although LGGNet attempts to combine global-local features, it still chooses to construct a simple graph structure for global-local representation. Therefore, it is still affected by the limitations of the graph structure itself.

[0021] Given the shortcomings of the existing technologies, there is an urgent need for a better EEG signal emotion recognition solution.

[0022] [1] Zangeneh Soroush, M., Maghooli, K., Setarehdan, S.K. et al. Anovel EEG-based approach to classify emotions through phase space dynamics.SIViP 13, 1149–1156 (2019).

[0023] [2] Gupta V, Chopda M D, Pachori R B. Cross-subject emotionrecognition using flexible analytic wavelet transform from EEG signals[J].IEEE Sensors Journal, 2018, 19(6): 2266-2274.

[0024] [3] Y. Zhao, J. Yang, J. Lin, D. Yu and X. Cao, "A 3D ConvolutionalNeural Network for Emotion Recognition based on EEG Signals," 2020 International Joint Conference on Neural Networks (IJCNN) , Glasgow, UK, 2020,pp. 1-6.

[0025] [4] Y. Yang, Q. Wu, M. Qiu, Y. Wang, and X. Chen, “Emotionrecognition from multi-channel eeg through parallel convolutional recurrentneural network,” in 2018 International Joint Conference on Neural Networks(IJCNN). IEEE, Conference Proceedings, pp. 1–7.

[0026] [5] P. Zhong, D. Wang, and C. Miao, "EEG-based emotion recognition using regularized graph neural networks," IEEE Trans. Affect. Comput., vol.13, no. 3, pp. 1290–1301, Jul. 2022.

[0027] [6] V. Delvigne, H. Wannous, T. Dutoit, L. Ris, and J.-P. Vandeborre, “PhyDAA: Physiological dataset assessing attention,” IEEE Trans. CircuitsSyst. Video Technol., vol. 32, no. 5, pp. 2612–2623, May 2022.

[0028] [7] T. Song, W. Zheng, P. Song, and Z. Cui, “EEG emotion recognition using dynamical graph convolutional neural networks,” IEEE Trans. Affect.Comput., vol. 11, no. 3, pp. 532–541, Jul. / Sep. 2020.

[0029] [8] Y. Ding, N. Robinson, C. Tong, Q. Zeng and C. Guan, "LGGNet: Learning From Local-Global-Graph Representations for Brain–ComputerInterface," in IEEE Transactions on Neural Networks and Learning Systems ,vol. 35, no. 7, pp. 9773-9786, July 2024. Summary of the Invention

[0030] In view of the above problems, the purpose of this invention is to provide a brainwave emotion recognition method and system based on EEG local-global spatiotemporal feature modulation, so as to solve the problems of existing brainwave signal emotion recognition schemes, such as reliance on manual work, single focusing features, and incomplete spatiotemporal information representation.

[0031] On one hand, the present invention provides a method for emotion recognition using electroencephalogram (EEG) signals, comprising:

[0032] The acquired EEG time-domain signals are subjected to channel mapping preprocessing and channel normalization preprocessing to obtain the local spatiotemporal tensor used for input to the preset multi-scale 3D convolution module and the global tensor used for input to the preset graph convolution module.

[0033] The local spatiotemporal tensor is processed by 3D convolution and fully connected layer compression through the multi-scale 3D convolution module to output the local spatiotemporal feature vector of the EEG time domain signal; and the global tensor is processed by graph convolution and SE mechanism through the graph convolution module to output the global feature vector of the EEG time domain signal.

[0034] The local spatiotemporal feature vector and the global feature vector are adaptively fused through a preset feature fusion modulation module to obtain the emotion recognition result; including:

[0035] The local spatiotemporal feature vector and the global feature vector are concatenated along the feature dimension to preserve the original distribution characteristics of the local spatiotemporal feature vector and the global feature vector, thus obtaining the concatenated feature.

[0036] The feature weights of the spliced ​​features are learned through a three-layer fully connected network.

[0037] The local spatiotemporal feature vector is modulated element-wise using the feature weights to selectively scale the local spatiotemporal feature vector.

[0038] The modulated local spatiotemporal feature vector is mapped to the emotion space using a classifier to obtain the emotion recognition result.

[0039] In addition, an alternative approach is to acquire an initial EEG time-domain signal before preprocessing the EEG time-domain signal; wherein, during the acquisition of the initial EEG time-domain signal, a pre-constructed EEG signal dataset based on music emotion-induced signals is used as music stimulation to induce emotions.

[0040] Alternatively, the preprocessed EEG time-domain signal can be... C × T Matrix, where, C For the number of channels, T The number of time points; the channel mapping preprocessing includes: mapping the number of channels of the EEG time-domain signal to a form based on the spatial distribution relationship. The local spatiotemporal tensor is obtained in the spatial grid. ,in, Used to simulate 2D spatial layout.

[0041] On the other hand, the present invention also provides an EEG signal emotion recognition system, including a data preprocessing module, a multi-scale 3D convolution module, a graph convolution module, and a feature fusion modulation module; wherein,

[0042] The data preprocessing module is used to perform channel mapping preprocessing and channel normalization preprocessing on the acquired EEG time-domain signals to obtain the local spatiotemporal tensor for input to the preset multi-scale 3D convolution module and the global tensor for input to the preset graph convolution module.

[0043] The multi-scale 3D convolution module is used to perform 3D convolution and fully connected layer compression processing on the local spatiotemporal tensor, and output the local spatiotemporal feature vector of the EEG time domain signal to capture the detailed spatiotemporal patterns of local brain regions.

[0044] The graph convolution module is used to perform graph convolution and SE mechanism processing on the global tensor and output the global feature vector of the EEG time domain signal, so as to capture the global spatiotemporal pattern under different emotional states by modeling the functional connection relationship between different channels.

[0045] The feature fusion modulation module is used to adaptively fuse the local spatiotemporal feature vector and the global feature vector through a preset feature fusion modulation module to obtain the emotion recognition result;

[0046] The feature fusion modulation module includes:

[0047] The feature concatenation unit is used to concatenate the local spatiotemporal feature vector and the global feature vector along the feature dimension to preserve the original distribution characteristics of the local spatiotemporal feature vector and the global feature vector, thereby obtaining the concatenated feature.

[0048] The weight learning unit is used to learn the feature weights of the spliced ​​features through a three-layer fully connected network.

[0049] A modulation unit is used to perform element-wise modulation of the local spatiotemporal feature vector using the feature weights, so as to selectively scale the local spatiotemporal feature vector;

[0050] The feature mapping unit is used to map the modulated local spatiotemporal feature vector to the emotion space using a classifier to obtain the emotion recognition result.

[0051] On the other hand, the present invention also provides an electronic device, the electronic device including a memory, a processor and an EEG signal emotion recognition program stored in the memory and executable on the processor, wherein the EEG signal emotion recognition program, when executed by the processor, implements the EEG signal emotion recognition method as described above.

[0052] In another aspect, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the EEG signal emotion recognition method as described above.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. The end-to-end structure of the present invention can be automatically learned by the data-driven model to obtain the feature representation that is most conducive to completing the task, without the need to introduce prior knowledge and rules;

[0055] 2. The local-global spatiotemporal feature modulation strategy of the present invention can systematically extract and fuse local detail features and global connectivity features to obtain a more complete emotion representation, thereby improving the performance of EEG signal emotion recognition.

[0056] 3. This invention adopts heterogeneous feature fusion decision-making, which can achieve more comprehensive and complementary spatiotemporal information representation and better overcome the limitations of single-structure representation.

[0057] To achieve the foregoing and related objectives, one or more aspects of the invention include their specific features, which will be described in detail below. The following description and accompanying drawings illustrate certain exemplary aspects of the invention. However, these aspects indicate only a few of the various ways in which the principles of the invention can be used. Furthermore, the invention is intended to encompass all such aspects and their equivalents. Attached Figure Description

[0058] Other objects and results of the invention will become more apparent and readily understood with a more complete understanding of the invention, taking into account the following description taken in conjunction with the accompanying drawings. In the drawings:

[0059] Figure 1 This is a schematic diagram of the network architecture of the EEG emotion recognition method based on EEG local-global spatiotemporal feature modulation according to an embodiment of the present invention, wherein (a) is an overview of the structure, (b) is a 3D convolutional module, and (c) is a graph convolutional module;

[0060] Figure 2 This is a schematic flowchart of the EEG emotion recognition method based on EEG local-global spatiotemporal feature modulation according to an embodiment of the present invention;

[0061] Figure 3 This is a schematic diagram of channel mapping according to an embodiment of the present invention;

[0062] Figure 4 This is a schematic diagram of the framework structure of an EEG signal emotion recognition system according to an embodiment of the present invention;

[0063] Figure 5This is a schematic diagram of the framework structure of an EEG signal emotion recognition system according to another embodiment of the present invention;

[0064] Figure 6 This is a schematic diagram of the logical structure of an electronic device for implementing an EEG signal emotion recognition method according to an embodiment of the present invention.

[0065] In all the accompanying drawings, the same reference numerals indicate similar or corresponding features or functions. Detailed Implementation

[0066] In the following description, numerous specific details are set forth for illustrative purposes and to provide a thorough understanding of one or more embodiments. However, it will be apparent that these embodiments may also be implemented without these specific details. In other instances, well-known structures and devices are shown in block diagram form for ease of description of one or more embodiments.

[0067] To address the shortcomings of existing EEG signal emotion recognition methods, this invention provides a novel EEG emotion recognition scheme based on EEG local-global spatiotemporal feature modulation, which simultaneously solves the following problems:

[0068] 1. How to achieve end-to-end spatiotemporal feature extraction from EEG to eliminate the reliance on manual priors such as brain region segmentation or frequency band division in existing methods, avoid manual feature extraction steps, use the original time-domain signal as dual-branch input, ensure information integrity, and achieve global joint training.

[0069] 2. How to construct a multi-scale and multi-granularity complementary feature space: Simultaneously extract emotion-related short-term and long-term features, as well as local and global features, to overcome the representational limitations of single-scale convolution or graph convolution;

[0070] 3. How to establish a dynamic feature fusion mechanism: Design a sample adaptive feature modulation strategy for different emotional states and individual differences to achieve dynamic weight allocation of local and global features and enhance the complementarity and effectiveness of heterogeneous features.

[0071] Figure 1 A schematic diagram of the network architecture of an EEG emotion recognition method based on EEG local-global spatiotemporal feature modulation according to an embodiment of the present invention is shown, wherein (a) is an overview of the structure, (b) is a 3D convolutional module, and (c) is a graph convolutional module. Figure 2 The flowchart of an EEG emotion recognition method based on EEG local-global spatiotemporal feature modulation according to an embodiment of the present invention is shown, consisting of... Figure 1 and Figure 2As shown in the accompanying drawings, the EEG emotion recognition method based on local-global spatiotemporal feature modulation provided by this invention mainly includes two feature extraction branches (a 3D convolution branch composed of a multi-scale 3D convolution block and a graph convolution branch composed of a graph convolution block) and a feature fusion modulation module. Furthermore, due to the insufficient data volume in EEG signal emotion classification tasks induced by music as a stimulus, this invention also collects an EEG-emotion dataset based on music stimulation, inducing emotions and collecting EEG signals accordingly. The method will be described in detail below with reference to the accompanying drawings.

[0072] This invention provides a brainwave emotion method based on EEG local-global spatiotemporal feature modulation, implemented using a dual-branch parallel neural network (hereinafter referred to as the network) architecture for EEG signal emotion recognition tasks. The core of this network lies in the parallel extraction and fusion of local spatiotemporal features and global connectivity features from the EEG, thereby obtaining a more comprehensive and discriminative emotion representation. Specifically, the network uses preprocessed EEG time-domain signals (… C × T matrix, C The number of channels (electrodes) in the EEG time-domain signal, also known as channel dimension. T The input consists of two core branches: a multi-scale 3D convolution branch and a graph convolution branch.

[0073] The multi-scale 3D convolutional branch focuses on extracting localized, detailed spatiotemporal features, while the graph convolutional branch focuses on capturing interaction patterns based on global spatial topology. These two types of features are significantly complementary in representing emotion-related brain activity. Subsequently, before inputting the features into the classifier, this invention introduces a feature fusion modulation module based on an attention mechanism to effectively fuse these two heterogeneous but complementary feature representations, avoiding dilution or conflicts caused by simple splicing or averaging.

[0074] The following sections will introduce each of the above modules in detail, using specific methods and procedures.

[0075] like Figure 2 As shown, firstly, in step S210, the acquired EEG time-domain signal is subjected to channel mapping preprocessing and channel normalization preprocessing to obtain the local spatiotemporal tensor used to input the preset multi-scale 3D convolution module and the global tensor used to input the preset graph convolution module.

[0076] Then, in step S220, the local spatiotemporal tensor is subjected to 3D convolution and fully connected layer compression processing by the multi-scale 3D convolution module to output the local spatiotemporal feature vector of the EEG time domain signal; and the global tensor is subjected to graph convolution and SE mechanism processing by the graph convolution module to output the global feature vector of the EEG time domain signal.

[0077] The multi-scale 3D convolution module, as a local spatiotemporal feature extractor, aims to capture detailed spatiotemporal patterns of local brain regions. By using parallel multi-scale convolutional paths and fusing spatiotemporal features of different receptive fields, this module can significantly improve the representation ability of emotional responses in local brain regions.

[0078] Specifically, as an example, during the channel mapping preprocessing, the multi-scale 3D convolution module maps the number of channels of the input signal according to the spatial distribution relationship to the form of... In the spatial grid, specifically in the multi-scale 3D convolution module, H For the height of the spatial grid, W This represents the width of the spatial grid. Here... H = W =9. Channel mapping methods suitable for self-built datasets (59 channels) are as follows: Figure 3 As shown.

[0079] The input to the multi-scale 3D convolution module is the preprocessed and channel-mapped EEG temporal signal, which can be represented as a local spatiotemporal tensor. , where L represents the input tensor of the convolutional layer after mesh mapping in the multi-scale 3D convolutional module; and L stands for local. It is a simulated 2D spatial layout. H For 2D spatial layout; W For the width of the 2D spatial layout, T The time points are the number of points; the output after 3D convolution and fully connected layers is the feature vector. , For the real number field. 3D convolution kernels in spatial dimension and time dimension Simultaneous sliding on and off can effectively model local spatiotemporal neighborhoods. Correlation within.

[0080] Specifically, as an example, the preprocessing method applied to the self-built dataset constructed in this embodiment includes:

[0081] (a) Downsample the data in the self-built dataset from 1000Hz to 200Hz and apply a bandpass filter from 0.1 to 45Hz and a notch filter from 50Hz;

[0082] (b) Perform independent component analysis on the filtered signal and manually remove suspected artifacts;

[0083] (c) Extract all channels C The baseline signal, and with a fixed length T Cut it into N Section, obtained N part C × T matrix;

[0084] (d) N Element-wise addition of the baseline signal segments and calculation of their average value ( C × T )

[0085] (e) Split the official data into M part C × T The matrix is ​​then subtracted from the average value of the baseline signal.

[0086] After the above processing, the form of a single input signal to the network is as follows: C × T The matrix. Wherein, when applying this self-built dataset, C =59, T =200 (i.e., 1s) N =3, M =25.

[0087] It should also be noted that in the self-built dataset of this embodiment, the duration of the formal data is 30 seconds. However, in order to ensure that the selected parts have been sufficiently emotionally induced, the first 5 seconds of formal data have been removed, and only the last 25 seconds have been retained.

[0088] like Figure 1 As shown in (b), the multi-scale 3D convolutional module contains a two-stage serial structure, whose outputs are denoted as tensors. and Each level contains three parallel convolutional layers with kernel sizes of (9×5×5), (5×3×3), and (1×1×1), respectively. The first... Level 1 Each convolutional layer is denoted as ,but:

[0089] ,

[0090] ;

[0091] The numbers 32 and 64 represent the set number of output channels.

[0092] The output tensor of the convolutional layer Flattening the vector yields the input vector of the fully connected layer in the multi-scale 3D convolutional module: ,

[0093] After compression by a fully connected layer, the final output of the multi-scale 3D convolutional module, i.e., the feature vector, can be obtained. This is also considered as a local feature vector learned by the model, and this process can be represented as:

[0094] ,

[0095] in, and These are the weights and biases of the fully connected layers in the multi-scale 3D convolutional module.

[0096] In the implementation details within Conv3D(·) above, the three dimensions involved in the 3D convolution module are H, W, and T, respectively, in terms of spatial dimensions. and time dimension T Simultaneous sliding of 3D convolution kernels can effectively model local spatiotemporal neighborhoods. The correlation within the tensor. Specifically, the input local spatiotemporal tensor and global tensor pass through the following structure: ① three parallel 3D convolutional layers; ② a ReLU layer; ③ a MaxPool3D layer, to obtain the output tensor ( and ).

[0097] The graph convolution module, acting as a global spatiotemporal feature extractor, aims to model the functional connectivity relationships between different channels and capture global spatiotemporal patterns under different emotional states. In the graph convolution module of this invention, dynamic graph convolution is introduced, and an adaptive adjacency matrix is ​​learned end-to-end. This is to capture global interaction patterns related to emotions. That is, the graph convolution module in this invention can also be called a dynamic graph convolution module. This is the adjacency matrix corresponding to the input tensor in the dynamic graph convolution module.

[0098] The "dynamic" aspect of the aforementioned "dynamic graph convolution" is reflected in the adjacency matrix of the graph structure. It is not pre-fixed or artificially defined, but can automatically learn and adjust during the training process (except during the inference phase). (It is also fixed). In typical graph convolution methods, the adjacency matrix is ​​often a constant pre-set based on prior knowledge, rather than a variable that needs to be dynamically learned.

[0099] Furthermore, in the graph convolution module of this embodiment, the SE (Squeeze-and-Excitation) mechanism is introduced to enhance key feature channels and suppress secondary channels, thereby improving the expressiveness of the extracted global spatiotemporal patterns.

[0100] Figure 1 Part (c) illustrates the structure of a graph convolution module according to an embodiment of the present invention.

[0101] Specifically, as an example, the input to the graph convolution module of the present invention is the preprocessed EEG time-domain signal. , Let G be the input tensor of the graph convolution module, where G stands for global. After processing by graph convolution and SE mechanisms, the output is a global feature vector. .

[0102] Specifically, the channels of the EEG time-domain signal are considered as graph nodes, and the node characteristics are the input time-domain signal. Its corresponding adjacency matrix is Then the graph convolution process can be represented as:

[0103] ,

[0104] in, For learnable, by The calculated symmetric normalized Laplace matrix, K Let be the order of the Chebyshev polynomial of the first kind, i.e., the number of terms in the Chebyshev polynomial. For the Chebyshev polynomial of the th Term coefficient, for The first kind of Chebyshev polynomial k item, for of K Chebyshev polynomial.

[0105] Subsequently, an SE mechanism is introduced after the graph convolution operation to enhance key channel features.

[0106] Specifically, first convolve the graph output Perform global average pooling to capture channel-level statistics This process can be represented as:

[0107] ,

[0108] Subsequently, channel weights are learned through two fully connected layers. :

[0109] ,

[0110] in, and The offsets for the two fully connected layers are shown above, with subscript 1 representing the first layer and subscript 2 representing the second layer. and The weights of the two fully connected layers mentioned above are respectively. , Compression ratio r =16.

[0111] Then use channel weights right By performing feature recalibration, the node features after adaptively adjusting the channel weights can be obtained. : ,

[0112] This node feature This refers to the tensors in the dynamic graph convolution module that have undergone feature recalibration using the SE model. Finally, the node features... Flattened into a one-dimensional vector, that is:

[0113] ,

[0114] Should This is the input vector to the fully connected layer of the dynamic graph convolutional mapping. Then, by dimensionality reduction using the fully connected layer, the global features can be obtained. , can be represented as:

[0115] ,

[0116] Should The final output of the dynamic graph convolution module is considered the global feature vector learned by the model. and These are the weights and biases of the fully connected layer in the graph convolution module, respectively.

[0117] After obtaining the local spatiotemporal feature vector and global feature vector of the EEG time-domain signal through the multi-scale 3D convolution module and the graph convolution module, step S230 can be executed to adaptively fuse the local spatiotemporal feature vector and the global feature vector through the preset feature fusion modulation module to obtain the emotion recognition result.

[0118] The feature fusion modulation module aims to adaptively fuse local spatiotemporal features. Features of global functionality Specifically, step S230 further includes:

[0119] S231: Will and By concatenating the first and last features along the feature dimension, we obtain the concatenated features. :

[0120] ,

[0121] This operation preserves the original distribution characteristics of both local spatiotemporal feature vectors and global feature vectors, thus avoiding information loss caused by early fusion.

[0122] S232: Learning feature weights through a three-layer fully connected network :

[0123] ,

[0124] ,

[0125] ,

[0126] in, and These are the first three fully connected layers of the network. i Weights and biases of fully connected layers.

[0127] S233: Using weights For local features Element-wise modulation is performed to obtain the feature vector:

[0128]

[0129] in, In the feature fusion modulation module, The feature weight vector obtained through the fully connected layer; this element-wise modulation operation is equivalent to selectively scaling local features, introducing additional information from global features while fully preserving the integrity of local features, so as to achieve dynamic feature fusion modulation.

[0130] S234: Use a classifier to map the modulated local spatiotemporal feature vector to the emotion space to obtain the emotion recognition result.

[0131] The "emotion space" here refers to the Valence-Arousal (VA) two-dimensional space. It's a continuous emotion model that measures emotion using two dimensions: "valence" and "arousal." Valence represents the degree of pleasure or aversion (positive / negative), while arousal represents the intensity or aversion (intense / calm). In emotion recognition, the VA emotion space is a commonly used output target. The model doesn't need to learn to distinguish dozens of discrete emotion labels; instead, it can regress and predict the values ​​of the two dimensions (V and A values) or perform four-quadrant classification. This significantly reduces the difficulty of machine learning tasks and allows for a more nuanced reflection of subtle emotional changes.

[0132] Specifically, in one embodiment of the present invention, a classifier comprising two fully connected layers is used to process the modulated features. Mapping this onto the emotion space yields the final output vector of the model, which is the emotion recognition result. :

[0133] ,

[0134] ,

[0135] ,

[0136] in, and The classifiers are respectively the first and second classes. The weights and biases of fully connected layers. The number of categories.

[0137] The EEG signal emotion recognition method provided by this invention was compared with the DEAP dataset and a self-built dataset. A binary classification task based on Valence and Arousal dimensions was used as the evaluation benchmark to compare with current mainstream EEG emotion recognition models. Experimental results show that this invention achieves significant breakthroughs in recognition accuracy in both Valence and Arousal dimensions (as shown in Tables 1 and 2).

[0138] Table 1. Comparison Experiment Results of DEAP Dataset

[0139]

[0140] Table 2. Comparison Experiment Results of Self-Built Dataset

[0141]

[0142] The above examples and experimental results demonstrate that the EEG signal emotion recognition method provided by this invention innovatively employs a dual-branch parallel network architecture for extracting local-global features. The core innovation of this network lies in the parallel extraction and fusion of local spatiotemporal features and global connectivity features from EEG, thereby obtaining a more comprehensive and discriminative emotion representation. The network takes preprocessed EEG temporal signals as input and includes multi-scale 3D convolutional branches and graph convolutional branches. The multi-scale 3D convolutional branches focus on extracting localized, detailed spatiotemporal features, while the graph convolutional branches focus on capturing interaction patterns based on global spatial topology. Experiments show that these two types of features are significantly complementary in representing emotion-related brain activity.

[0143] Based on the aforementioned dual-branch parallel network architecture, this invention proposes a feature fusion modulation mechanism: This invention proposes a feature fusion modulation module based on an attention mechanism to effectively fuse two heterogeneous but complementary feature representations, avoiding dilution or conflicts caused by simple concatenation or averaging. It utilizes global function-connected features (… Generate attention vectors Dynamically enhance local spatiotemporal features ( Through an asymmetric modulation mechanism of "global guidance of local", it can effectively integrate and utilize global features with low contribution while selectively enhancing highly discriminative local features.

[0144] Compared with existing technologies, the EEG signal emotion recognition method provided by this invention has the following advantages:

[0145] 1. It adopts an end-to-end structure, eliminating the need to introduce prior knowledge and rules; it directly processes EEG time-domain signals, saving the step of manual feature extraction;

[0146] 2. By systematically extracting and fusing local detail features and global connectivity features through a local-global spatiotemporal feature modulation strategy, a more complete emotion representation can be obtained;

[0147] 3. Adopt heterogeneous feature fusion decision-making to achieve more comprehensive and complementary spatiotemporal information representation, breaking through the limitations of single-structure representation.

[0148] Furthermore, visual stimuli such as images, videos, and text are commonly used in studies related to emotion induction. However, to date, there are few studies using auditory stimuli such as music to induce emotion. To establish the correlation between music and EEG signals in the emotional space, and to further develop an objective and accurate evaluation index of the emotional impact of music, in a specific embodiment of this invention, before preprocessing the EEG time-domain signals, the initial EEG time-domain signals are acquired; wherein, during the acquisition of the initial EEG time-domain signals, an EEG signal dataset is acquired as the initial EEG time-series signal based on music stimulation and emotion induction.

[0149] The EEG dataset constructed in this embodiment based on music-induced emotion used 32 non-vocal music tracks as stimuli, ranging in length from 60 to 150 seconds and encompassing various musical styles. Twenty of these tracks were generated by SunoAI based on different prompts (including but not limited to emotion and musical style), while the remaining 12 copyright-free tracks were sourced from an internet library to supplement the emotions not included in the former. This experimental design ensures that participants lack prior perception and memory of the music stimuli, thereby reducing the influence of cultural background and personal experience on the emotion-induced effect and improving the accuracy of the experimental results. The participants in this dataset were 10 students aged 20 to 25, including 6 male and 4 female participants.

[0150] In this embodiment, the initial sampling rate of the EEG time-domain signal was 1000Hz, and the number of channels was 59. By collecting EEG signals from multiple subjects listening to music stimuli with different emotional orientations, the EEG emotion dataset used in this embodiment was constructed. Compared with existing datasets, this music-emotion-induced EEG signal dataset uses music stimuli with diverse musical styles and clear emotional orientations to reduce the influence of subjective factors, improve the effectiveness of emotion induction, thereby effectively reducing experimental bias and improving data quality.

[0151]

[0152] experimental paradigm

[0153] As shown in the experimental paradigm in the table above, the experiment was divided into two phases. The difference lies in the fact that the first phase used a carefully selected 30-second music excerpt as a short, highly focused stimulus, aiming to quickly induce the target emotional state while avoiding the distraction and fatigue effects caused by prolonged listening; while the second phase used a complete music segment (between 60 and 150 seconds in length) as a long, continuous stimulus, aiming to simulate a real music listening scenario, collect complete neural representations of the dynamic evolution of emotions with music, and obtain EEG data on the long-term stable maintenance of emotions.

[0154] Furthermore, the EEG signal dataset based on music-induced emotion in this embodiment simultaneously collected the subjects' subjective evaluations of the music stimuli they heard. The subjective evaluations included both continuous value evaluations and discrete label evaluations. In the continuous value evaluation, for each music stimulus, subjects were asked to rate it on three dimensions: valence, arousal, and like, with scores ranging from 1 to 9. In the discrete label evaluation, for each music stimulus, subjects were asked to select the most appropriate label from four discrete labels: happiness, neutrality, sadness, and fear. The scores of valence and arousal, ranging from 1 to 9, in the subjective evaluations were used as the labeling benchmark for the EEG data. Specifically, in one embodiment of the invention, in the binary classification task of the valence dimension, data with scores greater than or equal to 5 were considered positive examples, and data with scores less than 5 were considered negative examples; the same principle applied to the binary classification task of the arousal dimension.

[0155] Corresponding to the above-mentioned brainwave signal emotion recognition method, the present invention also provides a brainwave signal emotion recognition system. Figure 4 The framework structure of an EEG signal emotion recognition system according to an embodiment of the present invention is shown. Figure 4 As shown, the EEG signal emotion recognition system provided by this invention uses the EEG signal emotion recognition method described above for emotion recognition. Depending on the functions implemented, the EEG signal emotion recognition system 400 may include a data acquisition and preprocessing module 410, a multi-scale 3D convolution module 420, a graph convolution module 430, and a feature fusion and modulation module 440. The module of this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0156] In this embodiment, the functions of each module / unit are as follows:

[0157] The data preprocessing module 410 is used to perform channel mapping preprocessing and channel normalization preprocessing on the acquired EEG time-domain signal to obtain the local spatiotemporal tensor for input to the preset multi-scale 3D convolution module and the global tensor for input to the preset graph convolution module.

[0158] The multi-scale 3D convolution module 420 is used to perform 3D convolution and fully connected layer compression processing on the local spatiotemporal tensor, and output the local spatiotemporal feature vector of the EEG time domain signal to capture the detailed spatiotemporal patterns of local brain regions.

[0159] Graph convolution module 430 is used to perform graph convolution and SE mechanism processing on the global tensor and output the global feature vector of the EEG time domain signal to capture the global spatiotemporal pattern under different emotional states by modeling the functional connectivity between different channels.

[0160] The feature fusion modulation module 440 is used to adaptively fuse the local spatiotemporal feature vector and the global feature vector through a preset feature fusion modulation module to obtain the emotion recognition result.

[0161] The feature fusion modulation module 440 may further include a feature splicing unit 441, a weight learning unit 442, a modulation unit 443, and a feature mapping unit 444.

[0162] The feature splicing unit 441 is used to splice the local spatiotemporal feature vector and the global feature vector along the feature dimension to preserve the original distribution characteristics of the local spatiotemporal feature vector and the global feature vector, thereby obtaining the spliced ​​feature.

[0163] Weight learning unit 442 is used to learn the feature weights of the spliced ​​features through a three-layer fully connected network.

[0164] The modulation unit 443 is used to perform element-wise modulation on the local spatiotemporal feature vector using the feature weights, so as to selectively scale the local spatiotemporal feature vector;

[0165] The feature mapping unit 444 is used to map the modulated local spatiotemporal feature vector to the emotion space using a classifier to obtain the emotion recognition result.

[0166] In addition, the aforementioned EEG signal emotion recognition system may also include an information acquisition module 450 (such as... Figure 5 The EEG signal emotion recognition system shown is used to collect initial EEG time-domain signals. In the process of collecting initial EEG time-domain signals, a pre-constructed EEG signal dataset based on music emotion induction can be further used as music stimulation to induce emotions.

[0167] The EEG signal emotion recognition system of this invention extracts local spatiotemporal features and global connectivity features from EEG time-domain signals in parallel using a multi-scale 3D convolution module and a graph convolution module. The multi-scale 3D convolution module focuses on extracting localized and detailed spatiotemporal features, while the graph convolution module focuses on capturing interaction patterns based on global spatial topology, thus providing complementarity in representing emotion-related brain activity. Then, a feature fusion modulation module based on an attention mechanism effectively fuses the two heterogeneous but complementary feature representations, avoiding dilution or conflict caused by simple splicing or averaging. Through an asymmetric modulation mechanism of "global guidance of local," it selectively enhances highly discriminative local features while effectively integrating and utilizing low-contribution global features, thereby obtaining a more comprehensive and discriminative emotion representation.

[0168] More specific implementations of the above-mentioned EEG signal emotion recognition system can be found in the foregoing description of the embodiments of the EEG signal emotion recognition method, and will not be detailed here.

[0169] like Figure 6 As shown, the present invention also provides an electronic device 1 for implementing a brainwave signal emotion recognition method. The electronic device 1 may include a processor 10, a memory 11, and a bus, and may also include a computer program, such as a brainwave signal emotion recognition program 12, stored in the memory 11 and executable on the processor 10. The memory 11 may include both internal storage units of the brainwave signal emotion recognition system and external storage devices. The memory 11 can be used not only to store application software and various types of data, such as the code of the brainwave signal emotion recognition program, but also to temporarily store data that has been output or will be output.

[0170] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory, magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart memory card, secure digital card, flash memory card, etc., equipped on the electronic device 1. Furthermore, the memory 11 can include both internal and external storage units of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of a brainwave signal emotion recognition program, but also to temporarily store data that has been output or will be output.

[0171] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units, microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control core of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes various functions of the electronic device 1 and processes data by running or executing programs or modules (such as EEG signal emotion recognition programs) stored in the memory 11 and calling data stored in the memory.

[0172] The bus can be a Peripheral Component Interconnect Standard (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0173] Figure 6 Only electronic devices with components are shown; those skilled in the art will understand that... Figure 6 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0174] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power various components. This power supply can be logically connected to the at least one processor 10 through a power management system, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power sources, a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator, or any other components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be elaborated further here.

[0175] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0176] Optionally, the electronic device 1 may further include a user interface, which may be a display or an input unit. Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (organic light-emitting diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0177] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0178] The EEG signal emotion recognition program 12 stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can implement the steps in the EEG signal emotion recognition method as described above.

[0179] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figure 2 The descriptions of the relevant steps in the corresponding embodiments are not repeated here. Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0180] This invention also provides a computer-readable storage medium, which may be non-volatile or volatile, and stores a computer program that, when executed by a processor, implements the EEG signal emotion recognition method as described above.

[0181] Specifically, the specific implementation method of the computer program when executed by the processor can be referred to the description of the relevant steps in the EEG signal emotion recognition method in the embodiment, and will not be repeated here.

[0182] In the several embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. The system embodiments described above are merely illustrative, and the division of modules is only a logical functional division; in actual implementation, there may be other division methods. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0183] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or in the form of hardware plus software functional modules. Obviously, the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or systems described above can also be implemented by a single unit or system through software or hardware.

[0184] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is not limited by the foregoing description. Thus, all variations that have the same or similar meaning and scope as the equivalents of the above-described technical content are intended to be included within the scope of the present invention.

[0185] The EEG signal emotion recognition method and system according to the present invention have been described above by way of example with reference to the accompanying drawings. However, those skilled in the art should understand that various modifications can be made to the EEG signal emotion recognition method and system proposed in the present invention without departing from the scope of the present invention.

Claims

1. A brain electrical signal emotion recognition method, characterized in that, include: The acquired EEG time-domain signals are subjected to channel mapping preprocessing and channel normalization preprocessing to obtain the local spatiotemporal tensor used for input to the preset multi-scale 3D convolution module and the global tensor used for input to the preset graph convolution module. The local spatiotemporal tensor is processed by 3D convolution and fully connected layer compression through the multi-scale 3D convolution module to output the local spatiotemporal feature vector of the EEG time domain signal; and the global tensor is processed by graph convolution and SE mechanism through the graph convolution module to output the global feature vector of the EEG time domain signal. The local spatiotemporal feature vector and the global feature vector are adaptively fused through a preset feature fusion modulation module to obtain the emotion recognition result; including: The local spatiotemporal feature vector and the global feature vector are concatenated along the feature dimension to preserve the original distribution characteristics of the local spatiotemporal feature vector and the global feature vector, thus obtaining the concatenated feature. The feature weights of the spliced ​​features are learned through a three-layer fully connected network. The local spatiotemporal feature vector is modulated element-wise using the feature weights to selectively scale the local spatiotemporal feature vector. The modulated local spatiotemporal feature vector is mapped to the emotion space using a classifier to obtain the emotion recognition result. 2.The electroencephalogram signal emotion recognition method of claim 1, wherein, Before preprocessing the EEG time-domain signal, the method further includes acquiring the initial EEG time-domain signal; wherein, During the initial acquisition of EEG time-domain signals, a pre-constructed dataset of EEG signals based on music-induced emotions was used as music stimulation to induce emotions.

3. The electroencephalogram signal emotion recognition method of claim 1 or 2, wherein, The EEG time domain signal is C X T Matrix, wherein, C is the number of channels, T is the number of time points; the channel mapping preprocessing comprises: The number of channels of the EEG time-domain signal is mapped according to the spatial distribution relationship to the form of The local spatiotemporal tensor is obtained in the spatial grid. ,in, Used to simulate 2D spatial layout H For the height of the spatial grid, W The width of the spatial grid.

4. The EEG signal emotion recognition method as described in claim 3, characterized in that, The multi-scale 3D convolutional module comprises a two-level serial structure, and its output is denoted as... and Each level contains three parallel convolutional layers with kernel sizes of (9×5×5), (5×3×3), and (1×1×1), respectively. The process of performing 3D convolution and fully connected layer compression on the local spatiotemporal tensor using the multi-scale 3D convolution module to output the local spatiotemporal feature vector of the EEG time-domain signal includes: The first stage of the first convolutional layer is denoted , then: , ； The numbers 32 and 64 represent the set number of output channels. flattening the output tensor of the convolutional layer flattening: , The feature vector can be obtained by compression through the fully connected layers of the multi-scale 3D convolutional module. , represented as: , wherein, with are the weights and biases of the fully connected layer of the multi-scale 3D convolution module, respectively.

5. The electroencephalogram signal emotion recognition method of claim 4, wherein, The process of performing graph convolution and SE mechanism processing on the global tensor through the graph convolution module to output the global feature vector of the EEG time-domain signal includes: The channels of the EEG time domain signals are regarded as graph nodes, and the node features are the input time domain signals The corresponding adjacency matrix is The graph convolution process is represented as ， in, For learnable, by The calculated symmetric normalized Laplace matrix, K Let the order of the first-kind Chebyshev polynomial be denoted as . For the Chebyshev polynomial of the th Term coefficient, for The first kind of Chebyshev polynomial k item, for of K Chebyshev polynomial; To perform global average pooling to capture channel-level statistics is expressed as: ， wherein, C is the number of channels; Learning channel weights through two fully connected layers : ， wherein and bias, weight bias, weight compression ratio r = 16; Use channel weights right Feature recalibration is performed to obtain node features with adaptively adjusted channel weights. : , said node features flattened into one-dimensional vectors: , Dimension reduction by fully connected layers to get global features is expressed as: ， wherein, with are weights and biases of a fully connected layer of the graph convolution module, respectively.

6. The electroencephalogram signal emotion recognition method of claim 5, wherein, The feature weights are expressed as: The learning of feature weights for the spliced ​​features through a three-layer fully connected network includes: ， ， ， in, For splicing features, and These are the first three fully connected layers of the network. i Weights and biases of fully connected layers.

7. The EEG signal emotion recognition method as described in claim 6, characterized in that, The modulated local spatiotemporal feature vector is mapped to the emotion space using a classifier to obtain the emotion recognition result, including: ， ， ， in, and The classifiers are respectively the first and second classes. The weights and biases of fully connected layers. The number of categories. This is the result of emotion recognition.

8. An electroencephalogram signal emotion recognition system, characterized by, It includes a data preprocessing module, a multi-scale 3D convolution module, a graph convolution module, and a feature fusion and modulation module; among which, The data preprocessing module is used to perform channel mapping preprocessing and channel normalization preprocessing on the acquired EEG time-domain signals to obtain the local spatiotemporal tensor for input to the preset multi-scale 3D convolution module and the global tensor for input to the preset graph convolution module. The multi-scale 3D convolution module is used to perform 3D convolution and fully connected layer compression processing on the local spatiotemporal tensor, and output the local spatiotemporal feature vector of the EEG time domain signal to capture the detailed spatiotemporal patterns of local brain regions. The graph convolution module is used to perform graph convolution and SE mechanism processing on the global tensor and output the global feature vector of the EEG time domain signal, so as to capture the global spatiotemporal pattern under different emotional states by modeling the functional connection relationship between different channels. The feature fusion modulation module is used to adaptively fuse the local spatiotemporal feature vector and the global feature vector through a preset feature fusion modulation module to obtain the emotion recognition result; The feature fusion modulation module includes: The feature concatenation unit is used to concatenate the local spatiotemporal feature vector and the global feature vector along the feature dimension to preserve the original distribution characteristics of the local spatiotemporal feature vector and the global feature vector, thereby obtaining the concatenated feature. The weight learning unit is used to learn the feature weights of the spliced ​​features through a three-layer fully connected network. A modulation unit is used to perform element-wise modulation of the local spatiotemporal feature vector using the feature weights, so as to selectively scale the local spatiotemporal feature vector; The feature mapping unit is used to map the modulated local spatiotemporal feature vector to the emotion space using a classifier to obtain the emotion recognition result.

9. An electronic device, comprising: The electronic device includes a memory, a processor, and an EEG emotion recognition program stored in the memory and executable on the processor, wherein the EEG emotion recognition program, when executed by the processor, implements the EEG emotion recognition method as described in any one of claims 1 to 7.

10. A computer readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the EEG signal emotion recognition method as described in any one of claims 1 to 7.

Images