Electroencephalogram emotion recognition method based on gated spatio-temporal modulation and bidirectional state space model

Abstract

This invention provides an EEG emotion recognition method based on gating spatiotemporal mechanisms and a bidirectional state-space model, relating to the fields of biomedical engineering and artificial intelligence. The EEG emotion recognition method includes: acquiring shallow features of an EEG signal; projecting the shallow features to obtain corresponding high-dimensional features; performing hierarchical gating processing on the high-dimensional features in both temporal and spatial dimensions to obtain spatiotemporal aggregated features; performing bidirectional state-space encoding on the spatiotemporal aggregated features to obtain fusion features; and performing weighted processing on the fusion features to obtain the emotion classification result of the EEG signal. The EEG emotion recognition method provided by this invention can effectively filter out local noise and efficiently model long-term and long-distance dependencies, constructing a computationally lightweight hybrid neural network architecture to meet the requirements of real-time and robust EEG emotion recognition.

Description

Technical Field

[0001] This invention relates to the fields of biomedical engineering and artificial intelligence, and in particular to a brainwave emotion recognition method based on gating time-space mechanism and bidirectional state space model. Background Technology

[0002] In recent years, emotion recognition based on electroencephalography (EEG) signals has attracted much attention due to its potential applications in brain-computer interfaces and affective computing. Unlike behavioral cues such as facial expressions, EEG signals directly reflect the central nervous system's response to emotional stimuli. However, the practical application of EEG signals is limited by their inherent characteristics: the signals are non-stationary, highly susceptible to physiological artifacts (such as electrooculography (EOG) and electromyography (EMG), and exhibit significant differences among different subjects. These challenges are particularly evident in fine-grained emotion recognition tasks.

[0003] Existing deep learning methods for EEG decoding primarily rely on convolutional neural networks (CNNs) or Transformer architectures. CNN models (such as EEGNet) are highly efficient at extracting local time-frequency features, but are limited by their local receptive fields, making it difficult to capture the long-range temporal dependencies required for analyzing the continuous evolution of emotional states. To address the limitations of local modeling, researchers have explored the Transformer architecture, which can capture global context. However, the self-attention mechanism in Transformers typically has computational complexity that increases quadratically with sequence length. Furthermore, the standard attention mechanism is sensitive to amplitude, and in a low signal-to-noise ratio (SNR) environment, it is easy for the model to incorrectly allocate attention to artifacts with high amplitude rather than the underlying emotional patterns.

[0004] Recent state-space models (SSMs), particularly the Mamba architecture, have achieved linear complexity modeling of long sequences by introducing a selective scanning mechanism. However, directly applying global SSMs to raw or noisy EEG signals remains challenging because the global scanning mechanism can still be affected by local artifacts in the absence of explicit modulation.

[0005] Therefore, there is an urgent need for a hybrid neural network architecture that can effectively filter out local noise, efficiently model long-distance time dependencies, and has a lightweight computational load, in order to meet the requirements of real-time and robust EEG emotion recognition. Summary of the Invention

[0006] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide an EEG emotion recognition method based on gating time-space mechanism and bidirectional state space model, which is used to solve the technical problems of filtering out local noise and long-distance time dependence in the prior art.

[0007] To achieve the above and other related objectives, a first aspect of the present invention provides a method for EEG emotion recognition based on gating time-space modulation and a bidirectional state-space model, comprising: acquiring shallow features of an EEG signal; projecting the shallow features to obtain corresponding high-dimensional features; performing hierarchical gating processing on the high-dimensional features in both time and space dimensions to obtain spatiotemporal aggregated features; performing bidirectional state-space encoding on the spatiotemporal aggregated features to obtain fusion features; and performing weighted processing on the fusion features to obtain an emotion classification result of the EEG signal.

[0008] In some embodiments of the first aspect of the present invention, in the step of projecting the shallow features to obtain corresponding high-dimensional features, a projection layer is used to project the shallow features, the projection layer includes multiple convolutional kernels, and the number of channels of the high-dimensional features is consistent with the number of convolutional kernels.

[0009] In some embodiments of the first aspect of the present invention, the step of performing hierarchical gating processing on the high-dimensional features in both time and space dimensions to obtain spatiotemporal aggregated features further includes: performing time gating processing on the high-dimensional features in the time dimension, wherein the time gating processing includes parallel first long-range feature extraction and first local feature extraction to obtain first long-range features and first local features respectively, and the first long-range features and first local features are weighted to obtain time features; performing spatial gating processing on the time features in the spatial dimension, wherein the spatial gating processing includes parallel second long-range feature extraction and second local feature extraction to obtain second long-range features and second local features respectively, and the second long-range features and second local features are weighted to obtain spatiotemporal aggregated features; and performing deep fusion of spatiotemporal feature aggregation using a global spatial operator.

[0010] In some embodiments of the first aspect of the present invention, the first long-range feature extraction and the first local feature extraction respectively use a first large convolution kernel and a first small convolution kernel to extract features from the high-dimensional features; the second long-range feature extraction and the second local feature extraction respectively use a second large convolution kernel and a second small convolution kernel to extract features from the time.

[0011] In some embodiments of the first aspect of the present invention, bidirectional state space encoding is performed on the spatiotemporal aggregation features to obtain fused features, which further includes: performing forward spatial state encoding on the spatiotemporal aggregation features to obtain forward state features; performing backward spatial state encoding on the spatiotemporal aggregation features to obtain backward state features; and fusing the forward spatial state encoding and the backward spatial state encoding to obtain fused features.

[0012] In some embodiments of the first aspect of the present invention, in the process of weighting the fused features to obtain the emotion classification result of the EEG signal, the fused features are pooled and then processed by a linear classification layer to obtain the emotion classification result of the EEG signal.

[0013] To achieve the above and other related objectives, a second aspect of the present invention provides an EEG emotion recognition system based on a gated temporal-space mechanism and a bidirectional state-space model, comprising: a feature acquisition module for acquiring shallow features of an EEG signal; a feature projection module for projecting the shallow features to obtain corresponding high-dimensional features; a gate processing module for performing hierarchical gate processing on the high-dimensional features in both time and space dimensions to obtain spatiotemporal aggregated features; a spatial encoding module for performing bidirectional state-space encoding on the spatiotemporal aggregated features to obtain fused features; and a classification module for performing weighted processing on the fused features to obtain an emotion classification result of the EEG signal.

[0014] To achieve the above and other related objectives, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the EEG emotion recognition method based on gating time-space control and a bidirectional state-space model.

[0015] To achieve the above and other related objectives, a fourth aspect of the present invention provides a computer program product comprising computer program code, wherein when the computer program code is run on a computer, the computer implements the EEG emotion recognition method based on gating time-space mechanism and bidirectional state space model.

[0016] To achieve the above and other related objectives, a fifth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the EEG emotion recognition method based on gating time-space mechanism and bidirectional state space model.

[0017] As described above, the EEG emotion recognition method based on gating time-space control and bidirectional state-space model of the present invention has the following beneficial effects:

[0018] 1) High accuracy and robustness: It achieved accuracy of 89.96% and 76.48% on the SEED and SEED-IV datasets, respectively, which is better than existing CNN and Transformer baseline models, and maintained performance stability among different subjects.

[0019] 2) Extremely lightweight: The model has only 0.06M parameters, which is much lower than similar Transformer models (such as Conformer's 0.30M), resulting in high computational efficiency.

[0020] 3) Effective noise reduction mechanism: The LS module (gating) effectively reduces the impact of high amplitude artifacts on the model in low signal-to-noise ratio environments by using soft threshold filtering of the analog signal envelope.

[0021] 4) High interpretability: The features learned by the model are biologically plausible and can identify spatial activation patterns of specific emotions (such as the activation of negative emotions in the frontal lobe). Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the emotion recognition method provided in an embodiment of the present invention.

[0023] Figure 2 This is an architecture diagram of the LS-Mamba model provided in an embodiment of the present invention.

[0024] Figure 3 This is a flowchart illustrating the sub-steps in step S3 of an embodiment of the present invention.

[0025] Figure 4 This is a flowchart illustrating the sub-steps in step S4 of an embodiment of the present invention.

[0026] Figure 5 This is the confusion matrix obtained using the SEED dataset in this embodiment of the invention.

[0027] Figure 6 This is the confusion matrix obtained using the SEED_IV dataset in this embodiment of the invention.

[0028] Figure 7 This is a schematic diagram of the fundus color photography restoration system based on the diffusion model in an embodiment of the present invention.

[0029] Figure 8 This is a schematic diagram of the structure of the computer device in an embodiment of the present invention. Detailed Implementation

[0030] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0031] To address the aforementioned technical problems, this invention introduces a novel method for identifying EEG emotion signals based on a gating time-space mechanism and a bidirectional state-space model. It should be understood that the method provided in this invention can be applied not only to emotion recognition but also to other areas of EEG signal processing, such as sleep monitoring, disease diagnosis, and motor imagery recognition. Furthermore, besides its application in the medical field, it can also be used in human-computer interaction, health management, state monitoring, autonomous driving, fatigue detection, and other fields; this invention does not limit its application to these areas.

[0032] like Figure 1 and Figure 2 As shown, this invention provides an EEG emotion recognition method based on gating time-space regulation and a bidirectional state-space model, where the gating time-space regulation and bidirectional state-space model is abbreviated as LS-Mamba model. This embodiment illustrates the application of this method to a terminal, but it is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, through interaction between the terminal and the server.

[0033] In this embodiment of the invention, the method includes the following steps:

[0034] Step S1) Obtain shallow features of the EEG signal.

[0035] The shallow features are single-channel EEG signals, i.e., a two-dimensional feature matrix of the EEG signal. The rows and columns of the two-dimensional feature matrix represent the number of electrodes and the time points, respectively.

[0036] In practice, medical testing equipment can be used to scan the brainwaves of the subject to obtain the EEG signal. The EEG signal is then input to a terminal, which preprocesses the signal to obtain its superficial features.

[0037] Step S2) Project the shallow features to obtain the corresponding high-dimensional features.

[0038] The projection refers to projecting a single-channel EEG signal into a multi-channel EEG signal, which is the three-dimensional feature matrix of the EEG signal.

[0039] In practice, the terminal projects the two-dimensional feature matrix of the EEG signal to obtain the three-dimensional feature matrix of the EEG signal, thus preliminarily suppressing high-frequency noise while preserving the spatiotemporal topology.

[0040] Step S3) Perform hierarchical gating processing on the high-dimensional features in both time and space dimensions to obtain spatiotemporal aggregated features.

[0041] The gating process can be constructed based on gating units. Simply put, gating determines which signals in the three-dimensional feature matrix of the EEG signal should be given priority or discarded.

[0042] The time dimension and the spatial dimension are two dimensions in the three-dimensional feature matrix of the EEG signal.

[0043] In the specific implementation, the terminal first performs feature processing on the three-dimensional feature matrix of the EEG signal in the time and space dimensions to obtain time features and space features respectively. Then, the time features and space features are weighted to obtain spatiotemporal aggregated features.

[0044] Step S4) Perform bidirectional state-space encoding on the spatiotemporal aggregation features to obtain fused features.

[0045] The spatial encoding can be built on the Mamba architecture. Simply put, Mamba is a model based on a Selective State Space Model (SSM) that dynamically determines which information to remember or ignore based on the input content.

[0046] Step S5) The fusion features are weighted to obtain the emotion classification result of the EEG signal.

[0047] The aforementioned emotion recognition method obtains shallow features from EEG signals, projects these shallow features to obtain corresponding high-dimensional features, performs hierarchical gating on these high-dimensional features in both time and space dimensions to obtain spatiotemporal aggregated features, performs bidirectional state-space encoding on these spatiotemporal aggregated features to obtain fusion features, and performs weighted processing on these fusion features to obtain the emotion classification result of the EEG signal. This method can effectively filter out local noise and efficiently model long-distance time dependencies.

[0048] In one embodiment, in step S1), the preprocessing includes: downsampling, for example downsampling to 128~200 Hz; bandpass filtering, for example bandpass filtering 0~75 Hz; FIR / IIR filtering to preserve emotion-related rhythms such as α, β, γ; filtering out DC drift (<0.5Hz) and high-frequency noise (>75Hz); and segmentation, for example truncating a sample length of 1 s.

[0049] In one embodiment, in step S2), the preprocessed shallow features are input into a projection layer. The projection layer comprises multiple convolutional kernels of the same size.

[0050] In one specific embodiment, as shown in Expression 1, the EEG signal is downsampled to 200Hz. Input a convolutional vector with 48 kernels and a size of The projection layer.

[0051]

[0052] in, High-dimensional features For activation function, This is represented as batch normalization processing. This is represented as a convolution operation. Represented as low-dimensional features, Represented as a projection layer with a size of The convolution kernel. This step projects the original signal into a high-dimensional feature space, preserving the spatiotemporal topology while initially suppressing high-frequency clutter.

[0053] In one embodiment, the specific implementation process of step S3 can be further divided into... Figure 3 The following steps are shown:

[0054] Step 3-1) Perform temporal gating on the high-dimensional features in the time dimension. The temporal gating includes parallel first long-range feature extraction and first local feature extraction to obtain first long-range features and first local features, respectively. The first long-range features and first local features are then weighted to obtain the temporal features. Specifically, the first long-range features and first local features are extracted from the high-dimensional features using a first large convolutional kernel and a first small convolutional kernel, respectively.

[0055] In one specific implementation, as shown in Expression 2, the first long-range feature is extracted using the first large convolutional kernel, and channel recalibration is achieved through point convolution to generate soft-gated weights. :

[0056]

[0057] in, Represented as Activation function Represented as point convolution, Represented as weights, This is represented as a convolution operation. Represented as high-dimensional features, This is represented as the largest convolution kernel.

[0058] Furthermore, the first largest convolutional kernel As a hyperparameter in the time dimension, its size is The aim is to capture rhythmic cycles of approximately 300ms. The activation function is not unique; it employs various methods. The activation function is used to compress the value of expression one to between 0 and 1. Alternatively, activation functions such as ReLU, Tanh, ELU, and GELU can also be used.

[0059] In one specific implementation, as shown in Expression 3, the first local features are extracted using the first small convolutional kernel, and channel recalibration is achieved through point convolution to generate soft-gated weights. :

[0060]

[0061] in, Represented as an activation function, This is represented as batch normalization processing. Represented as weights, This is represented as a convolution operation. This is represented as the high-dimensional feature output in step 2). This is represented as the first small convolution kernel.

[0062] Furthermore, the first small convolution kernel As a hyperparameter in the time dimension, its size is The aim is to capture spatial activation patterns of electrodes within a hemisphere of the brain. The activation function is not unique; different methods are employed. Activation functions are used to improve the nonlinear fitting ability of feature maps and the stability of gradient propagation. Relatively speaking, activation functions can also include ReLU, Tanh, ELU, Sigmoid, etc.

[0063] Subsequently, the soft-gating weights were adjusted. and soft gating weights Perform feature recalibration by performing element-wise Hadamard product as shown in Expression 4, and then obtain the temporal features by projecting the residuals:

[0064] in, Represented as a time feature, This is represented as the high-dimensional feature output in step 2). Represented as a point matrix, The soft-gating weights obtained from Expression 1 are represented as follows. The soft-gating weights obtained from expression two are represented as follows. It is represented as an element-level Hadamard product.

[0065] Step 3-2) Perform spatial gating on the temporal features in the spatial dimension. The spatial gating includes parallel second long-range feature extraction and second local feature extraction to obtain second long-range features and second local features, respectively. The second long-range features and second local features are then weighted to obtain spatiotemporal aggregated features. Specifically, the second long-range feature extraction and the second local feature extraction use a second large convolutional kernel and a second small convolutional kernel, respectively, to extract features from the time.

[0066] In one specific implementation, as shown in Expression 5, the second long-range feature is extracted using the second largest convolutional kernel, and channel recalibration is achieved through point convolution to generate soft-gated weights. :

[0067]

[0068] in, Represented as Activation function Represented as point convolution, Represented as weights, This is represented as a convolution operation. Represented as the time feature obtained in step 3-1), This is represented as the second largest convolution kernel.

[0069] Furthermore, the second largest convolutional kernel As a hyperparameter in the spatial dimension, its size is The aim is to model synchronization across hemispheres. The activation function is not unique; a method is employed. The activation function is used to compress the value of expression one to between 0 and 1. Alternatively, activation functions such as ReLU, Tanh, ELU, and GELU can also be used.

[0070] In one specific implementation, as shown in Expression 6, the second local features are extracted using the second small convolutional kernel, and channel recalibration is achieved through point convolution to generate soft-gated weights. :

[0071]

[0072] in, Represented as an activation function, This is represented as batch normalization processing. Represented as weights, This is represented as a convolution operation. Represented as the time feature obtained in step 3-1), This is represented as the second smallest convolution kernel.

[0073] Furthermore, the second smallest convolutional kernel As a hyperparameter in the spatial dimension, its size is The aim is to capture the spatial activation patterns of electrodes within a localized area. The activation function is not unique; it employs... Activation functions are used to improve the nonlinear fitting ability of feature maps and the stability of gradient propagation. Relatively speaking, activation functions can also include ReLU, Tanh, ELU, Sigmoid, etc.

[0074] Subsequently, the soft-gating weights were adjusted. and soft gating weights Perform element-wise Hadamard product as shown in Expression 7 for feature recalibration, and output the spatiotemporal aggregated features via residual projection:

[0075] in, Represented as spatiotemporal aggregation features, This is represented as the high-dimensional feature output in step 2). Represented as a point matrix, The soft-gating weights obtained from Expression 1 are represented as follows. The soft-gating weights obtained from expression two are represented as follows. It is represented as an element-level Hadamard product.

[0076] Step 3-3) Use global spatial operators to perform deep fusion of spatiotemporal aggregated features and compress them into sequence form.

[0077] In one specific implementation, as shown in Expression 8, the global spatial operator is a convolution kernel. The global spatial operator is used to perform deep fusion of information from all electrodes.

[0078]

[0079] in, Represented as the spatiotemporal aggregation features after deep fusion. Represented as an activation function, This is represented as batch normalization processing. Represented as weights, This is represented as a convolution operation. Represented as spatiotemporal aggregation features, It is represented as a global space operator.

[0080] Finally, the spatiotemporal aggregated features after deep fusion are... Using kernel size The convolutional kernel with 48 groups is compressed and transposed into a sequence form, ultimately yielding the spatiotemporal aggregated features in sequence form. .

[0081] In one specific embodiment, the specific implementation process of step S4) can be further divided into: Figure 4 The following steps are shown:

[0082] Step S4-1) Perform forward spatial state encoding on the spatiotemporal aggregation features to obtain forward state features. Specifically, a Mamba encoder is used to perform forward spatial state encoding on the spatiotemporal aggregation features.

[0083] In one specific implementation, as shown in Expression 9, a forward spatial state code is constructed using a Mamba encoder to capture the forward temporal evolution of spatiotemporal aggregation features.

[0084]

[0085] in, This is represented as forward space state coding. Represented as a Mamba encoder, Represented as standardization, Spatiotemporal aggregation features are represented as sequences.

[0086] Step S4-2) Perform backward spatial state encoding on the spatiotemporal aggregation features to obtain backward state features. Specifically, the spatiotemporal aggregation features are backward spatial state encoded using a Mamba encoder.

[0087] In one specific implementation, as shown in Expression 10, a backward spatial state code is constructed using a Mamba encoder to capture the backward temporal evolution of spatiotemporal aggregation features.

[0088]

[0089] in, This is represented as backward space state coding. This is represented as a sequence reversal operation. Represented as a Mamba encoder, Represented as standardization, Spatiotemporal aggregation features are represented as sequences.

[0090] Step S4-3) The forward spatial state code and the backward spatial state code are fused to obtain the fused features.

[0091] In a specific implementation, as shown in Expression 11, the forward spatial state encoding and the backward spatial state encoding are fused together, and residual processing is performed simultaneously to obtain fused features.

[0092]

[0093] in, Represented as fusion features, Spatiotemporal aggregated features represented in sequence form, This indicates regularization processing. This represents the forward space state encoding. This indicates backward space state encoding.

[0094] In a specific embodiment of step 5), the fused features are weighted to obtain the emotion classification result of the EEG signal. Specifically, the weighting process can be performed using a conventional classification head, such as a feedforward neural network, where the fused features are flattened into a one-dimensional vector and then input into a feedforward neural network for weighting. Further, the fused features are first pooled, and then processed through a linear classification layer (which can be a feedforward neural network) to obtain the emotion classification result of the EEG signal.

[0095] The pooling can employ global average pooling, global max pooling, or other pooling methods to downsample the fused features and accelerate computation. The linear classification layer can be a feedforward neural network, which includes multiple hidden layers and outputs corresponding to the sentiment classification results. The feedforward neural network may also include normalization and regularization processing, where the normalization processing can be any one of layer normalization, batch normalization, instance normalization, or group normalization.

[0096] To facilitate a deeper understanding of the present invention by those skilled in the art, a specific embodiment will be described below. The present invention provides a brainwave emotion recognition method based on a gating time-space mechanism and a bidirectional state-space model, which can effectively identify the emotions corresponding to brainwave signals. The method specifically includes the following:

[0097] 1. Data preparation and model configuration

[0098] Standard EEG emotion datasets (such as SEED or SEED-IV) were selected as training data. The original EEG signals were downsampled to 200Hz and segmented into 1-second non-overlapping samples, which were then Z-score standardized.

[0099] The above LS-Mamba model is built, with the optimizer configured as AdamW and the learning rate set to [value missing]. The weight decay coefficient is Set the batch size to 64.

[0100] 2. Training and Testing Process

[0101] A subject-dependent cross-validation strategy was employed for model training. During training, the model first automatically learned to suppress artifacts such as electrooculogram (EOG) interference through the LS module, and then captured the dynamic patterns of emotional state evolution over time through a bidirectional Mamba encoder.

[0102] 3. Application Effect Verification

[0103] like Figure 5 and Figure 6As shown, the LS-Mamba model provided by this invention achieves accuracies of 89.96% and 76.48% on the SEED and SEED-IV datasets, respectively, outperforming existing CNN and Transformer baseline models, and maintaining performance stability across different subjects. In summary, the EEG emotion recognition method based on gating temporal control and a bidirectional state-space model provided by this invention has the following advantages:

[0104] (1) High recognition accuracy: In the three-class classification task (positive, neutral, negative), the average recognition accuracy reaches 89.96%; in the complex four-class classification task (happy, sad, fear, neutral), the average accuracy reaches 76.48%. This result is better than the existing mainstream models such as EEGNet, LGGNet and Conformer.

[0105] (2) Low resource consumption: Thanks to Mamba's linear complexity and shared convolution design, the model has only 0.06 M parameters (about 60,000 parameters), which is much lower than similar Transformer models (usually greater than 0.3 M). This means that while ensuring high accuracy, the method significantly reduces the consumption of computing resources and can be deployed in portable or computationally limited brain-computer interface devices.

[0106] Since the model training process, inference process, and the entire implementation process have been described in detail in the above embodiments, they will not be repeated here.

[0107] It should also be noted that, in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect.

[0108] It should be noted that in the embodiments of the present invention, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0109] In this embodiment of the invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0110] Figure 7 This is a schematic block diagram of an EEG emotion recognition system based on gating time-space regulation and a bidirectional state-space model, provided in an embodiment of the present invention. Figure 7 As shown, the system includes: a feature acquisition module 51 for acquiring shallow features of the EEG signal; a feature projection module 52 for projecting the shallow features to obtain corresponding high-dimensional features; a gating processing module 53 for performing hierarchical gating processing on the high-dimensional features in both time and space dimensions to obtain spatiotemporal aggregated features; a spatial encoding module 54 for performing bidirectional state-space encoding on the spatiotemporal aggregated features to obtain fused features; and a classification module 55 for performing weighted processing on the fused features to obtain the emotion classification result of the EEG signal.

[0111] It should be understood that the specific processes by which each module performs the corresponding steps described above have been detailed in the above method embodiments, and will not be repeated here for the sake of brevity. It should also be understood that the module division in the embodiments of this invention is illustrative and merely a logical functional division; other division methods may exist in actual implementation. Furthermore, the functional modules in the various embodiments of this invention can be integrated into a single processor, exist as separate physical entities, or have two or more modules integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0112] Figure 8 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. Figure 8 As shown, the computer device includes at least one processor 601, a memory 602, at least one network interface 603, and a user interface 605. The various components in the device are coupled together via a bus system 604. It is understood that the bus system 604 is used to implement communication between these components. In addition to a data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 8 In EEG emotion recognition based on gating time-space control and bidirectional state-space model, all buses are labeled as bus systems.

[0113] The user interface 605 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0114] It is understood that memory 602 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0115] In this embodiment of the invention, the memory 602 is used to store various types of data to support the operation of the electronic terminal 600. Examples of this data include: any executable program for operation on the electronic terminal 600, such as the operating system 6021 and application program 6022; the operating system 6021 includes various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 6022 may include various applications, such as media players, browsers, etc., for implementing various application services. The EEG emotion recognition method based on gating time-space mechanism and bidirectional state space model provided in this embodiment of the invention can be included in the application program 6022.

[0116] The methods disclosed in the above embodiments of the present invention can be applied to processor 601, or implemented by processor 601. Processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 601 or by instructions in the form of software. The processor 601 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 601 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 601 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0117] In an exemplary embodiment, the electronic terminal 600 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to perform the aforementioned method.

[0118] According to the method provided in the embodiments of the present invention, the present invention also provides a computer program product, the computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform... Figures 1 to 4 The EEG emotion recognition method based on gating time-space mechanism and bidirectional state space model in any of the embodiments shown.

[0119] According to the method provided in the embodiments of the present invention, the present invention also provides a computer-readable storage medium storing program code, which, when run on a computer, causes the computer to perform the above-described method.

[0120] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0121] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0122] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0123] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0124] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0125] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0126] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0127] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0128] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0129] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

[0130] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A brainwave emotion recognition method based on gating time-space mechanism and bidirectional state-space model, characterized in that, The method includes: Obtain shallow features of EEG signals; Projecting the shallow features yields the corresponding high-dimensional features; The high-dimensional features are subjected to hierarchical gating processing in both time and space dimensions to obtain spatiotemporal aggregated features; The spatiotemporal aggregation features are bidirectionally state-space encoded to obtain fused features; The fusion features are weighted to obtain the emotion classification result of the EEG signal.

2. The method according to claim 1, characterized in that, In the process of projecting the shallow features to obtain the corresponding high-dimensional features, a projection layer is used to project the shallow features. The projection layer includes multiple convolutional kernels, and the number of channels of the high-dimensional features is consistent with the number of convolutional kernels.

3. The method according to claim 1, characterized in that, The step of performing hierarchical gating processing on the high-dimensional features in both time and space dimensions to obtain spatiotemporal aggregated features further includes: Time-gated processing is performed on high-dimensional features in the time dimension. The time-gated processing includes parallel first long-range feature extraction and first local feature extraction to obtain first long-range features and first local features, respectively. The first long-range features and first local features are weighted and processed to obtain time features. Spatial gating processing is performed on temporal features in the spatial dimension. The spatial gating processing includes parallel second long-range feature extraction and second local feature extraction to obtain second long-range features and second local features respectively. The second long-range features and second local features are weighted to obtain spatiotemporal aggregated features.

4. The method according to claim 3, characterized in that, The first long-range feature extraction and the first local feature extraction respectively use the first large convolution kernel and the first small convolution kernel to extract features from the high-dimensional features; the second long-range feature extraction and the second local feature extraction respectively use the second large convolution kernel and the second small convolution kernel to extract features from the time.

5. The method according to claim 1, characterized in that, The process of bidirectional state-space encoding of the spatiotemporal aggregation features to obtain fused features also includes: The spatiotemporal aggregation features are subjected to forward spatial state encoding to obtain forward state features; The spatiotemporal aggregation features are subjected to backward spatial state encoding to obtain backward state features; The forward spatial state encoding and the backward spatial state encoding are fused to obtain fused features.

6. The method according to claim 1, characterized in that, In the process of weighting the fused features to obtain the emotion classification result of the EEG signal, the fused features are pooled and then processed through a linear classification layer to obtain the emotion classification result of the EEG signal.

7. A brainwave emotion recognition system based on gating time-space regulation and a bidirectional state-space model, comprising: The feature acquisition module is used to acquire shallow features of EEG signals; The feature projection module is used to project the shallow features to obtain the corresponding high-dimensional features; The gating processing module is used to perform hierarchical gating processing on the high-dimensional features in both time and space dimensions to obtain spatiotemporal aggregated features; The spatial encoding module is used to perform bidirectional state-space encoding on the spatiotemporal aggregation features to obtain fused features; The classification module is used to perform weighted processing on the fused features to obtain the emotion classification result of the EEG signal.

8. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the EEG emotion recognition method based on gating temporal control and bidirectional state space model as described in any one of claims 1 to 6.

9. A computer program product comprising computer program code, wherein when the computer program code is run on a computer, the computer implements the EEG emotion recognition method based on gating time-space mechanism and bidirectional state space model as described in any one of claims 1 to 6.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory; the processor executing the computer program to implement the EEG emotion recognition method based on gating timing mechanism and bidirectional state space model as described in any one of claims 1 to 6.

Images