A motor imagery recognition method based on three-dimensional electroencephalogram signal representation and Mamba

Abstract

The present application relates to the field of brain-computer interface, and particularly relates to a motor imagery recognition method based on three-dimensional electroencephalogram signal representation and Mamba, which comprises collecting electroencephalogram data, saving the electroencephalogram data in the form of a two-dimensional matrix according to electrode positions, and stacking the electroencephalogram data along a time dimension after intercepting the electroencephalogram data according to a set time step to form three-dimensional electroencephalogram data; in a first Mamba module, each time slice of the three-dimensional electroencephalogram data is processed in turn, and the three-dimensional electroencephalogram data is stacked along the time dimension at a source end to obtain first three-dimensional feature data; the first three-dimensional feature data is respectively cut according to rows and columns of the form of a two-dimensional matrix, and is respectively input into a second Mamba module and a third Mamba module, the outputs of the second Mamba module and the third Mamba module are added to obtain fused features, and the features are used for classification; compared with the prior art, the present application realizes efficient and low-computing-cost motor imagery classification of EEG data.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interfaces (BCI), and more particularly to a method for recognizing motor imagery based on three-dimensional electroencephalogram (EEG) signal representation and Mamba. Background Technology

[0002] Brain-computer interfaces (BCIs) are technologies that enable communication between the brain and external devices via electroencephalogram (EEG) signals. In recent years, they have received widespread attention in fields such as medical rehabilitation, intelligent control, and human-computer interaction. Particularly in motor imagery (MI) classification tasks, users control external devices, such as prosthetics or robots, by imagining limb movements. However, traditional EEG signal processing and classification methods have limitations in handling complex spatiotemporal features and addressing individual differences.

[0003] Currently, most common methods for classifying motor imagery EEG signals are based on two-dimensional feature extraction, such as time series analysis, frequency domain features, or simple spatial projection. However, these methods fail to fully utilize the three-dimensional characteristics of EEG signals (spatial, electrode distribution, and temporal), making it difficult for the models to capture the rich spatiotemporal information of the signals. Furthermore, conventional feature extraction methods are inadequate when dealing with nonlinear and non-stationary signals, and are susceptible to noise and artifacts, thus affecting classification performance and model robustness. Summary of the Invention

[0004] To address the shortcomings of existing technologies in classifying motor imagery EEG signals, this invention proposes a method for motor imagery recognition based on three-dimensional EEG signal representation and Mamba, comprising the following steps:

[0005] Collect EEG data, preprocess the EEG data, and save the preprocessed EEG data in the form of a two-dimensional matrix according to the electrode position;

[0006] Set a time step, extract the EEG data in the form of a two-dimensional matrix according to the set time step, and stack the extracted data along the time dimension to form three-dimensional EEG data;

[0007] A feature extraction module consisting of three Mamba modules is constructed. In the first Mamba module, each time segment of the three-dimensional EEG data is processed sequentially, and the data is stacked along the time dimension at the source to obtain the first three-dimensional feature data.

[0008] The first three-dimensional feature data is divided into rows and columns in the form of a two-dimensional matrix;

[0009] The row slices and column slices obtained from the segmentation are input into the second Mamba module and the third Mamba module respectively. The outputs of the second Mamba module and the third Mamba module are added together to obtain the fused feature.

[0010] The obtained fused features are input into a pre-trained classifier to obtain the probability distribution of motion imagery categories.

[0011] Furthermore, preprocessing of the EEG data includes downsampling, bandpass filtering, and artifact removal.

[0012] Furthermore, the preprocessed EEG data is saved in the form of a two-dimensional matrix according to the electrode locations, that is, the preprocessed EEG data is mapped into a two-dimensional matrix that conforms to the electrode distribution of the international 10-20 system.

[0013] Furthermore, the three-dimensional EEG data is represented as Where C represents the number of electrodes, T represents the number of time point samples, and F represents the data length collected by a single electrode within a time step.

[0014] Furthermore, the classifier consists of two fully connected layers, a SELU activation layer, and a Softmax layer.

[0015] This invention combines 3D EEG representation and a Mamba feature extraction module to comprehensively and effectively extract spatiotemporal features from EEG signals, significantly improving the classification accuracy of motor imagery tasks. Compared with traditional methods, this invention can reduce information loss while maintaining model stability, enhance generalization ability for different users and tasks, and is suitable for real-time brain-computer interface applications such as prosthetic control, rehabilitation training, and other intelligent interactive systems. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the training process of the EEG signal motion recognition model of the present invention.

[0017] Figure 2 A cross-subject classification accuracy plot for different models on the BCI competition IV 2a dataset. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To address the shortcomings of existing technologies in classifying motor imagery EEG signals, this invention proposes a motor imagery recognition method based on three-dimensional EEG signal representation and Mamba, such as... Figure 1 The specific steps are as follows:

[0020] Collect EEG data, preprocess the EEG data, and save the preprocessed EEG data in the form of a two-dimensional matrix according to the electrode position;

[0021] Set a time step, extract the EEG data in the form of a two-dimensional matrix according to the set time step, and stack the extracted data along the time dimension to form three-dimensional EEG data;

[0022] A feature extraction module consisting of three Mamba modules is constructed. In the first Mamba module, each time segment of the three-dimensional EEG data is processed sequentially, and the data is stacked along the time dimension at the source to obtain the first three-dimensional feature data.

[0023] The first three-dimensional feature data is divided into rows and columns in the form of a two-dimensional matrix;

[0024] The row slices and column slices obtained from the segmentation are input into the second Mamba module and the third Mamba module respectively. The outputs of the second Mamba module and the third Mamba module are added together to obtain the fused feature.

[0025] The obtained fused features are input into a pre-trained classifier to obtain the probability distribution of motion imagery categories.

[0026] like Figure 1 As shown, this invention proposes a method for classifying motor imagery EEG signals based on 3D EEG representation and Mamba feature extraction, comprising the following steps:

[0027] S1. Collect EEG signals from users during different motor imagery tasks, and preprocess them through downsampling, bandpass filtering, and ICA artifact removal to remove noise and artifacts.

[0028] S2. Map the preprocessed EEG data into a two-dimensional electrode matrix conforming to the international 10-20 system, and stack them along the time dimension to form a three-dimensional EEG tensor X.

[0029] S3. Input the three-dimensional EEG tensor X into the Mamba feature extraction module, extract spatiotemporal features through forward and masked backward SSM layers, and obtain the output matrix;

[0030] S4. Input the extracted features into the classifier, which includes a fully connected layer, a SELU activation layer and a Softmax layer, and finally output the classification result of the motion imagination task.

[0031] S5. Calculate the overall loss function during model training, including classification loss, forward stability loss, and backward consistency loss, and use the ADAM optimizer for training and parameter updates.

[0032] This invention uses the BCI Competition IV 2a subtask dataset as experimental data. This dataset contains EEG signals from 9 subjects. Each subject participated in two experiments, each consisting of 6 rounds and 48 trials per round. Each motor imagery task (left hand, right hand, both feet, and tongue) was performed 12 times, for a total of 288 trials per experiment. During the experiment, subjects were asked to sit in a comfortable chair and perform motor imagery tasks in front of a computer screen according to prompts. At the beginning of each trial, a fixed crosshair was displayed on the screen accompanied by a prompting sound. After 2 seconds, an indicator arrow appeared and remained for 1.25 seconds, prompting the subject to perform the corresponding motor imagery task, until the prompting ended at 6 seconds.

[0033] Preprocessing of EEG signals includes:

[0034] First, the sampling rate was reduced from 250Hz to 200Hz to decrease computational complexity and data redundancy while retaining sufficient frequency information for subsequent analysis. Next, a bandpass filter of 0.5–50Hz was used to filter the signal to remove low-frequency drift and high-frequency noise, ensuring that the signal primarily contains useful information within the motion-image-related frequency band. Subsequently, independent component analysis (ICA) was applied to remove artifacts from the preprocessed EEG signal, particularly eliminating non-EEG artifacts such as electrooculography (EOG) and electromyography (EMG), to improve signal purity and analytical accuracy.

[0035] After data preprocessing, the EEG data is segmented into non-overlapping time windows of 1 second in length to generate uniform time segments. These time segments contain the EEG signals of each electrode within the predetermined time window. The data within each time window is reorganized into a two-dimensional matrix conforming to the international 10-20 system for electrode positions. This matrix encodes the spatial positional relationships of the electrodes, enabling subsequent feature extraction to simultaneously consider spatial correlations.

[0036] The two-dimensional electrode matrices of each time window are stacked along the time dimension to form a three-dimensional EEG tensor. Where C represents the number of electrode channels, T represents the number of time steps, and F represents the feature dimension calculated at each electrode location. This method of constructing a three-dimensional structure ensures the spatial and temporal integrity of the EEG data and can provide information including the spatial layout of the electrodes and their dynamic changes over time to the subsequent feature extraction module.

[0037] The core objective of this 3D EEG data construction process is to transform raw EEG signals into a multi-dimensional structure, providing multi-faceted feature representation and optimizing the performance of downstream tasks such as motor imagery classification. Through this preprocessing and 3D data construction, useful information from the EEG signals is extracted and organized, providing high-quality input data for subsequent feature extraction modules.

[0038] During training, Leave-One-Subject-Out (LOSO) cross-validation was used to split the dataset. In each validation iteration, data from one subject was selected as the test set, while data from the other eight subjects were used as the training and validation sets, respectively. This method ensures that the model does not see the test subject's data during training, thus allowing evaluation of the model's generalization performance on completely unknown subjects. By using each subject sequentially as the test set, the model's overall performance can be evaluated by averaging the results of multiple training iterations.

[0039] The preprocessed data is then fed into the Mamba feature extraction module to fully extract spatial and temporal features. The Mamba module is a bidirectional structure consisting of a forward SSM layer and a masked backward SSM layer, designed to capture the spatiotemporal dependencies and sequence features in EEG signals.

[0040] The feedforward SSM layer is responsible for processing the time-step features of the input tensor. Specifically, the input tensor X is first transformed linearly to generate query, key, and value matrices. These matrices are used to compute the attention weight matrix S, thereby highlighting features that are important at different time steps. This process helps the model capture the correlation and variation patterns of EEG signals over time, enabling it to focus on signal segments that are crucial to the classification task at different time steps.

[0041] The masked feedforward SSM layer further aggregates time-series information through recursive state matrix updates and calculations. This masking process ensures that the model can only access the current and previous steps at each time step, forming a feature capture mechanism similar to causality. This feedforward mechanism effectively enhances the model's consistency and stability over time, resulting in extracted features with stronger temporal relevance.

[0042] By combining forward and backward SSM layers, the Mamba module enables the fusion and enhancement of bidirectional information, thereby extracting richer spatiotemporal features. The output feature matrix contains a high-dimensional feature representation obtained after bidirectional feature extraction, which preserves the complex dynamic characteristics and spatial dependencies of the EEG signal.

[0043] In the Mamba module, the forward processing sequentially utilizes a linear layer, an SSM layer, a nonlinear activation layer, and a residual fusion unit to process the data, specifically including:

[0044] x′ t =Wx t +b

[0045] h t =Ah (t-1) +Bx′ t

[0046] y t =Ch t +Dx′ t

[0047] y′ t =ReLU(y t )

[0048] y out =y′ t +x′ t

[0049] Where, x t This represents the data input to the Mamba module at time step t, x′ t This represents the features obtained after the data input to the Mamba module at time step t is mapped through a linear layer; W and b are the weight and bias terms of the linear layer, respectively; h t Let h0 be the hidden state obtained in the forward feed of the SSM layer at time step t, where t is an integer greater than 0. h0 is initialized to x1, meaning h0 is initialized with the data input to the network at the first time step. (t-1) This represents the hidden state acquired in the forward SSM layer at time step t-1; y t Let y' be the feature representation output of the feedforward SSM layer at time step t; A, B, C, and D are the learnable state matrices; y'' t y represents the output of the nonlinear activation function; out This represents the output of the residual fusion unit.

[0050] To further improve the model's stability and generalization ability during training, regularization strategies are applied to the forward and backward SSM layers during feature extraction. This strategy ensures the stability of output features across different data batches by suppressing feature fluctuations and maintaining sequence consistency, reducing classification performance fluctuations caused by individual differences or external interference. The high-dimensional feature matrix output by the feature extraction module will be used for processing by downstream classifiers. Through this feature extraction process, the hidden spatiotemporal patterns in EEG signals are effectively captured and represented, thus providing strong support for accurate classification of motion imagery tasks.

[0051] The high-dimensional spatiotemporal feature matrix output by the Mamba feature extraction module is further input into the classifier for processing and classification. The classifier is designed to map complex spatiotemporal features to motion visualization task categories and output the probability distribution of each category. The classifier mainly consists of two fully connected layers, a SELU activation layer, and a Softmax layer to achieve efficient feature mapping and classification.

[0052] The first fully connected layer receives the features output from the Mamba module and performs dimensionality reduction. This step compresses the input feature space to a smaller dimension through a linear transformation of the weight matrix, thereby improving the efficiency of subsequent calculations and providing the classifier with a feature combination suitable for the classification task. The dimensionality-reduced features are then passed to the activation layer for nonlinear mapping.

[0053] The SELU activation layer is applied to the output of the first fully connected layer. The SELU (Scaled Exponential LinearUnit) activation function has self-regularization properties, effectively avoiding gradient explosion and vanishing gradient problems in deep neural networks, thus ensuring stable gradient flow in deep structures. This property is particularly important for the complexity and dynamics of features in EEG signals, guaranteeing that features extracted in motion visualization tasks can be fully represented.

[0054] The second fully connected layer further reduces the dimensionality of the features after SELU activation and combines them into the final classification input features. This process helps to perform a final transformation on the features before the output layer to better fit the input requirements of the Softmax layer. After this step, the features are adjusted to a shape and space suitable for classification.

[0055] The Softmax layer is the last layer of the classifier, used to map the output of the previous layer to the probability distribution of each motion imagery task. The Softmax function transforms the input features into a probability vector, where each component corresponds to a specific motion imagery category. The sum of the output probabilities is 1, and the model uses the category with the highest predicted probability as the final classification result for the motion imagery task. This process intuitively reflects the model's confidence in different categories.

[0056] The output of the Softmax layer not only predicts the category of the motor imagery task but also provides a probability value for each category as a confidence metric for the model's decision. This allows for adaptive decision-making based on the confidence of the prediction results in a real-time brain-computer interface system. To improve classifier performance, a cross-entropy loss function is used as the optimization objective, measuring the difference between the classification output and the true label. Throughout training, the learning rate is dynamically adjusted using the ADAM optimizer to ensure rapid model convergence and prevent overfitting. The classifier's performance is monitored in real-time using a validation set during training, and model parameters are adjusted as needed to further optimize performance.

[0057] The loss function for training the feature extraction module and the classifier is expressed as:

[0058] L total =L c +λ1L residual +λ2L backward

[0059] Among them, L total L represents the total loss function; c L represents classification loss; residual L represents the forward stability loss; backward Represents backward consistency loss; λ1 is used to control the residual loss term L. residual Importance weights; λ² is used to control the backpropagation loss term L. backward Importance weights.

[0060] Forward stability loss L residual Represented as:

[0061]

[0062] Where M represents the number of training iterations; δ is a threshold parameter used to control the acceptable error range; x i This represents the true classification label of the i-th batch of training data; This represents the predicted classification label of the i-th batch of training data.

[0063] Backward consistency loss L residual Represented as:

[0064]

[0065] Where N is the number of time steps used to calculate the backward consistency loss; h i This represents the hidden state obtained in the forward SSM layer at time step i; This represents the hidden state obtained from the SSM layer after time step i.

[0066] In this embodiment, the update process of the hidden state in the backward SSM layer includes:

[0067]

[0068] in, This represents the hidden state obtained from the SSM layer after time step t. This represents the hidden state obtained from the SSM layer after time step t+1; A′ and B′ are learnable parameters.

[0069] In this embodiment, if data from the current T time steps has been acquired, then N = T-1. During backpropagation, updates are performed sequentially from time step T-1 to time step 1. When t = T-1, initialization is performed. x ′ T This represents the feature obtained after the data from the input Mamba module at time step T is mapped through a linear layer.

[0070] The model training employs a combination of multiple loss functions to optimize different feature learning objectives. The primary loss function is cross-entropy loss, used to evaluate the matching degree between the classifier output and the true labels. Cross-entropy loss quantifies the accuracy of model predictions and is a widely used loss function in supervised learning. Furthermore, to improve the model's stability and consistency on time-series data, forward stability loss and backward consistency loss are employed. Forward stability loss constrains the model's output features in adjacent time steps, ensuring the smoothness and stability of the feature extraction process. Backward consistency loss, through recursive calculation on the time series, guarantees the consistency of the model output across the time dimension, avoiding drift and instability of long-timestep features.

[0071] During each training epoch, the model iterates through the entire training set and updates its parameters according to the loss function after each batch. After each epoch, the model's performance is evaluated using a validation set. The loss and accuracy on the validation set are used to determine whether the model is overfitting or underfitting. To prevent the model from overfitting the training data in the later stages of training, an early stopping strategy is applied, i.e., training stops when the performance on the validation set no longer improves.

[0072] During training, hyperparameters, such as the learning rate and weights of the loss function, are dynamically adjusted based on performance on the validation set. This adjustment strategy combines grid search and validation set monitoring to ensure the model maintains good robustness under different training conditions. To further improve the model's generalization ability, an L2 regularization term is added during training to suppress excessive growth of model parameters and reduce the risk of overfitting caused by complex models. Furthermore, dropout is applied in fully connected layers to randomly discard a certain proportion of neurons in each training epoch, thereby improving the model's robustness to noise and its generalization ability.

[0073] After training, the model is evaluated using a test set. Classification performance is comprehensively measured by calculating metrics such as classification accuracy, recall, and F1 score. Throughout training and validation, confusion matrices and other visualization tools are used to analyze the distribution of classification results and identify the model's strengths and weaknesses in different motion visualization tasks.

[0074] like Figure 2 Compared with current mainstream motor imagery classification models, including the traditional FBCSP, the pure Transformer EEG Transformer, the CNN-Transformer which combines CNN and Transformer, and the CNN channel selection method CMO-CNN, the three-dimensional EEG signal representation and Mamba feature extraction method proposed in this invention achieves the best average accuracy, exceeding existing methods by nearly 1 percentage point.

[0075] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for recognizing motor imagery based on three-dimensional electroencephalogram (EEG) signal representation and Mamba, characterized in that, The specific steps are as follows: Collect EEG data, preprocess the EEG data, and save the preprocessed EEG data in the form of a two-dimensional matrix according to the electrode position; Set a time step, extract the EEG data in the form of a two-dimensional matrix according to the set time step, and stack the extracted data along the time dimension to form three-dimensional EEG data; A feature extraction module consisting of three Mamba modules is constructed. In the first Mamba module, each time segment of the three-dimensional EEG data is processed sequentially, and the data is stacked along the time dimension at the source to obtain the first three-dimensional feature data. The first three-dimensional feature data is divided into rows and columns in the form of a two-dimensional matrix; The row slices and column slices obtained from the segmentation are input into the second Mamba module and the third Mamba module respectively. The outputs of the second Mamba module and the third Mamba module are added together to obtain the fused feature. The obtained fused features are input into a pre-trained classifier to obtain the probability distribution of motion imagery categories.

2. The method for recognizing motor imagery based on three-dimensional EEG signal representation and Mamba, as described in claim 1, is characterized in that... Preprocessing of EEG data includes downsampling, bandpass filtering, and artifact removal.

3. The method for recognizing motor imagery based on three-dimensional EEG signal representation and Mamba, as described in claim 1, is characterized in that... The preprocessed EEG data is saved in the form of a two-dimensional matrix according to the electrode location, that is, the preprocessed EEG data is mapped into a two-dimensional matrix that conforms to the electrode distribution of the international 10-20 system.

4. The method for recognizing motor imagery based on three-dimensional EEG signal representation and Mamba, as described in claim 3, is characterized in that... Three-dimensional EEG data are represented as X∈R C×T×F Where C represents the number of electrodes, T represents the number of time point samples, and F represents the data length collected by a single electrode within a time step.

5. The method for recognizing motor imagery based on three-dimensional EEG signal representation and Mamba, as described in claim 1, is characterized in that... The Mamba module includes linear layers, SSM layers, nonlinear activation layers, and residual fusion units for data processing, specifically including: x′ t =Wx t +b h t =Ah (t-1) +Bx′ t y t =Ch t +Dx′ t and' t =ReLU(y t ) and out =y′ t +x′ t Where, x t This represents the data input to the Mamba module at time step t, x′ t This represents the features obtained after the data input to the Mamba module at time step t is mapped through a linear layer; W and b are the weight and bias terms of the linear layer, respectively; h t h represents the hidden state obtained in the forward SSM layer at time step t; (t-1) This represents the hidden state acquired in the forward SSM layer at time step t-1; y t Let y' be the feature representation output of the feedforward SSM layer at time step t; A, B, C, and D are the learnable state matrices; y'' t y represents the output of the nonlinear activation function; out This represents the output of the residual fusion unit.

6. The method for recognizing motor imagery based on three-dimensional EEG signal representation and Mamba, as described in claim 1, is characterized in that... The classifier consists of two fully connected layers, a SELU activation layer, and a Softmax layer.

7. The method for recognizing motor imagery based on three-dimensional EEG signal representation and Mamba, as described in claim 1, is characterized in that... The loss function for training the feature extraction module and the classifier is expressed as: L total =L c +λ1L residual +λ2L backward Among them, L total L represents the total loss function; c L represents classification loss; residual L represents the forward stability loss; backward Represents backward consistency loss; λ1 is used to control the residual loss term L. residual Importance weights; λ² is used to control the backpropagation loss term L. backward Importance weights.

8. The method for recognizing motor imagery based on three-dimensional EEG signal representation and Mamba as described in claim 7, with forward stability loss L residual Represented as: in, M represents the number of training iterations; δ is a threshold parameter used to control the acceptable error range; y i This represents the true classification label of the i-th batch of training data; This represents the predicted classification label of the i-th batch of training data.

9. The method for recognizing motor imagery based on three-dimensional EEG signal representation and Mamba as described in claim 7, with backward consistency loss L residual Represented as: in, N is the number of time steps used to calculate the backward consistency loss; h i This represents the hidden state obtained in the forward SSM layer at time step i; This represents the hidden state obtained from the SSM layer after time step i.

10. The method for recognizing motor imagery based on three-dimensional EEG signal representation and Mamba, according to claim 9, wherein the update process of the hidden state in the backward SSM layer includes: in, This represents the hidden state obtained from the SSM layer after time step t. This represents the hidden state obtained from the SSM layer after time step t+1; A′ and B′ are learnable parameters.

Images