A motor imagery electroencephalogram signal decoding method, system and device based on a double-path hierarchical hybrid architecture

Abstract

The application discloses a motor imagery electroencephalogram signal decoding method, system and device based on a double-path hierarchical hybrid architecture. The method comprises the following steps: acquiring a multi-channel motor imagery electroencephalogram signal; extracting initial space-time features through a convolution embedding module to obtain embedded features; inputting the embedded features into a double-path hierarchical hybrid module stacked by a plurality of feature processing sub-modules, performing feature extraction on the embedded features layer by layer through a main path and an auxiliary path arranged in parallel, and adopting different feature extraction strategies according to the stacked levels; each layer fuses the output features of the double paths through an adaptive fusion mechanism to obtain deep features; and finally, a classifier module is used to output the prediction probability of the motor imagery category. The application effectively solves the problems of high calculation complexity, unbalanced feature extraction and insufficient robustness of the existing method, and significantly improves the decoding accuracy and efficiency of the motor imagery electroencephalogram signal.

Description

Technical Field

[0001] This invention relates to the field of motor imagery EEG signal decoding technology, and in particular to a method, system and device for decoding motor imagery EEG signals based on a dual-path hierarchical hybrid architecture. Background Technology

[0002] Motor imagery brain-computer interface (BCI) technology is a core technology connecting brain neural activity with external devices. It has irreplaceable application value in areas such as neurological rehabilitation training for patients with motor dysfunction and control of assistive interactive devices for people with disabilities. Its core technology lies in accurately decoding the electroencephalogram (EEG) signals generated when users imagine specific limb movements (such as hand and foot movements). However, EEG signals themselves have inherent properties such as extremely low signal-to-noise ratio, strong signal non-stationarity, and complex spatiotemporal dynamic characteristics. These characteristics make effective feature extraction difficult, becoming a key bottleneck restricting the accuracy and efficiency of decoding motor imagery EEG signals.

[0003] To improve EEG signal decoding performance, existing methods often employ hybrid architectures based on Transformers or convolutional neural networks to balance local feature extraction and global dependency modeling. However, when processing long-sequence EEG data, these methods suffer from high training and inference costs due to the secondary computational complexity of self-attention mechanisms, making them unsuitable for real-time applications. Furthermore, existing network structures fail to achieve a good balance between computational resource allocation and feature capture capabilities between shallow and deep layers, often introducing redundant computations in the early stages of feature extraction while lacking effective capture of multi-scale long-range dependencies in the crucial high-level semantic modeling stage. In addition, most methods rely on a single feature extraction path, making it difficult to enhance model robustness and expressive power from a complementary perspective. Therefore, there is an urgent need to improve existing technologies to enhance computational efficiency and feature robustness while maintaining decoding accuracy. Summary of the Invention

[0004] This application discloses a method, system, and device for decoding motor imagery EEG signals based on a dual-path hierarchical hybrid architecture, aiming to solve the problems of high computational complexity, unbalanced network structure resource allocation, and insufficient robustness caused by a single feature extraction path in existing EEG signal decoding methods when processing long sequence data.

[0005] The technical solution of this application is as follows:

[0006] In a first aspect, this application discloses a method for decoding motor imagery EEG signals based on a dual-path hierarchical hybrid architecture, including:

[0007] Acquire multi-channel motor imagery EEG signals;

[0008] The multi-channel motion imagery EEG signal is input into the convolutional embedding module to extract preliminary spatiotemporal features and obtain the embedding features;

[0009] The embedded features are input into a dual-path hierarchical hybrid module consisting of multiple stacked feature processing sub-modules. The main path and auxiliary path set in parallel by the dual-path hierarchical hybrid module are used to extract features from the embedded features layer by layer, and different feature extraction strategies are adopted according to the stacking level.

[0010] The output features of the main path and the auxiliary path are fused through an adaptive fusion mechanism to obtain deep features for classification.

[0011] The deep features are input into the classifier module, which outputs the predicted probability of the motion image category.

[0012] This technical solution achieves a deep balance between computational efficiency and feature representation capability by constructing a dual-path hierarchical hybrid architecture and employing differentiated feature extraction strategies at different network levels. Lower levels utilize the computationally efficient Mamba module for basic feature extraction, while higher levels introduce a more expressive hybrid attention module to capture multi-scale dependencies. This, combined with the multi-head self-attention mechanism of the auxiliary path, provides complementary features. Finally, an adaptive fusion mechanism dynamically integrates the dual-path information, significantly improving the decoding accuracy and robustness of motor imagery EEG signals.

[0013] Furthermore, the step of the convolutional embedding module extracting preliminary spatiotemporal features includes:

[0014] The multi-channel motion imagery EEG signal is input into a temporal convolutional layer, and convolution operation is performed on each electrode channel along the time dimension.

[0015] The features obtained after temporal convolution are input into the spatial convolutional layer, and convolution operation is performed covering all electrode channels.

[0016] The features after spatial convolution are sequentially subjected to batch normalization and activation function processing.

[0017] The processed features are downsampled in the time dimension and adjusted in the feature dimension to obtain the embedded features.

[0018] Furthermore, the dual-path hierarchical hybrid module includes M stacked feature processing sub-modules, where M is a positive even number. The first M / 2 feature processing sub-modules are defined as low-level feature processing sub-modules, and the last M / 2 feature processing sub-modules are defined as high-level feature processing sub-modules.

[0019] The different feature extraction strategies adopted according to the stacking level include:

[0020] In the low-level feature processing submodule, the main path uses dual Mamba modules for feature extraction.

[0021] In the high-level feature processing submodule, the main path uses a combination of a temporal Mamba module and a hybrid attention module for feature extraction.

[0022] The auxiliary path employs a multi-head self-attention mechanism for feature extraction in all feature processing submodules.

[0023] Furthermore, the dual Mamba module includes a temporal Mamba module and a spatial Mamba module, and the step of using the dual Mamba module for feature extraction of the main path in the low-level feature processing submodule includes:

[0024] The temporal Mamba module performs layer normalization on the output features of the previous layer and then scans along the sequence dimension. Local dependencies are captured through a selective state space mechanism to obtain temporally processed features.

[0025] The spatial Mamba module performs layer normalization on the temporal processing features and then scans along the feature dimensions to obtain the output features of the main path at the current lower layer.

[0026] Furthermore, the hybrid attention module includes parallel local attention branches and global attention branches;

[0027] The steps for feature extraction in the high-level feature processing submodule, which employ a combination of a temporal Mamba module and a hybrid attention module, include:

[0028] The temporal Mamba module performs layer normalization on the output features of the previous layer and then scans along the sequence dimension to process the temporal features, thus obtaining the temporal processed features.

[0029] The temporal processing features are input into the hybrid attention module, and the attention calculation range is limited to the local neighborhood by the local attention branch based on the window mask to obtain the local attention features.

[0030] The global attention branch enhances the long-range dependency modeling capability based on relative position bias and temperature adjustment parameters, thus obtaining global attention features;

[0031] The local attention features and the global attention features are concatenated and fused through a linear layer to obtain hybrid attention features. These hybrid attention features are then residually connected to the temporal processing features and normalized using a layer to obtain the output features of the main path at the current high layer.

[0032] Furthermore, the multi-head self-attention mechanism includes a spatial multi-head self-attention module and a temporal multi-head self-attention module;

[0033] The auxiliary path employs a multi-head self-attention mechanism for feature extraction, including the following steps:

[0034] Perform layer normalization and linear transformation on the output features of the previous layer to obtain the query matrix, key matrix and value matrix;

[0035] The query matrix, key matrix, and value matrix are input into the spatial multi-head self-attention module and processed along the feature dimension. Then, a residual connection is performed with the output feature of the previous layer to obtain the spatial processed feature.

[0036] After the spatial processing features are input into the temporal multi-head self-attention module and processed along the sequence dimension, a residual connection is made with the spatial processing features to obtain the output features of the auxiliary path in the current layer.

[0037] Furthermore, the adaptive fusion mechanism includes:

[0038] Concatenate the output features of the current layer's main path and auxiliary path along the feature dimension;

[0039] The concatenated features are input into a linear layer for processing, and then the dynamic gating weights are calculated using the Softmax function.

[0040] The output features of the two paths are weighted and summed according to the dynamic gating weights to obtain the final output features of the current layer.

[0041] The deep features used for classification are obtained by fusing all feature processing submodules layer by layer.

[0042] Furthermore, the steps of the classifier module in processing deep features include:

[0043] Flatten the depth feature into a one-dimensional vector;

[0044] Feature mapping is performed through a multi-layer fully connected network, with activation functions and Dropout regularization used in the intermediate layers.

[0045] The predicted probability of the motion image category is output through the Softmax function of the output layer.

[0046] The method also includes a model training process, which includes: calculating the loss value between the predicted probability and the true label using the cross-entropy loss function, and optimizing the model parameters using the Adam optimizer.

[0047] Secondly, this application also discloses a motor imagery EEG signal decoding system based on a dual-path hierarchical hybrid architecture, comprising:

[0048] The signal input module is used to acquire multi-channel motor imagery EEG signals;

[0049] The convolutional embedding module is used to process the EEG signal and extract preliminary spatiotemporal features to obtain embedding features;

[0050] The dual-path hierarchical hybrid processing module is used to extract features from the embedded features layer by layer through the main path and auxiliary path set in parallel, and to adopt different feature processing strategies according to the stacking level. The dual-path output features are fused through the adaptive fusion module to obtain deep features.

[0051] The classification module is used to process the depth features and output the predicted probability of the motion image category.

[0052] Thirdly, this application also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0053] The motor imagery EEG signal decoding method disclosed in this application, based on a dual-path hierarchical hybrid architecture, is innovative in that it employs a hierarchical dual-path architecture design. This design strategically combines the efficiency of the Mamba module with the powerful expressiveness of the attention mechanism at different network stages. Specifically, at the lower network layers, dual Mamba modules with linear computational complexity are used to quickly extract basic spatiotemporal features, avoiding redundant computation in the initial feature extraction phase. At the higher network layers, a hybrid attention module is introduced to accurately capture the multi-scale long-range dependencies crucial for decoding through parallel combination of local and global attention. The auxiliary path consistently employs a multi-head self-attention mechanism to provide stable global feature supplementation, effectively complementing the main path. The adaptive fusion mechanism adaptively integrates the information flow of the two paths through dynamic gating weights, further enhancing the model's robustness and expressiveness.

[0054] Through this innovative architectural design, this application effectively solves the core problems faced by existing EEG signal decoding methods: First, by replacing part of the attention calculation with the Mamba module, the computational complexity of the model when processing long-sequence EEG data is significantly reduced; Second, the hierarchical feature extraction strategy achieves a precise match between computational resources and feature requirements, improving computational efficiency while ensuring performance; Finally, the dual-path design extracts complementary features from different perspectives, and the adaptive fusion mechanism greatly enriches the model's feature representation capabilities. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating a method for decoding motor imagery EEG signals based on a dual-path hierarchical hybrid architecture, as provided in this application.

[0056] Figure 2 A schematic diagram of a method for decoding motor imagery EEG signals based on a dual-path hierarchical hybrid architecture provided in this application.

[0057] Figure 3 is a schematic diagram of the feature extraction method using the multi-head self-attention mechanism of this application.

[0058] Figure 4 This application provides a schematic diagram of a motor imagery EEG signal decoding system based on a dual-path hierarchical hybrid architecture.

[0059] Figure 5 This application provides a schematic diagram of a computer device structure for decoding motor imagery EEG signals based on a dual-path hierarchical hybrid architecture.

[0060] The following are the labeling elements in the figure:

[0061] 100. Signal input module; 200. Convolution embedding module; 300. Dual-path hierarchical hybrid processing module; 400. Classification module; 50. Processor; 51. Memory; 52. Computer program. Detailed Implementation

[0062] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0063] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0064] Please refer to Figure 1 and Figure 2 This application proposes a method for decoding motor imagery EEG signals based on a dual-path hierarchical hybrid architecture, comprising the following steps:

[0065] S100: Acquires multi-channel motor imagery EEG signals;

[0066] S200: Input the multi-channel motion imagery EEG signal into the convolutional embedding module to extract preliminary spatiotemporal features and obtain the embedding features;

[0067] S300: The embedded features are input into a dual-path hierarchical hybrid module consisting of multiple stacked feature processing sub-modules. The main path and auxiliary path set in parallel by the dual-path hierarchical hybrid module are used to extract features from the embedded features layer by layer, and different feature extraction strategies are adopted according to the stacking level.

[0068] S400: The output features of the main path and the auxiliary path are fused through an adaptive fusion mechanism to obtain deep features for classification;

[0069] S500: Input the deep features into the classifier module and output the predicted probability of the motion image category.

[0070] The multi-channel motor imagery EEG signal is obtained by synchronously acquiring multiple electrode channels arranged according to specific standards on the surface of the human scalp. It records the weak voltage fluctuations generated by the electrical activity of nerve cell groups in the sensorimotor cortex of the cerebral cortex when the subject is imagining specific limb movements (such as imagining the movement of the left hand, right hand, foot, or tongue). The convolutional embedding module can be specifically implemented as a neural network structure containing temporal and spatial convolutional layers. The temporal convolutional layer is responsible for processing single-channel signals along the time dimension to capture the temporal dynamic changes of motor imagery, while the spatial convolutional layer covers all electrode channels to establish spatial topological relationships. The dual-path hierarchical hybrid module refers to a feature processing unit formed by stacking multiple feature processing sub-modules. It contains parallel main and auxiliary paths and can dynamically adjust the feature extraction strategy according to the stacking hierarchy of feature processing. Furthermore, this module can employ a convolutional neural network-based feature extraction strategy at lower levels to efficiently capture local spatiotemporal patterns, and a recurrent neural network-based feature extraction strategy at higher levels to enhance long-range dependency modeling capabilities. The auxiliary path can utilize a wavelet transform-based feature extraction strategy to provide complementary time-frequency domain information, such as decomposing the signal frequency band and extracting time-frequency features through discrete wavelet transform, or generating a time-frequency representation matrix using continuous wavelet transform. The adaptive fusion mechanism refers to the process of dynamically generating fusion weights based on the characteristics of the output features of the main path and auxiliary paths.

[0071] In the decoding process of motor imagery EEG signals, multi-channel motor imagery EEG signals are first acquired as raw data input for subsequent feature extraction. This signal is then input into a convolutional embedding module, which performs preliminary feature extraction along the temporal and spatial dimensions, thereby separating the spatiotemporal coupling characteristics in the signal and effectively overcoming the feature aliasing problem caused by the non-stationarity of the EEG signal, resulting in embedded features. The embedded features are further input into a dual-path hierarchical hybrid module, which consists of multiple stacked feature processing sub-modules. This module extracts features layer by layer from the embedded features through parallel main and auxiliary paths, dynamically adjusting the feature extraction strategy according to the stacked hierarchy. The lower-level feature processing stage focuses on efficiently capturing local dependencies, while the higher-level feature processing stage strengthens the ability to model multi-scale long-range dependencies, thus achieving a reasonable allocation of computational resources and avoiding the defects of shallow redundant computation and insufficient deep feature capture. The output features of the main and auxiliary paths are then dynamically integrated through an adaptive fusion mechanism. This mechanism generates gating weights based on spliced ​​features and performs a weighted summation of the dual-path features, thereby adaptively balancing the contribution of local details and global semantics to obtain deep features for classification. Finally, the deep features are input into the classifier module, and through feature flattening and mapping with a fully connected network, the predicted probability of the motion image category is output, completing the decoding process.

[0072] For example, in one specific implementation, a method for decoding motor imagery EEG signals based on a dual-path hierarchical hybrid architecture includes the following steps:

[0073] Batch acquisition B Multichannel EEG signals X ∈ R B×C×T , C The number of EEG electrode channels is , T This represents the number of sampling points in the time series.

[0074] EEG signals X The convolutional embedding module is used to process the data and extract preliminary spatiotemporal features to obtain the embedded feature representation. E ∈ R B×L×D ,in L The length of the sequence after dimensionality reduction. D For feature dimensions, specifically:

[0075] embed features E Input to M The network employs a dual-path hierarchical hybrid module, composed of stacked modules, to process data. This module includes parallel main and auxiliary paths, and different strategies are used depending on the network layer. Finally, it outputs feature representations for classification. H L ∈ R B×L×D .

[0076] Finally, the output features from the dual-path hierarchical hybrid module will be processed. H L The input is fed into the classifier module to obtain the predicted probability of the motion image category.

[0077] Therefore, this technical solution enhances feature robustness from a complementary perspective through a dual-path collaborative mechanism, avoiding the limitations of a single feature extraction path. Simultaneously, the hierarchical heterogeneous feature extraction strategy ensures that the lower layers focus on efficient local feature processing to reduce computational complexity, while the higher layers strengthen multi-scale dependency modeling to improve semantic expressiveness. This systematically solves the problems of high training and inference costs and unbalanced allocation of computational resources between shallow and deep layers caused by the secondary computational complexity of the self-attention mechanism in existing methods. Furthermore, the adaptive fusion mechanism dynamically adjusts the feature contributions of the two paths, effectively addressing the challenges of low signal-to-noise ratio and complex spatiotemporal dynamics of EEG signals, significantly improving the adaptability of feature extraction and decoding accuracy, and providing reliable technical support for the real-time application of motor imagery brain-computer interfaces.

[0078] This application further proposes that the steps for the convolutional embedding module to extract preliminary spatiotemporal features include:

[0079] Multi-channel motion imagery EEG signals are input into a temporal convolutional layer, and convolution operations are performed on each electrode channel along the time dimension.

[0080] The features obtained after temporal convolution are input into the spatial convolutional layer, and convolution operation is performed covering all electrode channels.

[0081] The features after spatial convolution are sequentially subjected to batch normalization and activation function processing.

[0082] The processed features are downsampled in the time dimension and the feature dimension is adjusted to obtain the embedded features.

[0083] The temporal convolutional layer uses a one-dimensional convolutional kernel to independently process the time series of each electrode channel; the spatial convolutional layer integrates the spatial information of all electrode channels using a two-dimensional convolutional kernel. The features are then subjected to batch normalization and activation function processing, where the activation function can be an ELU or a sigmoid-like function. Subsequently, a pooling mechanism is used to achieve temporal downsampling, and a linear projection layer is used to adjust the number of feature channels.

[0084] As one specific implementation method, the solution of this application is implemented as follows:

[0085] Signal X Input is fed into a temporal convolutional layer, using a kernel size of 1× K Two-dimensional convolution, independently along the time dimension T Perform a convolution operation on each electrode channel;

[0086] The feature map after temporal convolution is input into the spatial convolutional layer, using a convolutional kernel size of [size missing]. C A 2D convolution of ×1, covering all electrode channels;

[0087] After the spatial convolutional layer, a batch normalization layer and the ELU activation function are applied;

[0088] Further downsampling of the time dimension is performed using an average pooling layer, reducing the sequence length from... T Down to L Then, the temporal feature dimension is adjusted to [value] using a 1×1 convolution. D , to obtain embedded features E ;

[0089] Through the above scheme, this application effectively reduces the redundant computational overhead in the early stage of feature extraction, fully captures the spatiotemporal dynamic characteristics of EEG signals, and establishes an optimized balance mechanism between computational efficiency and feature representation ability, thereby improving the accuracy and real-time performance of subsequent hierarchical feature processing.

[0090] This application further proposes that the dual-path hierarchical hybrid module includes M stacked feature processing sub-modules, where M is a positive even number. The first M / 2 feature processing sub-modules are defined as low-level feature processing sub-modules, and the last M / 2 feature processing sub-modules are defined as high-level feature processing sub-modules.

[0091] The different feature extraction strategies adopted according to the stacking level include:

[0092] In the low-level feature processing submodule, the main path uses dual Mamba modules for feature extraction.

[0093] In the high-level feature processing submodule, the main path uses a combination of a temporal Mamba module and a hybrid attention module for feature extraction.

[0094] The auxiliary path employs a multi-head self-attention mechanism for feature extraction in all feature processing submodules.

[0095] In practical applications, M being a positive even number means that the total number of feature processing submodules is set to an even value, which can be implemented using specific values ​​such as 4, 6, or 8. The purpose is to ensure symmetrical hierarchical division, so that the feature processing strategies of shallow and deep layers can be balanced to adapt to signal characteristics. Among them, the dual Mamba module refers to a sequence modeling structure based on a selective state-space model, which can be implemented using a scanning mechanism that decouples temporal and spatial dimensions or other efficient sequence processing techniques. The purpose is to replace the high-complexity attention mechanism to reduce the computational burden of shallow layers. Specifically, the hybrid attention module refers to a composite structure that integrates local and global attention mechanisms. It can be implemented using a combination of window masking and relative position encoding or other attention variants. The purpose is to balance fine-grained feature interaction and long-range dependency modeling. In addition, the multi-head self-attention mechanism refers to a feature extraction mechanism that computes multiple attention heads in parallel. It can be implemented using standard multi-head attention or cascaded variants of multi-head mechanisms. The purpose is to provide stable spatial-temporal joint representations to enhance model robustness.

[0096] Specifically, the proposed solution divides the dual-path hierarchical hybrid module into low-level and high-level feature processing sub-modules in a stacked order, matching the feature extraction process with the spatiotemporal dynamics of EEG signals. In the low-level feature processing sub-module, the main path uses dual Mamba modules to scan input features along both the sequence and feature dimensions, efficiently capturing short-term dependencies and spatial correlations between electrode channels and avoiding redundant computations introduced in the shallow layer. In the high-level feature processing sub-module, the main path combines a temporal Mamba module with a hybrid attention module. The temporal Mamba module first processes long-sequence temporal dynamics, while the hybrid attention module uses local branches to limit local neighborhood computations and global branches to enhance cross-regional correlations, thus accurately modeling multi-scale long-range dependencies. Simultaneously, the auxiliary path employs a multi-head self-attention mechanism for feature extraction throughout the module. Through cascaded processing in the spatial and temporal dimensions, it provides complementary joint representations for the dual paths, creating a synergistic enhancement effect with the main path. This hierarchical differentiation strategy allows the shallow layer to focus on refined local feature extraction while the deep layer focuses on semantic integration, effectively resolving the core contradictions of resource mismatch in the early stages of feature extraction and insufficient modeling in key stages.

[0097] As a specific implementation method, the scheme of this application is implemented as follows: M is set to 6, that is, the dual-path hierarchical hybrid module contains 6 feature processing sub-modules, of which the first 3 are defined as low-level feature processing sub-modules and the last 3 are defined as high-level feature processing sub-modules. In the low-level feature processing sub-modules, the dual Mamba module of the main path first performs layer normalization on the input features and then scans along the sequence dimension to capture temporal local dependencies, and then scans along the feature dimension; in the high-level feature processing sub-modules, the temporal Mamba module of the main path first performs temporal feature processing on the output of the previous layer, and then inputs the result into the hybrid attention module. Its local attention branch limits the calculation range based on window mask to achieve fine-grained interaction, and the global attention branch introduces relative position bias to enhance cross-regional correlation. The two are fused and connected with the temporal processing feature residual; the auxiliary path adopts a multi-head self-attention mechanism in all feature processing sub-modules. After processing the feature dimension through the spatial multi-head self-attention module, the sequence dimension is processed through the temporal multi-head self-attention module to form a cascaded feature extraction structure.

[0098] Through the above scheme, this application effectively reduces the waste of computational resources in shallow feature extraction, avoids the redundant overhead of high-complexity attention mechanisms in the local signal processing stage, and strengthens the ability of deep features to model the multi-scale spatiotemporal dynamic characteristics of EEG signals. It achieves a dynamic balance between fine extraction of local features and integration of global semantics, thereby improving the feature expression accuracy and computational efficiency in the decoding process of motor imagery EEG signals.

[0099] This application further proposes that the dual Mamba module includes a temporal Mamba module and a spatial Mamba module. The steps for feature extraction in the main path of the low-level feature processing submodule using the dual Mamba module include:

[0100] The temporal Mamba module performs layer normalization on the output features of the previous layer and then scans along the sequence dimension. Local dependencies are captured through a selective state space mechanism to obtain temporally processed features.

[0101] After performing layer normalization on the temporal processing features using the spatial Mamba module, the output features of the main path at the current lower layer are obtained by scanning along the feature dimensions.

[0102] Among them, the dual Mamba module refers to a feature extraction structure designed specifically for low-level EEG signal processing. It can be implemented using an architecture that decouples spatiotemporal dimension processing, aiming to optimize the extraction efficiency of temporal dynamics and spatial distribution features respectively. The temporal Mamba module refers to a component that scans features along the sequence dimension. It can be implemented based on a selective state-space model, aiming to efficiently capture local dependencies in the temporal dimension with linear computational complexity. The spatial Mamba module refers to a component that scans features along the feature dimension. It can use a state-space mechanism to handle the relationships between electrode channels, aiming to enhance the modeling ability of spatial distribution patterns of EEG signals. Layer normalization refers to the operation of normalizing features. It can be implemented using layer normalization technology, aiming to stabilize the feature distribution state and avoid gradient fluctuations during training. The selective state-space mechanism is a mechanism that can dynamically adjust input dependencies. It can be implemented using parametric selective modeling, aiming to accurately extract key local dependencies.

[0103] Specifically, the proposed solution first performs layer normalization on the output features of the previous layer to stabilize the feature distribution. Then, the temporal Mamba module scans along the sequence dimension and uses a selective state-space mechanism to capture local dependencies in the temporal dimension, generating temporal processing features. Based on this, the spatial Mamba module performs layer normalization on the temporal processing features and scans along the feature dimension, focusing on modeling the spatial relationships between electrode channels. Finally, residual connections are used to fuse the spatial processing results with the temporal processing features, ensuring that the output features retain both temporal dynamics and enhanced spatial expressiveness. This step-by-step, decoupled spatiotemporal feature extraction process allows low-level feature processing to be specifically adapted to the spatiotemporal dynamics of EEG signals. While avoiding the high computational overhead of self-attention mechanisms, it effectively improves the accuracy of capturing local dependencies, thus providing high-quality feature input for subsequent processing in the dual-path hierarchical hybrid architecture.

[0104] As a specific implementation, in the lower layers of the network (front) M / (2 layers) The main path uses dual Mamba modules for efficient feature extraction, and its specific implementation is as follows:

[0105] The output features of the previous layer H l−1 Perform layer normalization, then input the time-series Mamba module along the sequence dimension. L A scan is performed, and local dependencies in the time series are captured through a selective state-space mechanism. The state update satisfies the formula:

[0106]

[0107] in, ∈ R DIndicates time step The hidden state at that time It is the state space dimension that controls the memory capacity of the model; Through learnable time step parameters and The state transition matrix and input matrix obtained by discretization; ∈ R D Indicates time step Input features at that time ∈ R D This is the corresponding output; It is the output projection matrix. It is the residual connection matrix; after layer normalization and residual connection, the time-processed features are obtained. :

[0108]

[0109] Features Perform layer normalization, and then pass it along the feature dimension using the spatial Mamba module. D Perform a scan and, with After performing residual connections, the output of the current layer of the main path is obtained. :

[0110]

[0111] Through the above scheme, this application effectively solves the problem of balancing computational efficiency and feature quality in shallow feature extraction. While efficiently extracting key temporal dynamic features in a low signal-to-noise ratio environment, it enhances the modeling ability of EEG spatial distribution patterns, avoids redundant computation in shallow feature extraction, and significantly improves the support ability of low-level features for subsequent high-level semantic modeling.

[0112] This application further proposes, please refer to Figure 3The hybrid attention module includes parallel local attention branches and global attention branches. In the high-level feature processing submodule, the main path uses a combination of the temporal Mamba module and the hybrid attention module for feature extraction. This involves: normalizing the output features of the previous layer using the temporal Mamba module and then scanning along the sequence dimension to perform temporal feature processing, resulting in temporal processed features; inputting these temporal processed features into the hybrid attention module, where the local attention branch restricts the attention calculation range to the local neighborhood based on a window mask, resulting in local attention features; enhancing long-range dependency modeling capabilities through the global attention branch using relative position bias and temperature adjustment parameters, resulting in global attention features; concatenating the local and global attention features and fusing them through a linear layer to obtain hybrid attention features, then performing residual connections with the temporal processed features and applying layer normalization to obtain the output features of the main path at the current high level.

[0113] Among them, the hybrid attention module refers to a structure that processes local and global attention in parallel. It can be implemented using convolutional neural networks or Transformer variants, aiming to simultaneously process local fine-grained patterns and global contextual relationships. Window mask is a mechanism that limits the scope of attention computation. It can be implemented using a sliding window with a fixed window size, aiming to focus on short-range interactions between adjacent electrode channels. Relative position bias is a parameter that encodes positional relationships. It can be predefined based on the electrode space topology, aiming to enhance the model's ability to perceive positional information. Temperature adjustment parameter can be understood as a coefficient that dynamically adjusts the attention distribution. It can be implemented using learnable parameters, aiming to optimize the accuracy of long-range dependency modeling. Linear layer fusion is a feature integration method that can be implemented using fully connected layers, aiming to generate semantically richer feature representations. Residual connection is an operation that fuses processed features with original features. It can be implemented by feature addition, aiming to retain key information of the original features and prevent feature degradation. Layer normalization is a feature standardization operation that is implemented using LayerNorm, aiming to stabilize the feature distribution layer, thereby accelerating model training and mitigating gradient vanishing.

[0114] Specifically, the proposed scheme first performs layer normalization on the output features of the previous layer using a temporal Mamba module to stabilize the feature distribution and accelerate model convergence. Then, it scans along the sequence dimension to process temporal features, efficiently capturing key temporal dependencies. The obtained temporal-processed features are input into a hybrid attention module. The local attention branch uses window masks to limit the computational range to extract local spatiotemporal patterns, while the global attention branch enhances long-range dependency modeling capabilities by adjusting parameters through relative position bias and temperature. The local and global attention features are then concatenated and fused through a linear layer to integrate complementary information. A residual connection is then established with the original temporal-processed features to retain key motion imagery signals. Finally, layer normalization is applied to output the final features. This design achieves parallelization of local and global feature extraction, avoiding the limitations of a single path, while ensuring feature integrity through residual connections and normalization, effectively balancing computational efficiency and feature expressive power.

[0115] As one specific implementation method, this application is implemented at a higher level (later) of the network. M (Layer 2), the main path uses a combination of time-series Mamba modules and hybrid attention modules, and its specific implementation is as follows:

[0116] The time-series Mamba module is used to perform time feature processing to obtain features. ;

[0117] Will Input a hybrid attention module, which contains parallel local attention branches and global attention branches;

[0118] The local attention is achieved through a window mask. When the attention scope is restricted to a local neighborhood, its calculation satisfies the formula:

[0119]

[0120] Step 3.5.4: The global attention is achieved by introducing a relative positional bias. and temperature adjustment parameters To enhance long-range dependency modeling, its calculation satisfies the formula:

[0121]

[0122] Step 3.5.5: Output local attention With global attention output The data is concatenated and then fused through a linear layer to obtain the output of the hybrid attention module. :

[0123]

[0124] in It is a learnable linear fusion matrix. Its corresponding bias vector;

[0125] Step 3.5.6: Convert the output of the hybrid attention module With the input of this module The output of the high-level main path is obtained by merging residual connections and applying layer normalization. :

[0126]

[0127] By employing the above approach, this application reduces the computational complexity of the attention mechanism, improves the completeness of feature representation, and effectively solves the bottleneck problem in high-level semantic feature extraction of EEG signals.

[0128] Please refer to Figure 3 This application further proposes a multi-head self-attention mechanism, including a spatial multi-head self-attention module and a temporal multi-head self-attention module;

[0129] The auxiliary path uses a multi-head self-attention mechanism for feature extraction, and the steps include:

[0130] Perform layer normalization and linear transformation on the output features of the previous layer to obtain the query matrix, key matrix and value matrix;

[0131] The query matrix, key matrix, and value matrix are input into the spatial multi-head self-attention module, processed along the feature dimension, and then residually connected with the output features of the previous layer to obtain the spatial processed features.

[0132] After the spatial processing features are input into the temporal multi-head self-attention module and processed along the sequence dimension, a residual connection is made with the spatial processing features to obtain the output features of the auxiliary path in the current layer.

[0133] Among them, the spatial multi-head self-attention module refers to a module that specifically performs attention calculations on the feature dimension. It can be implemented using attention mechanisms with different numbers of heads, such as setting a 4-head or 8-head attention structure. Its purpose is to focus on the local spatial association patterns between electrodes in different brain regions, avoiding interference from temporal dynamic changes on spatial feature extraction. The temporal multi-head self-attention module refers to a module that specifically performs attention calculations on the sequence dimension. It can be implemented using a sliding window or segmented processing mechanism. Its purpose is to accurately model the temporal evolution of EEG signals and prevent spatial features from contaminating temporal dependencies. Layer normalization refers to the operation of standardizing the mean and variance of feature vectors. It can be implemented using learnable scaling and translation parameters. Its purpose is to stabilize the feature distribution in deep networks and avoid internal covariate shift problems. Residual connection refers to the operation of adding the original input features with the processed features. It can be implemented using identity mapping or weighted summation. Its purpose is to preserve low-level information and alleviate the gradient vanishing problem.

[0134] Specifically, the proposed scheme first stabilizes the distribution of the output features from the previous layer through layer normalization. Then, a linear transformation maps the features to a representation space suitable for attention computation, generating a query matrix, a key matrix, and a value matrix. Based on this, a spatial multi-head self-attention module processes the query matrix, key matrix, and value matrix along the feature dimension, focusing on the spatial topological relationships between electrode channels. Its output is fused with the original features through residual connections to preserve spatial structural information. Subsequently, the spatially processed features are processed along the sequence dimension by an input temporal multi-head self-attention module, focusing on modeling temporal dynamics. Its output is then fused with the spatially processed features through residual connections, achieving a progressive integration of spatial and temporal features. This step-by-step processing mechanism ensures that the spatial feature extraction process is not disturbed by changes in the temporal dimension. Simultaneously, temporal feature extraction is built upon optimized spatial features, effectively avoiding dimensional confusion and maintaining the integrity of feature transfer through residual connections.

[0135] As a specific implementation method, the auxiliary path in this application uses a multi-head self-attention mechanism for feature extraction as follows:

[0136] The auxiliary path employs multi-head self-attention at all levels, including one spatial MHSA layer and one temporal MHSA layer, but the processing order is spatial processing first, followed by temporal processing; the output features of the previous layer are... H l−1 The input space MHSA module, after layer normalization and linear transformation, yields the query (Q), key (K), and value (V) matrices. This module proceeds along the feature dimension. D The attention mechanism is calculated according to the following formula:

[0137]

[0138] in, The dimension of the key vector; after layer normalization and residual connections, the spatially processed intermediate features are obtained. :

[0139]

[0140] Step 3.3: Transfer intermediate features Input is fed into the time-series MHSA module, which operates along the sequence dimension. L The process involves layer normalization and residual connections to obtain the output features of the auxiliary path. :

[0141]

[0142] Through the above technical solution, this application effectively separates the spatial electrode distribution characteristics and temporal dynamic characteristics of EEG signals, avoids dimensional confusion in the feature extraction process, reduces computational redundancy caused by dimensional coupling, and improves the modeling accuracy of spatial association patterns and temporal evolution laws of brain regions, thereby enhancing the robustness and feature expression ability of motor imagery EEG signal decoding.

[0143] In some of the embodiments described above in this application, a dual-path hierarchical hybrid architecture is proposed to enhance the robustness of feature extraction. However, during its implementation, the feature contributions of the main path and auxiliary path change dynamically at different stacking levels. The static fusion mechanism cannot adaptively adjust the weights, resulting in insufficient feature fusion and affecting decoding performance.

[0144] Please refer to the following for details. Figure 3 This application further proposes the following steps for an adaptive fusion mechanism:

[0145] Concatenate the output features of the current layer's main path and auxiliary path along the feature dimension;

[0146] The concatenated features are input into a linear layer for processing, and then the dynamic gating weights are calculated using the Softmax function.

[0147] The output features of the two paths are weighted and summed according to the dynamic gating weights to obtain the final output features of the current layer.

[0148] The deep features used for classification are obtained by fusing all feature processing submodules layer by layer.

[0149] Specifically, feature dimension concatenation refers to connecting the output features of the main path and auxiliary path along the feature dimension. This can be achieved using vector concatenation, aiming to preserve the complementary information of the two paths and provide a complete feature context for subsequent weight calculation. Dynamic gating weight calculation involves generating adaptive weights through linear transformation and normalization. This can be achieved using fully connected layers with a Softmax function, allowing the weights to be dynamically adjusted based on the feature distribution of the current layer, ensuring a reasonable allocation of features between the main and auxiliary paths at different levels. In practical applications, weighted summation refers to linearly combining the features of the two paths based on the calculated weight coefficients. This can be achieved by summing after element-wise multiplication, aiming to achieve adaptive fusion at the feature level and avoid suboptimal fusion results caused by fixed weight strategies. Specifically, layer-by-layer fusion refers to repeatedly applying the above fusion mechanism in each feature processing submodule. This can be achieved using cascaded feature extraction, aiming to progressively refine deep features and ultimately form a high-quality feature representation for classification.

[0150] The proposed solution preserves complementary information between two paths by concatenating feature dimensions, providing a complete context for weight calculation. The concatenated features are then input into a linear layer for processing, and a dynamic gating weight is generated using the Softmax function. This allows the weights to adaptively adjust to different feature distributions. Furthermore, the output features of the two paths are weighted and summed based on the dynamic gating weights to achieve feature-level adaptive fusion. Finally, deep features are progressively refined through layer-by-layer fusion of all feature processing submodules. This ensures that the spatial feature extraction capability of the auxiliary path is prioritized at lower levels, while the semantic modeling capability of the main path is emphasized at higher levels, effectively addressing the problem of dynamically changing feature contributions.

[0151] As a specific implementation method, the adaptive fusion mechanism in this application is implemented as follows:

[0152] Output features of the main path and auxiliary path and characteristics The features are concatenated along the feature dimension, and the concatenated features are input into a linear layer. Then, the dynamic gating weights are calculated using the Softmax function. :

[0153]

[0154] Based on the calculated weights and The output features of the main path and auxiliary paths are weighted and summed to obtain the final output of the current layer. ∈ R B×L×D :

[0155]

[0156] in and For learnable parameters, Indicates feature concatenation operation;

[0157] By using the same auxiliary paths and adaptive fusion mechanisms as the lower layers, the output of the higher-level main path is... and the output of auxiliary paths The fusion is performed by calculating dynamic gating weights and then performing a weighted summation; after... L After processing by the stacked modules, the final output is a feature representation used for classification. H L ∈ R B×L×D ;

[0158] Through the above scheme, this application can dynamically adjust the fusion weights of the main path and auxiliary path according to the feature distribution of the current layer, effectively adapting to the changes in feature contributions at different stacking levels, avoiding the problem of insufficient feature fusion caused by static fusion mechanisms, thereby improving the feature robustness of motor imagery EEG signal decoding.

[0159] This application further proposes that the steps for the classifier module to process deep features include:

[0160] Flatten the deep features into a one-dimensional vector;

[0161] Feature mapping is performed through a multi-layer fully connected network, with activation functions and Dropout regularization used in the intermediate layers.

[0162] The predicted probability of the motion image category is output through the Softmax function of the output layer.

[0163] The method also includes a model training process, which includes calculating the loss value between the predicted probability and the true label using the cross-entropy loss function, and optimizing the model parameters using the Adam optimizer.

[0164] Among these, deep feature flattening refers to the standardization process of converting multidimensional feature tensors into one-dimensional vectors, which can be achieved using tensor reshaping operations. Its purpose is to adapt feature data to the fixed-dimensional input requirements of fully connected networks. Multilayer fully connected networks refer to feature mapping structures composed of multiple stacked fully connected layers, which can be implemented using cascaded two or three fully connected layers. Their purpose is to gradually extract discriminative features through hierarchical nonlinear transformations. Activation functions are mathematical functions that introduce nonlinear transformation capabilities, aiming to enhance the model's ability to capture complex correlations in deep features. Dropout regularization is a regularization technique that randomly deactivates some neuron connections during the training phase. It can be implemented using fixed-dimensional vectors. The random masking mechanism with a fixed dropout rate is used to reduce feature co-fitness and suppress overfitting. The Softmax function is a normalization function that transforms feature vectors into probability distributions. It can be implemented using exponential normalization and aims to output standardized class prediction results that conform to probabilistic interpretation. The cross-entropy loss function is a loss measure that quantifies the difference between the predicted distribution and the true distribution. It can be implemented using the cross-entropy formula for multi-class classification and aims to accurately guide the direction of model optimization. The Adam optimizer is an optimization algorithm that integrates adaptive learning rate and momentum mechanism. It can be implemented using gradient first moment and second moment estimation and aims to improve training convergence efficiency and avoid local optima.

[0165] Specifically, the proposed scheme adapts multidimensional features to the input dimension requirements of a fully connected network through deep feature flattening, ensuring consistency in feature data format. During feature mapping in the multi-layer fully connected network, the activation function of the intermediate layer introduces nonlinear transformation capabilities to analyze complex patterns in deep features. Meanwhile, Dropout regularization reduces feature co-adaptability by randomly discarding neuron connections. The synergistic effect of these two methods effectively suppresses the risk of overfitting in high-noise environments. The Softmax function of the output layer converts the mapping result into a standardized probability distribution, ensuring the interpretability of the prediction results. In the training mechanism, the cross-entropy loss function directly quantifies the difference between the predicted probability and the true label distribution, providing accurate gradient signals for parameter optimization. The Adam optimizer, combined with adaptive learning rate adjustment and momentum accumulation, dynamically optimizes the parameter update step size, thereby accelerating the convergence process and avoiding local optima traps. Overall, this forms a closed-loop enhancement mechanism for feature processing and training optimization.

[0166] As a specific implementation method, this application inputs the output features HL of the dual-path hierarchical hybrid module into the classifier module to obtain the predicted probability of the motion imagery category. The specific implementation is as follows:

[0167] Features H L Flatten it to make it a one-dimensional vector;

[0168] The flattened vector is input into a multilayer perceptron consisting of three fully connected layers. The intermediate layers of the network use the ELU activation function and Dropout is used for regularization to prevent overfitting.

[0169] In the last layer of the network, the Softmax function is used to output the corresponding... F Predicted probability of each category of motion imagination ;

[0170] The model is trained using the cross-entropy loss function and the Adam optimizer, and its calculation satisfies the following formula:

[0171]

[0172] in N For batch size, C For the number of categories, One-hot encoding of the real label. To predict probabilities.

[0173] The above technical solutions effectively suppress overfitting of the model in high-noise EEG signal environments and significantly improve the generalization ability to complex spatiotemporal features. At the same time, they optimize the convergence stability of the training process and avoid local optima, thereby ensuring the accuracy and reliability of the prediction results for motor imagery categories.

[0174] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0175] Corresponding to the motor imagery EEG signal decoding method based on a dual-path hierarchical hybrid architecture described in the above embodiments, this application also provides a motor imagery EEG signal decoding system based on a dual-path hierarchical hybrid architecture, wherein each module of the system corresponds to each step of the above method. Figure 2 The diagram shows the structure of the motor imagery EEG signal decoding system based on a dual-path hierarchical hybrid architecture, as presented in this application.

[0176] Please refer to Figure 4 The system includes a signal input module 100, a convolution embedding module 200, a dual-path hierarchical hybrid processing module 300, and a classification module 400.

[0177] The signal input module 100 refers to the component used to acquire raw multi-channel EEG signals. Specifically, it can be implemented by using the data interface of the EEG acquisition device or by reading data from a standard EEG database. For example, it can receive raw signals from the electrode cap through an amplifier and an analog-to-digital converter, or load preprocessed data from public datasets such as BCICompetition.

[0178] The convolutional embedding module 200 is a component used to perform preliminary spatiotemporal feature extraction on the raw EEG signal. Specifically, it can be implemented using a neural network structure that cascades temporal and spatial convolutions. For example, a one-dimensional convolutional layer is used to process temporal dynamics, and a two-dimensional convolutional layer is used to integrate spatial topology. Batch normalization and pooling layers are then used to complete the embedding.

[0179] The dual-path hierarchical hybrid processing module 300 refers to the core component that performs hierarchical feature extraction and fusion through parallel main paths and auxiliary paths. Specifically, it can be implemented by stacking feature processing sub-modules. The main path uses dual Mamba modules at the lower level and a strategy combining temporal Mamba and hybrid attention at the higher level. The auxiliary path uses a multi-head self-attention mechanism throughout and dynamically integrates the outputs of the two paths through an adaptive fusion mechanism.

[0180] The classification module 400 refers to the component used to map deep features to motion image category probabilities. Specifically, it can be implemented using a multi-layer perceptron structure containing fully connected layers, Dropout layers, and Softmax output layers. Its function is to transform the high-dimensional feature space into a category probability space.

[0181] Specifically, the signal input module 100 first acquires multi-channel motor imagery EEG signals in batch format. Then, the convolutional embedding module 200 preprocesses the signal, extracting the temporal dynamic features of each electrode channel through a temporal convolutional layer, and then integrating the spatial correlations of all electrode channels through a spatial convolutional layer. Finally, it obtains the embedded features through batch normalization, activation functions, and downsampling operations. Next, the dual-path hierarchical hybrid processing module 300 receives the embedded features. Its stacked feature processing sub-modules perform hierarchical feature extraction through parallel main and auxiliary paths: in the lower-level sub-modules, the main path uses dual Mamba modules to efficiently capture basic spatiotemporal features; in the higher-level sub-modules, the main path combines temporal Mamba with a hybrid attention module to enhance multi-scale long-range dependency modeling; the auxiliary path uses spatial and temporal multi-head self-attention mechanisms at all levels to provide complementary features; each sub-module dynamically weights and fuses the dual-path outputs through an adaptive fusion mechanism. Finally, the classification module 400 flattens the final deep features and outputs the predicted probability of the motor imagery category through a multi-layer fully connected network and a Softmax function.

[0182] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0183] This application also provides a computer device, such as... Figure 5 As shown, the computer device 5 in this embodiment includes: at least one processor 50, at least one memory 51, and a computer program 52 stored in at least one memory 51 and executable on at least one processor 50. When the processor 50 executes the computer program 52, it enables the computer device 5 to implement the steps in the above-described arbitrary motion imagery EEG signal decoding method embodiment, or enables the computer device 5 to implement the functions of each module / unit in the above-described system embodiment.

[0184] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for decoding motor imagery EEG signals based on a dual-path hierarchical hybrid architecture, characterized in that, Includes the following steps: Acquire multi-channel motor imagery EEG signals; The multi-channel motion imagery EEG signal is input into the convolutional embedding module to extract preliminary spatiotemporal features and obtain the embedding features; The embedded features are input into a dual-path hierarchical hybrid processing module consisting of multiple stacked feature processing sub-modules. The main path and auxiliary path set in parallel by the dual-path hierarchical hybrid processing module are used to extract features from the embedded features layer by layer, and different feature extraction strategies are adopted according to the stacking level. The dual-path hierarchical hybrid processing module includes M stacked feature processing sub-modules, where M is a positive even number. The first M / 2 feature processing sub-modules are defined as low-level feature processing sub-modules, and the last M / 2 feature processing sub-modules are defined as high-level feature processing sub-modules. The different feature extraction strategies adopted according to the stacking level specifically include: The main path in the low-level feature processing submodule uses a spatiotemporal dual Mamba module for feature extraction. The spatiotemporal dual Mamba module includes a temporal Mamba module and a spatial Mamba module. The feature extraction steps include: The temporal Mamba module performs layer normalization on the output features of the previous layer and then scans along the sequence dimension. Local dependencies are captured through a selective state space mechanism to obtain temporally processed features. The spatial Mamba module performs layer normalization on the temporal processing features and then scans along the feature dimension to obtain the output features of the main path at the current lower layer. The main path in the high-level feature processing submodule uses a combination of a temporal Mamba module and a hybrid attention module for feature extraction. The hybrid attention module includes parallel local attention branches and global attention branches, and its feature extraction steps include: The temporal Mamba module performs layer normalization on the output features of the previous layer and then scans along the sequence dimension to process the temporal features, thus obtaining the temporal processed features. The time-series processing features are input into the hybrid attention module, and the attention calculation range is limited to the local neighborhood by the local attention branch based on the window mask to obtain the local attention features. The global attention branch enhances the long-range dependency modeling capability based on relative position bias and temperature adjustment parameters, thus obtaining global attention features; The local attention features and the global attention features are concatenated and fused through a linear layer to obtain hybrid attention features. After residual connection with the temporal processing features, layer normalization is applied to obtain the output features of the main path at the current high layer. The auxiliary path employs a multi-head self-attention mechanism for feature extraction in all feature processing submodules; The output features of the main path and the auxiliary path are fused through an adaptive fusion mechanism to obtain deep features for classification. The deep features are input into the classifier module, which outputs the predicted probability of the motion image category.

2. The method according to claim 1, characterized in that, The steps for the convolutional embedding module to extract preliminary spatiotemporal features include: The multi-channel motion imagery EEG signal is input into a temporal convolutional layer, and convolution operation is performed on each electrode channel along the time dimension. The features obtained after temporal convolution are input into the spatial convolutional layer, and convolution operation is performed covering all electrode channels. The features after spatial convolution are sequentially subjected to batch normalization and activation function processing. The processed features are downsampled in the time dimension and adjusted in the feature dimension to obtain the embedded features.

3. The method according to claim 1, characterized in that, The multi-head self-attention mechanism includes a spatial multi-head self-attention module and a temporal multi-head self-attention module; The auxiliary path uses a multi-head self-attention mechanism for feature extraction, which includes: performing layer normalization and linear transformation on the output features of the previous layer to obtain the query matrix, key matrix and value matrix; The query matrix, key matrix, and value matrix are input into the spatial multi-head self-attention module and processed along the feature dimension. Then, a residual connection is performed with the output feature of the previous layer to obtain the spatial processed feature. After the spatial processing features are input into the temporal multi-head self-attention module and processed along the sequence dimension, a residual connection is made with the spatial processing features to obtain the output features of the auxiliary path in the current layer.

4. The method according to claim 1, characterized in that, The adaptive fusion mechanism includes: concatenating the output features of the current layer's main path and auxiliary path along the feature dimension; The concatenated features are input into a linear layer for processing, and then the dynamic gating weights are calculated using the Softmax function. The output features of the two paths are weighted and summed according to the dynamic gating weights to obtain the final output features of the current layer. By fusing all feature processing submodules layer by layer, the deep features used for classification are obtained and then fed into the next level dual-path processing or classifier.

5. The method according to claim 1, characterized in that, The steps for the classifier module to process deep features include: Flatten the depth feature into a one-dimensional vector; Feature mapping is performed through a multi-layer fully connected network, with activation functions and Dropout regularization used in the intermediate layers. The predicted probability of the motion imagination category is output through the Softmax function of the output layer; The method also includes a model training process, which includes: calculating the loss value between the predicted probability and the true label using the cross-entropy loss function, and optimizing the model parameters using the Adam optimizer.

6. A motor imagery EEG signal decoding system based on a dual-path hierarchical hybrid architecture, characterized in that, include: The signal input module is used to acquire multi-channel motor imagery EEG signals; The convolutional embedding module is used to process the EEG signal and extract preliminary spatiotemporal features to obtain embedding features; The dual-path hierarchical hybrid processing module is used to extract features from the embedded features layer by layer through the parallel main path and auxiliary path, and adopt different feature extraction strategies according to the stacking level. The dual-path output features are fused through the adaptive fusion module to obtain deep features. The dual-path hierarchical hybrid processing module includes M stacked feature processing sub-modules, where M is a positive even number. The first M / 2 feature processing sub-modules are defined as low-level feature processing sub-modules, and the last M / 2 feature processing sub-modules are defined as high-level feature processing sub-modules. The different feature extraction strategies adopted according to the stacking level specifically include: The main path in the low-level feature processing submodule uses a spatiotemporal dual Mamba module for feature extraction. The spatiotemporal dual Mamba module includes a temporal Mamba module and a spatial Mamba module. The feature extraction steps include: The temporal Mamba module performs layer normalization on the output features of the previous layer and then scans along the sequence dimension. Local dependencies are captured through a selective state space mechanism to obtain temporally processed features. The spatial Mamba module performs layer normalization on the temporal processing features and then scans along the feature dimension to obtain the output features of the main path at the current lower layer. The main path in the high-level feature processing submodule uses a combination of a temporal Mamba module and a hybrid attention module for feature extraction. The hybrid attention module includes parallel local attention branches and global attention branches, and its feature extraction steps include: The temporal Mamba module performs layer normalization on the output features of the previous layer and then scans along the sequence dimension to process the temporal features, thus obtaining the temporal processed features. The time-series processing features are input into the hybrid attention module, and the attention calculation range is limited to the local neighborhood by the local attention branch based on the window mask to obtain the local attention features. The global attention branch enhances the long-range dependency modeling capability based on relative position bias and temperature adjustment parameters, thus obtaining global attention features; The local attention features and the global attention features are concatenated and fused through a linear layer to obtain hybrid attention features. After residual connection with the temporal processing features, layer normalization is applied to obtain the output features of the main path at the current high layer. The auxiliary path employs a multi-head self-attention mechanism for feature extraction in all feature processing submodules; The classification module is used to process the depth features and output the predicted probability of the motion image category.

7. An electronic device comprising a processor and a memory, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 5.

Images