A method for classifying motor imagery tasks based on brain region-enhanced attention convolutional networks.

By using brain region-enhanced attention convolutional networks to process EEG data, the problem of neglecting brain region relationships in existing technologies is solved, the classification accuracy of motor imagery tasks is improved, and the attention to features inside and outside brain regions is enhanced, resulting in more efficient feature representation and data understanding.

CN117725464BActive Publication Date: 2026-11-10CHONGQING UNIV OF POSTS & TELECOMM
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311705865.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-12
Publication Date
2026-11-10
Estimated Expiration
2043-12-12

AI Technical Summary

Technical Problem

Existing technologies ignore the relationships between different brain regions when processing motor imagery EEG signals, resulting in low classification accuracy and an inability to effectively utilize the spatial and temporal features in EEG data.

Method used

We employ a brain region-based enhanced attention convolutional network approach. This approach uses a brain region-based enhanced attention module, a deep multi-scale temporal convolution module, a spatial convolution module, and a global and local information fusion module to process EEG data, generating richer feature representations and improving the model's attention to the correlations between different brain regions and between electrodes within brain regions.

Benefits of technology

It improves the classification accuracy of motor imagery tasks, better captures the collaborative work between different brain regions, enhances the understanding of complex neural activities, and improves the model's feature discrimination and data representation capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117725464B_ABST
    Figure CN117725464B_ABST
Patent Text Reader

Abstract

This invention belongs to the biomedical field, specifically relating to a method for classifying motor imagery tasks based on a brain region-enhanced attention convolutional network. The method includes: acquiring motor imagery EEG data; preprocessing the acquired EEG data; inputting the preprocessed EEG data into a trained brain region-enhanced attention convolutional network to obtain the final classification result. The brain region-enhanced attention convolutional network includes: a brain region-enhanced attention module, a deep multi-scale temporal convolution module, a spatial convolution module, a global and local information fusion module, and a classification module. This invention employs a brain region-enhanced attention convolutional network, which can better focus on the correlation between different brain regions and between electrodes within brain regions, allowing the model to focus more on the key information of each region, improving the discriminative power of features, and providing a new approach for the development of brain-computer interface technology.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of biomedicine, specifically relating to a method for classifying motor imagery tasks based on brain region-enhanced attention convolutional networks. Background Technology

[0002] Brain-computer interfaces (BCIs) are systems that convert acquired neurophysiological signals into computer commands to control output devices. BCI systems enable direct communication between the brain and the outside world, without relying on muscles and nerve tissue outside the brain, providing a new possibility for communication for people who have lost motor abilities. Among BCI systems, electroencephalography (EEG) signals have become the most widely used EEG signal acquisition technology due to their non-invasiveness, convenience, low cost, and high temporal resolution. Unlike other paradigms, motor imagery (MI) is an endogenous BCI system; when a user imagines a bodily movement, signals are generated in the sensorimotor cortex. The power of the rhythm will change. People can distinguish different body movements by decoding these changes and convert the pattern recognition results into corresponding instructions to control external devices. Therefore, the MI-BCI system is widely used in rehabilitation medicine applications, such as the neurocontrol of prosthetics.

[0003] Before applying deep learning methods to motor intention EEG classification tasks, proper processing of EEG data is crucial. EEG data reflects the brain's electrical signal activity, and the MI task aims to analyze signals from brain regions related to movement, with different regions corresponding to different functions. However, due to the volumetric conduction effect during neuronal signal transmission, EEG signals recorded by electrodes may contain mixed activity. Furthermore, the interaction between different electrodes can lead to acquired EEG signals not only corresponding to the electrical signals of functional brain regions. To address these issues, methods such as short-time Fourier transform and wavelet transform are used to convert EEG time series into two-dimensional images; to account for the spatial characteristics of multi-channel signals, EEG activity is transformed into a series of topology-preserving multispectral images.

[0004] In situations involving the coordinated work of multiple brain regions, the relationships between different brain regions are crucial for EEG analysis. However, the aforementioned existing techniques neglect the relationships between different regions of the EEG and have limitations in considering the correlations between different brain regions and between electrodes within brain regions, thus limiting the classification accuracy of the model in the MI task. Summary of the Invention

[0005] To address the problems of the prior art, this invention employs a motor imagery task classification method based on a brain region-enhanced attention convolutional network, comprising:

[0006] Motor imagery EEG data is acquired and preprocessed. The preprocessed EEG data is then input into a trained brain region enhanced attention convolutional network to obtain the final classification result. The brain region enhanced attention convolutional network includes: a brain region enhanced attention module, a deep multi-scale temporal domain convolution module, a spatial domain convolution module, a global and local information fusion module, and a classification module.

[0007] The training process of a brain region-based attention-enhancing convolutional network includes:

[0008] S1. Obtain the EEG dataset of motor imagery, preprocess the data in the dataset, and divide the preprocessed data into training set and test set;

[0009] S2. Input the data from the training set into the brain region enhanced attention module to obtain enhanced features;

[0010] S3. Input the enhanced features into the deep multi-scale temporal convolution module to obtain temporal features at different scales;

[0011] S4. Input the temporal features at different scales into the spatial convolution module to obtain compressed features;

[0012] S5. Input the compressed features into the global and local information fusion module to obtain the fused features;

[0013] S6. Input the fused features into the classification module to obtain the classification result;

[0014] S7. Calculate the loss function value based on the classification result, update the model parameters based on the loss function value, and obtain the trained network when the loss function value converges.

[0015] Preprocessing the data in the dataset includes:

[0016] The motor imagery EEG data in the dataset was downsampled to 250Hz, and the EEG data of [0,4]s was extracted from the downsampled EEG data to obtain the preprocessed dataset.

[0017] The data from the training set is input into the brain region-enhancing attention module, which includes:

[0018] S21. Perform dimensional transformation on the EEG signal;

[0019] S22. Divide the dimensionally transformed EEG signal into multiple partitions along the channel dimension. Where G is the number of partitions;

[0020] S23, to Perform global average pooling to obtain Global features;

[0021] S24. Use Global feature generation Importance coefficient;

[0022] S25, to The importance coefficients were normalized to obtain The normalized value;

[0023] S26, to Scale and translate the normalized values ​​to obtain The final coefficient;

[0024] S27, according to The final coefficients are obtained by using the sigmoid function. Scaling is performed to obtain Enhanced features.

[0025] The regions include: the central region, the frontal lobe region, and the parietal lobe region.

[0026] right Importance coefficient normalization includes:

[0027]

[0028] in, Normalized value For the i-th importance coefficient of the j-th electrode position in the k-th partition, The sum of the importance coefficients of the j-th electrode position in the k-th partition. Let be the mean square error of the j-th electrode position in the k-th partition. This is a constant that is set.

[0029] The deep multi-scale temporal convolution module includes multiple branches, each branch containing multiple convolutional layers. The kernel size of each branch is the same, but the number of convolutional layers is different.

[0030] Inputting temporal features at different scales into the spatial convolution module includes:

[0031] S41. Use depthwise convolution to process temporal features at different scales to obtain convolution results;

[0032] S42. Process the convolution result using a squared nonlinear function to obtain the output result;

[0033] S43. Perform average pooling on the output results to obtain pooling features;

[0034] S44. The pooling features are processed using a logarithmic nonlinear function to obtain compressed features.

[0035] The compressed feature input to the global and local information fusion module includes:

[0036] S51. Perform global average pooling on the compressed features to obtain the global feature compression vector;

[0037] S52. Use the Sigmoid activation function to process the global feature compression vector to obtain the weight vector;

[0038] S53. Multiply the weight vector with the compressed feature to obtain the fused feature.

[0039] The loss function is the cross-entropy loss function.

[0040] Beneficial effects:

[0041] 1. This invention employs a brain region-based attention enhancement module. This module generates an attention factor for each spatial location within each brain region to adjust the importance of each feature, enabling each brain region to autonomously enhance its learning representation and suppress potential noise. By utilizing the overall information of a specific brain region to strengthen the learning of local features, it can better focus on the correlations between different brain regions and between electrodes within brain regions, allowing the model to focus more on the key information of each region and improve feature discriminativeness. 2. In terms of region division, this invention divides EEG data into the central, frontal, and parietal regions. This division not only preserves the temporal characteristics of EEG signals but also, to some extent, the spatial characteristics of brain regions, helping to capture the collaborative work between different brain regions and improving the understanding of complex neural activities. 3. This invention uses a deep multi-scale temporal feature extraction module to enable the network to gradually build higher-level abstract representations and extract more abstract features from input signals. 4. This invention generates richer and more complex feature expressions by fusing global and local information, allowing the model to consider both overall context and local details simultaneously, thereby improving its ability to represent input data and better capturing abstract features in the data. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the brain partitioning structure of the present invention;

[0043] Figure 2 This is a flowchart illustrating the training process of the brain region-enhanced attention convolutional network of the present invention.

[0044] Figure 3 This invention illustrates the impact of the number of branches in the deep multi-scale temporal convolution module on accuracy.

[0045] Figure 4 This is a schematic diagram of the brain region-based attention enhancement module of the present invention;

[0046] Figure 5 This is a schematic diagram of the deep multi-scale temporal convolution module of the present invention;

[0047] Figure 6 This is a schematic diagram of the spatial convolution module of the present invention;

[0048] Figure 7 Schematic diagram of the global and local information fusion module of this invention;

[0049] Figure 8 This is a schematic diagram of the classification module of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] This invention employs a motor imagery task classification method based on a brain region-enhanced attention convolutional network. The brain region-enhanced attention convolutional network includes: a brain region-enhanced attention module, a deep multi-scale temporal convolutional module, a spatial convolutional module, a global and local information fusion module, and a classification module.

[0052] Brain Region Enhanced Attention (BREA) adjusts the importance of each feature by generating an attention factor for each spatial location within each brain region, allowing each region to autonomously enhance its learned representation and suppress potential noise. The attention factor is guided solely by the similarity between global and local features within each group, making the BREA module very lightweight with almost no additional parameters or computational cost. A deep multi-scale temporal convolution module is used to extract temporal features at different levels from the enhanced EEG signal. A spatial convolution module is used to extract the corresponding spatial features from the temporally filtered signal. A global and local feature fusion module is employed to extract the most discriminative spatiotemporal features. Finally, the features are fed into a classification module for classification.

[0053] Preprocessing of motor imagery EEG data included: to ensure consistency of the study, all EEG data were downsampled to 250Hz, and motor imagery data in the range of [0,4]s were selected;

[0054] To comprehensively evaluate the classification performance of the decoding algorithm used in this invention, two different evaluation strategies were employed in this embodiment: cross-validation and hold-out. This invention selected two widely used public datasets in brain-computer interface research: the BCI-IV-2A dataset and the High Gamma dataset. The BCI-IV-2A dataset was acquired by 22 electrodes at a sampling rate of 250Hz, including four categories of motor imagery signals (left hand, right hand, foot, and tongue) from nine subjects. The High Gamma dataset contains four categories of motor imagery EEG signals (left hand, right hand, foot, and resting) from 14 healthy subjects, acquired by 128 electrodes at a sampling rate of 500Hz.

[0055] In the cross-validation analysis, for the BCI-IV-2A dataset, 288 trials collected from the first phase of subjects were selected as training and testing data. For the High Gamma dataset, approximately 880 trials performed by subjects in the first 11 sessions were selected for model training and testing. This invention employs a 10-fold cross-validation method for experimental evaluation. In this process, all data are equally divided into 10 mutually exclusive subsets. Each time, one subset is selected as the test set, and the remaining nine subsets are used as the training set. This process is repeated 10 times to ensure that each subset is used as the test set.

[0056] The hold-out method is a simple and direct data partitioning approach. For the BCI-IV-2A dataset, the 288 trials collected in the first phase by the subjects were used as training data, and the 288 trials collected in the second phase were used as test data. For the High Gamma dataset, following the dataset partitioning method in existing research, approximately 880 trials performed by the subjects in the first 11 sessions were used as training data, and approximately 160 trials performed by the subjects in the last two sessions were used as test data. Each subject was trained individually, and the decoding accuracy of the model on each subject was recorded. Finally, the average decoding accuracy of all subjects was calculated as the final decoding accuracy of this invention.

[0057] like Figure 2 As shown, the training process of the brain region-enhanced attention convolutional network includes:

[0058] S1. Obtain EEG data on motor imagery from different subjects and at different stages of the same subject, preprocess the data, and divide the preprocessed EEG data into training set and test set.

[0059] S2. Enhance attention by dividing brain regions and generating an attention factor for each spatial location of each brain region to adjust the importance of each feature, so that each brain region can autonomously enhance its learning expression and suppress possible noise.

[0060] Different regions of the brain play crucial roles in motor control and coordination. During motor execution, the parietal cortex processes sensory information to provide cognition of the surrounding environment, while the frontal lobe participates in decision-making and motor planning. Simultaneously, the central motor cortex directly controls muscle movement. The coordinated work among these three regions is essential for achieving precise, coordinated, and goal-oriented movement. Figure 1 As shown, in most BCI experiments, the electrodes used to collect EEG signals are placed according to the international 10-20 system. In terms of region division, this invention uses raw EEG data from the central, frontal, and parietal regions. This method not only preserves the temporal characteristics of EEG signals but also preserves the spatial characteristics of brain regions to a certain extent.

[0061] Brain region-based attention enhancement leverages holistic information from specific brain regions to strengthen the learning of local features. This allows models to better focus on correlations between different brain regions and between electrodes within those regions. This helps capture the collaborative work between different brain areas, improving our understanding of complex neural activity. Figure 4 As shown, the detailed implementation steps of this mechanism include:

[0062] S21: Dimensional Transformation: For the input EEG signal , , , These represent the number of channels, the number of electrodes, and the number of sampling points, respectively. Since it's necessary to divide different electrodes into different brain regions along the channel dimension, the EEG signal is first transformed to... .

[0063] S22: Regional Division: [This section appears to be incomplete and requires further context.] Divided into G zones along the channel dimension , , This represents the number of electrodes in each partition, and the feature vector of each partition at each location is... ,in, , .

[0064] S23: Global Average Pooling: Using the global average pooling function The obtained global statistical features are used to approximate the semantic vector representation learned by this group: .

[0065] S24: Generating Importance Coefficients: Utilizing Global Features For each local feature Generate the corresponding importance coefficients It measures the similarity between global features and local features.

[0066] S25: Coefficient Normalization: For Normalization is performed to ensure that the coefficients are unbiased across different samples:

[0067] , ,

[0068] in, The sum of the importance coefficients of the j-th electrode position in the k-th partition. Let be the mean square error of the j-th electrode position in the k-th partition. A constant set to ensure numerical stability.

[0069] S26: Scaling and Translation: Introducing a pair of parameters , The normalized values ​​are scaled and shifted to obtain the final coefficients:

[0070]

[0071] S27: Sigmoid scaling: sigmoid function Spatially, the primitive Scaling is performed to obtain the enhanced features:

[0072]

[0073] S3: The results obtained in S2 are used to extract multi-scale temporal features through a deep multi-scale temporal convolution module. The deep multi-scale temporal convolution module not only helps to extract richer information, but also improves the performance when dealing with more complex tasks by increasing the network depth.

[0074] like Figure 3 As shown, by comparing deep multi-scale temporal convolution modules using different numbers of branches, three branches were ultimately selected because they showed the most significant performance in decoding.

[0075] like Figure 5 As shown, the deep multi-scale temporal convolution module includes three branches, each consisting of a convolutional kernel of the same size. It consists of convolutional layers of different depths. The receptive field of the nth layer is:

[0076]

[0077] in, It is the first The size of the convolution kernel, It is the first Layer step size; through stacking Each has a convolution kernel size ( The convolutional layer ultimately outputs a feature map with a receptive field size of (default). For 1):

[0078]

[0079] This structure allows each branch to capture features with different receptive fields, enabling the extraction of temporal features at multiple scales. Furthermore, the varying depths of each branch ensure that the learned features contain information at different levels. Additionally, by using depthwise convolutions instead of standard convolutions, a one-to-one correspondence exists between the input and output feature maps of each convolutional layer. The localization of the convolutional kernels on the input feature maps helps extract local structural information without being affected by other feature maps, mitigating overfitting and reducing sensitivity to the amount of training data.

[0080] S4: The results obtained in S3 are used by the spatial domain convolution module to learn the corresponding spatial domain filter for the data filtered in a specific frequency band, and the compressed features are obtained.

[0081] like Figure 6 As shown, the spatial convolution module learns spatial filters and further compresses the extracted spatiotemporal features. Specifically, the temporal filtering stage of the EEG signal can be viewed as a frequency band separation layer, obtaining filtered data for different frequency bands. Depth convolution is used to learn the corresponding spatial filters for data filtered in specific frequency bands, where the convolution kernel size is set to (C,1) to span all channels, and the depth parameter D controls the number of spatial filters learned for each temporal filter. Existing research has demonstrated the effectiveness of using square and logarithmic nonlinear functions; therefore, these two functions are introduced into the module to enhance the network's nonlinear expressive power. Furthermore, average pooling layers are used to compress the spatiotemporal features extracted by the depth convolution, which reduces the number of model parameters, lowers information redundancy, and further avoids overfitting.

[0082] S5: The results obtained in S4 are used to obtain feature information after being fused into global information through the global and local information fusion module, so that the network can understand the input signal more comprehensively, while focusing on the extraction of key features of the task.

[0083] like Figure 7 As shown, the global and local information fusion module obtains the global feature compression amount through global average pooling:

[0084]

[0085] in, yes The Each feature map By compressing the first Each feature map The real numbers are obtained from the pixel values. The weight vector of global features is obtained by activating the sigmoid function. :

[0086]

[0087] in, This represents the sigmoid function operation. This represents the feature vector obtained through global average pooling. To incorporate local information, the weights of each channel in the output are element-wise multiplied with the input feature map to obtain a weighted feature map representation:

[0088]

[0089] In this approach, global information typically captures the overall context of the input data, while local information focuses on specific details and local patterns within the input data. Combining these two allows the network to gain a more comprehensive understanding of the input signal while simultaneously focusing on extracting key task features.

[0090] like Figure 8 As shown, the classification module of this invention flattens the feature map and inputs it into a fully connected layer, and then obtains the classification result through an activation function.

[0091] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for classifying motor imagery tasks based on brain region-enhanced attention convolutional networks, characterized in that, include: EEG data of motor imagery is acquired and preprocessed. The preprocessed EEG data is then input into a trained brain region-enhanced attention convolutional network to obtain the final classification result. The brain region-enhanced attention convolutional network includes: a brain region-enhanced attention module, a deep multi-scale temporal convolution module, a spatial convolution module, a global and local information fusion module, and a classification module; The training process of a brain region-based attention-enhancing convolutional network includes: S1. Obtain the EEG dataset of motor imagery, preprocess the data in the dataset, and divide the preprocessed data into training set and test set; S2. Input the data from the training set into the brain region enhanced attention module to obtain enhanced features; The data from the training set is input into the brain region-enhancing attention module, which includes: S21. Perform dimensional transformation on the EEG signal; S22. Divide the dimensionally transformed EEG signal into multiple partitions along the channel dimension. Where G is the number of partitions; S23, to Perform global average pooling to obtain Global features; S24. Use Global feature generation Importance coefficient; S25, to The importance coefficients were normalized to obtain The normalized value; S26, to Scale and translate the normalized values ​​to obtain The final coefficient; S27, according to The final coefficients are obtained by using the sigmoid function. Scaling is performed to obtain Enhanced features; S3. Input the enhanced features into the deep multi-scale temporal convolution module to obtain temporal features at different scales; The deep multi-scale temporal convolution module includes multiple branches, each branch includes multiple convolutional layers, and the convolutional kernels of each branch are the same size but the number of convolutional layers is different; S4. Input the temporal features at different scales into the spatial convolution module to obtain compressed features; Inputting temporal features at different scales into the spatial convolution module includes: S41. Use depthwise convolution to process temporal features at different scales to obtain convolution results; S42. Process the convolution result using a squared nonlinear function to obtain the output result; S43. Perform average pooling on the output results to obtain pooling features; S44. Use a logarithmic nonlinear function to process the pooling features to obtain compressed features; S5. Input the compressed features into the global and local information fusion module to obtain the fused features; The compressed feature input to the global and local information fusion module includes: S51. Perform global average pooling on the compressed features to obtain the global feature compression vector; S52. Use the Sigmoid activation function to process the global feature compression vector to obtain the weight vector; S53. Multiply the weight vector with the compressed feature to obtain the fused feature; S6. Input the fused features into the classification module to obtain the classification result; S7. Calculate the loss function value based on the classification result, update the model parameters based on the loss function value, and obtain the trained network when the loss function value converges.

2. The method for classifying motor imagery tasks based on brain region-enhanced attention convolutional networks according to claim 1, characterized in that, Preprocessing the data in the dataset includes: The motor imagery EEG data in the dataset was downsampled to 250Hz, and the EEG data of [0,4]s was extracted from the downsampled EEG data to obtain the preprocessed dataset.

3. The method for classifying motor imagery tasks based on brain region-enhanced attention convolutional networks according to claim 1, characterized in that, The partitions include: Central region, frontal lobe region, and parietal lobe region.

4. The method for classifying motor imagery tasks based on brain region-enhanced attention convolutional networks according to claim 1, characterized in that, right Importance coefficient normalization includes: in, Normalized value For the i-th importance coefficient of the j-th electrode position in the k-th partition, The sum of the importance coefficients of the j-th electrode position in the k-th partition. Let be the mean square error of the j-th electrode position in the k-th partition. This is a constant that is set.

5. The method for classifying motor imagery tasks based on brain region-enhanced attention convolutional networks according to claim 1, characterized in that, The loss function is the cross-entropy loss function.

Citation Information

Patent Citations

  • Motor imagery classification method and system based on dual-scale brain region features

    CN116561654A

  • Improved Method And System For Predicting Outcomes Based On Spatio / Spectro-Temporal Data

    US20160210552A1