An electroencephalogram signal decoding method, a terminal device and a storage medium
By constructing a three-dimensional signal matrix and an EEG signal classification model based on attention mechanisms, the problem of neglecting local correlation of channels in existing technologies is solved, thereby improving the classification accuracy and real-time efficiency of EEG motor imagery.
Patent Information
- Application Number
- CN202310502848.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-06
- Publication Date
- 2026-08-25
- Estimated Expiration
- 2043-05-06
AI Technical Summary
Existing EEG decoding methods based on motor imagery neglect the local correlation between channels in their network design, leading to the loss of spatial features and affecting classification accuracy.
A three-dimensional signal matrix is constructed and an EEG signal classification model is adopted using multiple convolutional modules, spatial attention modules, and channel attention modules. The spatial features of EEG data are learned through local convolution, and important features are enhanced by combining attention mechanisms while preserving the correlation between channels.
It improves the recognition accuracy and universality of EEG motor imagery classification and achieves real-time efficiency of brain-computer interaction.
Smart Images
Figure CN117763323B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of electroencephalogram (EEG) signal analysis, and more particularly to an EEG signal decoding method, terminal device, and storage medium. Background Technology
[0002] Currently, EEG decoding methods based on motor imagery can be divided into two main categories. The first category is based on machine learning. These methods first preprocess the acquired raw data, including filtering, denoising, signal amplification, and resampling. Then, features are extracted from the preprocessed data, typically including time-domain, frequency-domain, and time-frequency-domain features. Next, the most representative and discriminative features are selected from the extracted features to reduce model complexity and improve classification performance. However, this feature selection process relies on prior knowledge, which can lead to the neglect of some important features. The second category, decoding methods based on deep learning, do not require extensive data preprocessing. Their "end-to-end" nature allows them to obtain classification results simply by inputting raw data. However, most current deep learning methods treat channels as a whole in their network design, thus learning global features of the space. In reality, there are local correlations between channels, meaning that there is a certain correlation between the EEG signals of adjacent channels. Considering only global features while ignoring local spatial features leads to the loss of spatial features, thereby affecting classification accuracy. Summary of the Invention
[0003] To address the aforementioned problems, this invention proposes a method for decoding electroencephalogram (EEG) signals, a terminal device, and a storage medium.
[0004] The specific plan is as follows:
[0005] A method for decoding electroencephalogram (EEG) signals includes the following steps:
[0006] S1: Collect EEG signal data from different parts of the human body and construct a three-dimensional signal matrix according to the distribution of the collection electrodes on the scalp corresponding to the EEG signal data. Based on the obtained three-dimensional signal matrix, construct a training set and set the human body movement parts as the classification labels corresponding to the three-dimensional signal matrix.
[0007] S2: Construct an EEG signal classification model and train the model using a training set;
[0008] The model's network structure consists of multiple convolutional modules, a spatial attention module, a channel attention module, and a classification module. After the 3D signal matrix is input into a convolutional module, the outputs of the convolutional modules are input into the spatial attention module and the channel attention module, respectively. The outputs of the spatial attention module and the channel attention module are element-wise added and merged, and then a sigmoid operation is performed. The sigmoid operation result is then element-wise multiplied with the output of the convolutional module, and the result of this multiplication is added to the output of the convolutional module. The sum is then passed through other convolutional modules in sequence, and the output of the last convolutional module is input into the classification module to obtain the classification result.
[0009] S3: After converting the EEG signal data to be classified into a three-dimensional signal matrix, input it into the trained EEG signal classification model to obtain the classification result of the EEG signal data to be classified.
[0010] Furthermore, step S1 also includes filtering the acquired EEG signal data through a finite-length unit impulse response filter to extract EEG signals in the α and β bands.
[0011] Furthermore, step S1 also includes downsampling the EEG signal data.
[0012] Furthermore, the convolution module includes 3D convolutional layers, batch normalized layers, exponential linear units, and average pooling layers.
[0013] Furthermore, the convolutional module comprises four modules.
[0014] Furthermore, the kernel size of the spatial dimension of the convolution module for the input three-dimensional signal matrix is (2,2), and the stride is (1,1); the kernel sizes of the three convolution modules through which the summation result passes are (2,2), (1,2), and (2,2), and the strides are (2,2), (1,2), and (2,2), respectively.
[0015] Furthermore, the classification module consists of a global average pooling layer and a fully connected layer.
[0016] A brainwave signal decoding terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described above in the embodiments of the present invention.
[0017] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above in the embodiments of the present invention.
[0018] The present invention adopts the above technical solution. On the one hand, in terms of function, it retains the correlation between EEG channels, strengthens the important features of the spatial domain and feature map, and weakens unimportant information, which can improve the classification accuracy of EEG motor imagery classification and recognition. On the other hand, in terms of structure, it transforms two-dimensional EEG signal data into a three-dimensional signal matrix and adopts a convolution method of first local and then global to learn the spatial features of EEG data, thus preserving the correlation between channels to the greatest extent. Attached Figure Description
[0019] Figure 1 The diagram shown is a flowchart of Embodiment 1 of the present invention.
[0020] Figure 2 The diagram shown is a schematic of the three-dimensional signal matrix conversion process in this embodiment.
[0021] Figure 3 The diagram shown is a framework diagram of the model in this embodiment. Detailed Implementation
[0022] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention.
[0023] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.
[0024] Example 1:
[0025] This invention provides a method for decoding electroencephalogram (EEG) signals, such as... Figure 1 As shown, the method includes the following steps:
[0026] S1: Collect EEG signal data from different parts of the human body and construct a three-dimensional signal matrix according to the distribution of the collection electrodes on the scalp corresponding to the EEG signal data. Based on the obtained three-dimensional signal matrix, construct a training set and set the human body movement parts as the classification labels corresponding to the three-dimensional signal matrix.
[0027] In this embodiment, the selected human body movement parts are the left and right hands. In other embodiments, other human body movement parts can also be selected, and there is no limitation here.
[0028] Furthermore, this embodiment also includes data filtering of EEG signal data, specifically filtering by a finite-length unit impulse response (FIR) filter to extract EEG signals in the α and β bands corresponding to the region of EEG motor imagery, i.e., EEG signals in the 8-30 rhythm segment.
[0029] In addition, to improve the running speed of the model and reduce memory consumption, the embodiment also includes downsampling the EEG signal data, specifically downsampling the 1000Hz data to 400Hz.
[0030] The data extracted from the 31 acquisition channels of the motion imagination zone in this embodiment are as follows: Figure 2 ( Figure 2 As shown in the dark electrode diagram, when constructing the three-dimensional signal matrix, the arrangement of each element in the matrix is based on the position of the acquisition electrodes on the scalp, and zeros are used to fill the positions where no electrodes are distributed.
[0031] S2: Construct an EEG signal classification model and train the model using a training set.
[0032] like Figure 3 As shown, the model's network structure consists of multiple convolutional blocks, a spatial attention module, a channel attention module, and a classification module.
[0033] The convolution module includes 3D convolutional layers, batch normalized layers, exponential linear units, and average pooling layers.
[0034] First, the 3D signal matrix is used as the input to the first convolutional module. The 3D convolutional layer of the module can simultaneously learn the temporal and spatial features of the EEG signal. To ensure that the convolution process only learns the data features between adjacent channels, the kernel size of the convolutional module's spatial dimension is set to (2,2), and the stride is set to (1,1). Then, the learned spatiotemporal features are standardized, regularized, and downsampled.
[0035] Next, the output of the convolution module is input into the spatial attention module and the channel attention module, respectively. The spatial attention module first uses global average pooling in the spatial dimension to reduce computational cost, and then the fully connected layer calculates the spatial attention matrix to enhance the correlation of features within the spatial data. The channel attention module first performs global average pooling on each channel (the feature map generated by convolution); then it generates attention weight parameters through two stages: squeezing and excitation; then it uses the reduction ratio parameter r and a fully connected layer with ReLU activation to squeeze the number of channels to C / r (r=8); finally, it restores the scaled vector to its original shape.
[0036] Then, the outputs (feature maps) of the spatial attention module and the channel attention module are element-wise summed and merged before a sigmoid operation is performed. The sigmoid operation result is then element-wise multiplied with the output of the convolution module, and the result of the element-wise multiplication is then added back to the output of the convolution module.
[0037] The summation result is then passed through other convolutional modules to further learn spatiotemporal features.
[0038] In this embodiment, it is preferable to pass through three convolutional modules sequentially for optimal results. To preserve local information, the kernel sizes of Conv Block2, Conv Block3, and Conv Block4 are set to (2,2), (1,2), and (2,2), respectively, with strides of (2,2), (1,2), and (2,2), respectively. After further learning of spatiotemporal features through the convolutional modules, the channel features are divided into four regions: the middle region of the left parietal lobe, the middle region of the right parietal lobe, the lower half of the left parietal lobe, and the lower half of the right parietal lobe.
[0039] Finally, the output of the last convolutional module is input into the classification module to obtain the classification result.
[0040] The classification module consists of a global average pooling layer and a fully connected layer. Unlike traditional classifiers, this embodiment uses a global average pooling layer plus a fully connected layer as the final classification module. This significantly reduces the number of network parameters while retaining the information extracted by each convolutional and pooling layer. Finally, the probability value for each class is generated using a sigmoid function.
[0041] S3: After converting the EEG signal data to be classified into a three-dimensional signal matrix, input it into the trained EEG signal classification model to obtain the classification result of the EEG signal data to be classified.
[0042] This invention maps the acquired low-dimensional EEG signal data to a high-dimensional cortical space, a representation method that effectively preserves the local correlations between channels. Then, a local convolution strategy and attention mechanism are employed to enhance features, thereby improving the accuracy and universality of EEG motor imagery recognition and achieving real-time efficiency in brain-computer interaction.
[0043] Example 2:
[0044] The present invention also provides an EEG signal decoding terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the method embodiment described above in Embodiment 1 of the present invention.
[0045] Furthermore, as an executable solution, the EEG signal decoding terminal device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The EEG signal decoding terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described structure of the EEG signal decoding terminal device is merely an example and does not constitute a limitation on the EEG signal decoding terminal device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the EEG signal decoding terminal device may also include input / output devices, network access devices, buses, etc., and this embodiment of the invention does not limit this.
[0046] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices. The general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the EEG signal decoding terminal device, connecting all parts of the device via various interfaces and lines.
[0047] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the EEG signal decoding terminal device by running or executing the computer programs and / or modules stored in the memory and by calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0048] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of the present invention.
[0049] If the modules / units integrated in the EEG signal decoding terminal device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.
[0050] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.
Claims
1. A method for decoding electroencephalogram (EEG) signals, characterized in that, Includes the following steps: S1: Collect EEG signal data from different parts of the human body and construct a three-dimensional signal matrix according to the distribution of the collection electrodes on the scalp corresponding to the EEG signal data. Based on the obtained three-dimensional signal matrix, construct a training set and set the human body movement parts as the classification labels corresponding to the three-dimensional signal matrix. S2: Construct an EEG signal classification model and train the model using a training set; The model's network structure consists of multiple convolutional modules, a spatial attention module, a channel attention module, and a classification module. After the 3D signal matrix is input into a convolutional module, the output of the convolutional module is input into both the spatial attention module and the channel attention module. The outputs of the spatial attention module and the channel attention module are element-wise summed and merged before a sigmoid operation is performed. The sigmoid operation result is then element-wise multiplied with the output of the convolutional module, and the result of the element-wise multiplication is then added back to the output of the convolutional module. The summation result is passed through other convolutional modules in sequence, and the output of the last convolutional module is input into the classification module to obtain the classification result. S3: After converting the EEG signal data to be classified into a three-dimensional signal matrix, input it into the trained EEG signal classification model to obtain the classification result of the EEG signal data to be classified.
2. The EEG signal decoding method according to claim 1, characterized in that: Step S1 also includes filtering the acquired EEG signal data through a finite-length unit impulse response filter to extract EEG signals in the α and β bands.
3. The EEG signal decoding method according to claim 1, characterized in that: Step S1 also includes downsampling the EEG signal data.
4. The EEG signal decoding method according to claim 1, characterized in that: The convolution module includes 3D convolutional layers, batch normalized layers, exponential linear units, and average pooling layers.
5. The EEG signal decoding method according to claim 1, characterized in that: The convolution module consists of four parts.
6. The EEG signal decoding method according to claim 5, characterized in that: The kernel size of the convolution module for the input three-dimensional signal matrix is (2,2), and the stride is (1,1). The kernel sizes of the three convolution modules through which the summation result passes are (2,2), (1,2), and (2,2), and the strides are (2,2), (1,2), and (2,2), respectively.
7. The EEG signal decoding method according to claim 1, characterized in that: The classification module consists of a global average pooling layer and a fully connected layer.
8. A brainwave signal decoding terminal device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the method as described in any one of claims 1 to 7.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.