A spatial feature enhancement method based on EEG motion imagery

Abstract

A method for enhancing spatial features based on EEG motor imagery, which relates to the technical field of EEG signal processing, comprises: step 1, data filtering; step 2, data segmentation; step 3, building a data structure; and step 4, feature extraction. The present invention improves the recognition effect of motor imagery by enhancing key spatial features, improves the accuracy and universality of EEG motor imagery recognition, and improves the real-time efficiency of brain-computer interaction.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) signal processing, and in particular to a method for enhancing spatial features based on EEG motion imagery. Background Art

[0002] Traditional EEG feature extraction methods are difficult to take into account the time domain and frequency spectrum density. To avoid this problem, in the research of EEG machine learning, people began to use the end-to-end model to achieve classification and recognition to meet the high real-time requirements of brain-computer interaction. To achieve end-to-end machine learning of EEG. To achieve end-to-end machine learning of EEG, from raw data input to final output classifier, recognition results, etc., without too much manual setting, using layer neural network to couple any nonlinear function, this is not only to pre-process the input data, but also to classify it according to prior knowledge using traditional network learning features; it is also possible to embed neural pattern conversion algorithms in nonlinear learning networks, such as attention mechanism, back propagation, decision tree, greedy pursuit and other algorithms, making the fitting of data more flexible, it has high robustness and noise signal fault tolerance, and can completely approximate complex nonlinear relationships. However, since EEG information is easily affected by the external environment, interfered with, lost, and individual activity patterns vary greatly, the classification accuracy of motor imagery needs to be improved. The application of brain-computer interaction has high requirements for timeliness, complex tasks, or uncertain environments, and machine learning training with small samples is difficult to identify multiple tasks without image processing, making it difficult to identify EEG motor sensory rhythms with machine learning that relies on large samples, multiple source domains, and confusing deep learning methods.

[0003] Among the machine learning methods for learning EEG spatial features, the most prominent ones are convolutional neural networks (CNN) and deep belief networks (DBN). However, the network structure is too complex and the scalability needs to be improved. Currently, traditional machine learning methods such as convolution and pooling are mainly used for end-to-end EEG motion imagery spatial feature extraction, and their classification accuracy is not high. Summary of the invention

[0004] In view of the above defects or deficiencies in the prior art, the purpose of the present invention is to provide a method for enhancing spatial features based on EEG motor imagery. The method adopts an EEG temporal feature extraction method with a self-attention mechanism, which enhances key spatial features, improves the motor imagery recognition effect, improves the accuracy and universality of EEG motor imagery recognition, and realizes real-time efficiency of brain-computer interaction.

[0005] In order to solve the above problems, in a first aspect, the present invention provides a method for enhancing spatial features based on EEG motion imagery, the method comprising:

[0006] Data filtering, the fast Fourier transform algorithm FFT is used for filtering to extract the EEG signals in the α and β bands, that is, the 4Hz to 40Hz rhythm segment, of the EEG motor imagery area;

[0007] For data segmentation, a 2-s time window and a sliding window with a step size of 100 ms were used to obtain EEG data;

[0008] Construct a data structure, extract the data of 20 acquisition channels in the motor imagery area, arrange them in a front-to-back and left-right symmetrical manner, and construct a two-dimensional signal matrix;

[0009] Feature extraction.

[0010] In a second aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the embodiment of the present application when executing the computer program.

[0011] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to:

[0012] When the computer program is executed by a processor, the method described in the embodiment of the present application is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The embodiments of the present invention will be described below with reference to the accompanying drawings, wherein

[0014] Figure 1 The schematic diagram of the spatial feature enhancement process based on EEG motion imagery of the present application is shown;

[0015] Figure 2 A schematic diagram of 20 training electrodes extracted in this application is shown;

[0016] Figure 3 A schematic diagram of the SSA module of the present application is shown. DETAILED DESCRIPTION

[0017] In order to make the invention purpose, technical process and technical innovation of the present invention more clearly described, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in each embodiment of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0018] To achieve the above objectives, the present invention provides a method for enhancing spatial features based on EEG motion imagery. The main process is as follows: Figure 1 As shown, the method includes:

[0019] Data filtering, the fast Fourier transform algorithm FFT is used for filtering to extract the EEG signals in the α and β bands, that is, the 4Hz to 40Hz rhythm segment, of the EEG motor imagery area;

[0020] For data segmentation, a 2-s time window and a sliding window with a step size of 100 ms were used to obtain EEG data;

[0021] Construct a data structure, extract the data of 20 acquisition channels in the motor imagery area, arrange them in a front-to-back and left-right symmetrical manner, and construct a two-dimensional signal matrix; Figure 2 As shown in the figure, different EEG electrodes reflect the electrical fluctuations in different brain regions, and there is a strong relationship between different EEG electrodes. Currently, a large number of experiments have shown that left-right movement imagination corresponds to different brain regions, and the left / right hand movement imaging areas are mainly in the supplementary motor area (SMA) and the primary motor area (PMA). Therefore, the present invention extracts 20 acquisition points of SMA and PMA collected by the KU-MI experiment, Figure 2 Electrodes 1-20 are marked in dark colors. In addition, considering that the calculations of machine learning such as convolution and pooling are only limited local filtering, important spatial feature information may be lost during the calculation process. To avoid this problem as much as possible, the input data structure of the training of the present invention is arranged in the order of electrodes. Figure 2 The index values ​​of the dark-colored electrodes are marked in the middle, and the collecting electrodes are arranged symmetrically from the front to the back, so that after the convolution and pooling calculations from the 1D to the 2D data structure, the spatial information of the left and right brain EEG collection points can be learned.

[0022] Feature extraction.

[0023] In particular, the feature extraction step includes: weighting the intermediate features of the EEG 2D feature values ​​through the Conv2D encoder, and then entering the self-attention calculation to learn and collect the spectral features in the lead electrode space. The self-attention mechanism extracts the features of sparse data through convolution and pooling. After calculation through the attention mechanism, it will reduce the dependence on external features and strengthen the correlation of internal features of the data;

[0024] Each lead of the EEG signal is processed independently, and the spectral features of the spatial dimension are still hidden in the network, and their rules have not been learned. Before entering the feature learning of the self-attention mechanism, the feature output tensor of the TSA module is first converted into the input tensor of the SSA module. Among them, i, j, k are three-dimensional values, B is the number of output feature maps, R and C are the reconstruction coefficients of 1D feature values ​​converted into 2D tensors, then R×C=D, and D is the output feature of the TSA module.

[0025] After extracting the time domain eigenvalues, the SSA module focuses on how to extract the spectral feature information of the left and right brain spatial domains. For spatial domain feature extraction, it should be noted that if the convolution calculation is too short or too long, the amplitude information of the ERD / ERS phenomenon will not be extracted. In particular, if triple feature extraction is initially performed on the network's input data, further reduction through convolution and maximum pooling calculations will result in the loss of valuable information and cannot be used for action recognition. To avoid this problem, the SSA module first converts the 1D feature values ​​output by the TSA module into a 2D tensor with a 4-column structure, Conv2D=(2,2), so that the symmetrical lead signals of the two hemispheres can be calculated through convolution to obtain the feature map weights. Before and after the self-attention calculation, convolution and Dropout calculations are added. The former is to obtain dynamic weights based on the feature information to prepare for the self-attention calculation, and the latter is to compress the eigenvalues ​​to facilitate the improvement of the calculation speed of the next module, such as Figure 3 shown.

[0026] In order to better understand the present invention, the above is described in detail in conjunction with specific examples of the present invention, but it is not intended to limit the present invention. Any simple modification made to the above embodiments based on the technical essence of the present invention still falls within the scope of the technical solution of the present invention.

[0027] As another aspect, the present application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the embodiment of the present application when executing the computer program.

[0028] As another aspect, the present application further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the aforementioned device in the above-mentioned embodiment; or a computer-readable storage medium that exists independently and is not assembled into the device. The computer-readable storage medium stores one or more programs, and the aforementioned programs are used by one or more processors to execute the method described in the embodiment of the present application.

[0029] Any reference to the storage medium used in the embodiments of the present application may include non-volatile, volatile memory. Suitable non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), optical disk (including CD-ROM read-only memory (CD-ROM) and digital versatile disk (DVD)), or flash memory. Volatile memory may include random access memory (RAM), which is used as an external high-speed cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0030] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (Programmable Gate Array; hereinafter referred to as: PGA), a field programmable gate array (Field Programmable Gate Array; hereinafter referred to as: FPGA), etc.

[0031] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0032] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software and firmware. If the integrated module is implemented in the form of software and firmware and sold or used as an independent product, it can be transferred from a storage medium or a network to a computer with a dedicated hardware structure for functional implementation.

[0033] It should also be noted that the steps of performing the above series of processes can naturally be performed in chronological order according to the order of description, but do not necessarily need to be performed in chronological order. Some steps can be performed in parallel or independently of each other.

[0034] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present application. Any person skilled in the art can change, modify, replace and modify the above embodiments without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention is not limited thereto, and should be based on the scope defined by the claims.

Claims

1. A method for enhancing spatial features based on EEG motion imagery, the method comprising: Data filtering, the fast Fourier transform algorithm FFT is used for filtering to extract the EEG signals in the α and β bands, that is, the 4Hz to 40Hz rhythm segment, of the EEG motor imagery area; For data segmentation, a 2-s time window and a sliding window with a step size of 100 ms were used to obtain EEG data; Construct a data structure, extract the data of 20 acquisition channels in the motor imagery area, arrange them in a front-to-back and left-right symmetrical manner, and construct a two-dimensional signal matrix; Feature extraction; The feature extraction step includes: weighting the intermediate features of the EEG 2D feature values ​​through the Conv2D encoder, and then entering the self-attention calculation to learn and collect the spectral features in the lead electrode space. The self-attention mechanism extracts the features of sparse data through convolution and pooling. After calculation through the attention mechanism, it will reduce the dependence on external features and strengthen the correlation of internal features of the data; Each lead of the EEG signal is processed independently, and the spectral features of the spatial dimension are still hidden in the network, and their rules have not been learned. Before entering the feature learning of the self-attention mechanism, the feature output tensor of the TSA module is first converted into the input tensor of the SSA module. Among them, i, j, k are three-dimensional values, B is the number of output feature maps, R and C are the reconstruction coefficients of 1D feature values ​​converted into 2D tensors, then R×C=D, and D is the output feature of the TSA module.

2. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to claim 1 is implemented.

3. A computer-readable storage medium having a computer program stored thereon, the computer program being used to: When the computer program is executed by a processor, the method according to claim 1 is implemented.

Images