A noise identification method for EEG data using deep neural network
Through the combined convolutional structure of deep neural network, the EEG signal is automatically processed, which solves the problem of noise recognition and denoising of patients with single-limb movement disorders after trance, and realizes efficient acquisition of limb rehabilitation training data, reduces the impact of human intervention, and is suitable for trance. Graded rehabilitation training for patients with posterior single limb movement disorders.
Patent Information
- Application Number
- CN202210307716.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-03-27
AI Technical Summary
The prior art is difficult to effectively process EEG signals in patients with single-limb movement disorders after trauma, especially in noise recognition and denoising, which cannot meet the needs of real-time limb rehabilitation training, and traditional methods rely on human intervention and have subjective effects.
A deep neural network method was designed, combining one-dimensional convolution, two-dimensional convolution and densely connected convolution, and noise recognition and denoising EEG data through windowing, automatic processing of EEG signals to reduce noise impact, and according to The appropriate exercise paradigm is chosen for the recovery process of different patients.
High sensitivity recognition and denoising of EEG noise are achieved, real-time noise-free data is provided, and the stray patients are assisted in hierarchical rehabilitation training is assisted, which improves the function recovery effect of brain motor area and reduces the subjective impact of human intervention.
Smart Images

Figure CN114861702B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural network algorithm technology, and in particular to a method for noise recognition of EEG data using a deep neural network. Background Art
[0002] Motor imagery (MI) refers to using the brain to subjectively imagine the movement of limbs or muscles when there is no objective limb or muscle movement, thereby stimulating the corresponding brain area. At present, MI is mainly used to assist in the training and treatment of some limb dysfunction diseases such as post-stroke rehabilitation and muscle atrophy.
[0003] The commonly designed MI paradigms nowadays mostly imagine the movement of the left and right limbs, but most patients after stroke have unilateral movement disorders. Therefore, the classic MI paradigm cannot meet the doctors' training and treatment of patients' affected limbs to a certain extent. At the same time, in order to obtain good electroencephalogram (EEG) signals under the MI paradigm, experienced technicians are required to perform operations such as filtering, independent component analysis, and re-reference on the original signals. However, limb rehabilitation requires real-time feature extraction and classification of EEG for controlling exoskeletons and other assistive devices.
[0004] The MI paradigm for post-stroke unilateral limb movement disorders and the automatic denoising algorithm of EEG signals were thus generated. Considering that post-stroke patients have significant motor impairments, the movements that need to be imagined in the paradigm should be started from easy to difficult. With the rise of deep learning, there have been many research results on applying neural networks to EEG denoising. However, considering the time domain of EEG alone cannot maximize the improvement of signal quality, and the choice of signal length also leads to different results. Summary of the invention
[0005] In view of the shortcomings of existing algorithms, the present invention designs an MI paradigm suitable for post-stroke unilateral limb movement disorder, which mainly performs individual and linkage training on the shoulder, elbow, wrist and finger joints of the patient's right upper limb; and proposes a noise recognition method for EEG data using a deep neural network constructed by one-dimensional convolution, two-dimensional convolution and densely connected convolution, and removes noise from the EEG data by windowing.
[0006] The paradigm design is based on the needs of patients with unilateral movement disorders after stroke. An MI paradigm suitable for unilateral movement disorders after stroke is designed to assist patients in performing limb training to varying degrees. Rehabilitation doctors evaluate the upper limb motor function of stroke patients through the Brunnstrom hemiplegia functional evaluation method, the Fugl-Meyer method, and the Ueda Min method. The appropriate paradigm is selected based on the scores after the evaluation to diagnose the patient's joint flexibility.
[0007] EEG data processing includes downsampling, filtering, re-referencing and ICA analysis. The part that requires human intervention is transformed into automatic data processing through deep neural networks. Human intervention means that during ICA analysis, experienced technicians are required to review the results of ICA, determine what the technicians consider to be noise, and reduce the impact of subjective consciousness through deep learning methods.
[0008] The technical solution adopted by the present invention is: a method for noise recognition of EEG data using a deep neural network, comprising the following steps:
[0009] S1. Collect EEG data induced by combined movements of the four joints of shoulder, elbow, wrist and fingers under stroke status through an EEG cap, and divide the movements into single joint training and joint comprehensive training;
[0010] Further, the single joint training is to stimulate the brain motor function area to generate EEG data through seven actions including shoulder flexion 30°, shoulder extension 30°, elbow flexion 90°, maximum elbow flexion, left wrist rotation, right wrist rotation, and finger fisting; the multi-joint comprehensive training is to stimulate the brain motor function area to generate EEG data through eight actions including forearm pronation, forearm supination, upper limb flexion 90°, upper limb extension and lifting to touch the waist, upper limb abduction 90°, upper limb flexion 180°, shoulder touching experiment, and finger-nose experiment;
[0011] S2, downsampling, coarse filtering and ICA independent component decomposition are performed on the data in the dataset to obtain IC activation and IC topography of multiple independent components;
[0012] S3, cropping the IC topographic map, dividing the IC activation into windows of certain length according to a certain time, and setting the overlapping time of adjacent windows;
[0013] S4. Construct a deep neural network consisting of one-dimensional CNN convolution, two-dimensional CNN convolution and DDN convolution, feed the IC topography into the two-dimensional CNN convolution network, feed the IC activation into the one-dimensional convolution network, and feed the output values of the two convolution networks into the DNN convolution network, and set the hyperparameters;
[0014] Furthermore, the one-dimensional CNN convolution includes: a first convolution layer, a kernel width of 20, a number of filters of 5, and a step size of 2; a second convolution layer, a kernel width of 6, a number of filters of 16, and a step size of 2; a third convolution layer, a kernel width of 15, a number of filters of 34, and a step size of 1; a fourth convolution layer, a kernel width of 11, a number of filters of 21, and a step size of 1; a fifth convolution layer, a kernel width of 6, a number of filters of 45, and a step size of 1; a sixth convolution layer, a kernel width of 7, a number of filters of 30, and a step size of 1; the first, second, third, fifth, and sixth convolution layers are all connected to a pooling layer with a kernel width of 4 and a step size of 2; the activation functions are all PreLU functions; and finally, the Flatten layer is connected to reduce the dimension to obtain a feature column vector activated by IC;
[0015] Furthermore, the two-dimensional CNN convolution includes: the first convolution layer of the IC map, the kernel size is 20*20, the number of filters is 40, and the step size is 2*2; the second convolution layer of the IC map, the kernel size is 5*5, the number of filters is 50, and the step size is 2*2; the third convolution layer of the IC map, the kernel size is 6*6, the number of filters is 14, and the step size is 1*1; the fourth convolution layer of the IC map, the kernel size is 9*9, the number of filters is 35, and the step size is 2*2; the fifth convolution layer of the IC map, the kernel size is 12*12 , the number of filters is 21, and the step size is 1*1; the sixth convolutional layer of the IC map has a kernel size of 16*16, the number of filters is 45, and the step size is 2*2; the seventh convolutional layer of the IC map has a kernel size of 8*8, the number of filters is 30, and the step size is 2*2; the second, third, fifth, and seventh convolutional layers of the IC map are all connected to a pooling layer with a size of 2*2 and a step size of 1*1; the activation function is the PreLU function; finally, the Flatten layer is connected to reduce the dimension to obtain the feature column vector of the IC topographic map.
[0016] Further, the DDN convolution includes a first fully connected layer, a second fully connected layer and a third fully connected layer, the first fully connected layer has an output dimension of 135, the second fully connected layer has an output dimension of 210 and the third fully connected layer has an output dimension of 28;
[0017] S5. The output value of the DNN convolutional network is classified by the softmax classifier, and the EEG data is screened and removed according to the classification results. The signal-to-noise ratio and root mean square difference of the EEG data after noise removal are calculated to evaluate the denoising effect.
[0018] The classification results are eye movement noise, electrocardiogram noise, head movement noise, and electromyography noise;
[0019] The EEG data was detected in a time window of 5000ms, and the four types of noise classified by the softmax classifier were used as detection indicators. The EEG data was step-by-step detected according to the detection window, and the time windows containing noise components were screened out and eliminated.
[0020] The beneficial effects of the present invention are:
[0021] 1. The designed single-limb motor imagery paradigm can assist patients after stroke in rehabilitation training of the affected limb, which is helpful for the functional recovery of the brain motor area; at the same time, different paradigms can be selected according to different post-stroke rehabilitation processes to form a hierarchical rehabilitation training system;
[0022] 2. Using one-dimensional CNN convolution, two-dimensional CNN convolution and DDN convolution to build a deep neural network can effectively classify the noise components. It is highly sensitive to the EEG noise collected in real time and can effectively remove the noise components;
[0023] 3. Assist patients with different degrees of stroke to carry out rehabilitation training and obtain real-time noise-free EEG data of patients during rehabilitation training. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 The present invention is a noise identification method for EEG data using a deep neural network;
[0025] Figure 2 Schematic diagram of the time flow of two single-limb motor imagery paradigms designed for the present invention;
[0026] Figure 3 It is a structural diagram of the data communication and format conversion module;
[0027] Figure 4 It is a specific implementation flow chart of the present invention. DETAILED DESCRIPTION
[0028] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.
[0029] The paradigm design part of the present invention is developed using the JavaScript language in the Windows environment, and the deep neural network part uses Pycharm software and a GPU computing unit with Keras 2.4.0 and Tensorflow 2.4.1 deep learning environments.
[0030] like Figure 2 As shown, the subjects put their hands naturally on their thighs and their eyes were 1 meter away from the screen. The examiner selected single joint training or multi-joint comprehensive training according to the subjects' limb function status.
[0031] Single joint training: The stimulation sources are divided into seven movements: shoulder flexion 30°, shoulder extension 30°, elbow flexion 90°, maximum elbow flexion, left wrist rotation, right wrist rotation, and fist. There will be a 2000ms prompt before the stimulation source appears. Each stimulation source will appear randomly for 4000ms. After the end, there will be a 2000ms rest before the next 2000ms prompt. Each movement will appear randomly 20 times in each round of training, for a total of 140 times. The entire rehabilitation training paradigm has a total of 5 rounds.
[0032] Multi-joint comprehensive training: Compared with the single-joint training paradigm, the multi-joint comprehensive paradigm combines four joints to form 8 compound movements such as the finger-to-nose experiment and the shoulder-to-shoulder lift. After a 2000ms prompt, the multi-joint action-guided GIF image appears as a stimulus source for 6000ms (looped twice), and there will also be a 2000ms break after the end. The difference is that due to the complexity of the training movements, each movement appears randomly 10 times in each round of training, for a total of 80 times. The entire rehabilitation training paradigm has a total of 5 rounds.
[0033] like Figure 3 As shown, the communication module in the data communication and format conversion module connects the signal output end of the EEG cap with the input end of the data analysis module; the communication module transmits the data collected by the EEG cap hardware to the data analysis module through the socket protocol, realizing online data processing; the format conversion mainly converts the 64 or 128-channel raw data into .raw, .mff or .bdf format data files for subsequent readability and storage.
[0034] like Figure 1 As shown in FIG. 1 , a noise recognition method for EEG data using a deep neural network is shown in FIG. 1 . The main steps are as follows:
[0035] S1. Collect EEG data induced by the combined movements of the shoulder, elbow, wrist and fingers of the subjects in the stroke state through the EEG cap.
[0036] S2, first downsample the collected data to 250Hz, then perform a coarse filter of 0.1-50Hz on the data stream, and then perform independent component analysis (ICA). The number of independent components (ICs) is selected from 8 to 24 according to the number of leads. The present invention selects 12 ICs for 64 leads to obtain the activation and topographic map of each component;
[0037] S3, crop the topographic map of 12 ICs to 160 pixels * 160 pixels * 3 colors;
[0038] The 12 IC activations were divided into several trails (trials) of 8000 ms each, with two adjacent trails overlapping for 2000 ms. When the data length was less than 8000 ms, it was automatically overlapped forward to 8000 ms.
[0039] S4. Select the hyperparameters required for building a neural network, and build a deep neural network consisting of a one-dimensional convolutional neural network (CNN), a two-dimensional CNN, and a dense convolutional network (DenseNet), specifically:
[0040] The two-dimensional CNN structure used to extract features from IC topography is:
[0041] Layer 1: convolutional layer, kernel size is 20*20, number of filters is 40, step size is 2*2, activation function is PreLU, which is used to improve the overfitting of the model;
[0042] Layer2: convolutional layer, kernel size is 5*5, number of filters is 50, step size is 2*2, activation function is also set to PreLU, and a maximum pooling layer is added after Layer2, with a size of 2*2 and a step size of 1*1;
[0043] Layer3: Convolutional layer with PreLU, kernel size is 6*6, number of filters is 14, stride is 1*1, and a maximum pooling layer of the same size as Layer2 is added after this layer;
[0044] Layer4: convolutional layer with PreLU, kernel size is 9*9, number of filters is 35, and step size is 2*2;
[0045] Layer5: Convolutional layer with PreLU, kernel size of 12*12, number of filters of 21, stride of 1*1, followed by a maximum pooling layer, the same size as Layer3;
[0046] Layer6: convolutional layer with PreLU, kernel size is 16*16, number of filters is 45, and stride is 2*2;
[0047] Layer7: Convolutional layer with PreLU, kernel size 8*8, number of filters 30, stride 2*2, followed by a maximum pooling layer with pooling size 2*2 and stride 2*2;
[0048] Layer8: After the IC topography passes through Layer7, the result is reduced in dimension to obtain the feature column vector of the IC topography;
[0049] The one-dimensional CNN structure used to extract features from IC temporal activation is:
[0050] Layer 1: Convolutional layer with PreLU activation function, kernel width 20, 5 filters, stride 2, followed by a max pooling layer with kernel width 4 and stride 2;
[0051] Layer2: Convolutional layer with PreLU, kernel width 6, number of filters 16, stride 2, followed by the same max pooling layer as Layer1;
[0052] Layer3: Convolutional layer with PreLU, kernel width 15, number of filters 34, stride 1, followed by the same max pooling layer as Layer1;
[0053] Layer4: convolutional layer with PreLU, kernel width 11, number of filters 21, stride 1;
[0054] Layer5: Convolutional layer with PreLU, kernel width 6, number of filters 45, stride 1, followed by the same max pooling layer as Layer1;
[0055] Layer 6: Convolutional layer with PreLU, kernel width 7, number of filters 30, stride 1, followed by the same max pooling layer as Layer 1.
[0056] Layer7: Flatten layer. After the temporal activation of IC passes through Layer6 of the one-dimensional CNN, the result is reduced in dimension to obtain the feature column vector of IC temporal activation.
[0057] The features of the IC topography and the time activation features are merged to obtain a new feature vector, which is input to the Dense layer. The specific structure of the Dense network is as follows:
[0058] Layer 1: Fully connected layer, further processing the merged feature vector, output dimension 135;
[0059] Layer2: fully connected layer, output dimension 210;
[0060] Layer3: Fully connected layer, output dimension is 28, followed by softmax to output the result, and the output dimension is 4.
[0061] S5. The output value of the DNN convolutional network is classified by the softmax classifier, and the EEG data is screened and removed according to the classification results. The signal-to-noise ratio and root mean square difference of the EEG data after noise removal are calculated to evaluate the denoising effect.
[0062] Denoising effect test: After the EEG data is filtered and removed by the deep neural network, it is necessary to select appropriate indicators to evaluate the denoising effect of the EEG data; the evaluation of the denoising effect mainly considers the proportion of noise in the overall signal and the deviation before and after denoising;
[0063] The signal-to-noise ratio SNR and root mean square difference RMSD are used to evaluate the denoising effect of the present invention, and the formula is as follows:
[0064]
[0065]
[0066] In the formula, x eeg [n] represents the effective component in the EEG after denoising, x noise [n] represents the noise component in the EEG after denoising. represents the power of the effective component in the EEG after denoising, Represents the power of the noise component in the EEG after denoising, V eeg Represents the voltage effective value of the effective component in the EEG after denoising, V noise Represents the voltage effective value of the noise component in the EEG after denoising; EEG i represents the denoised signal, eeg i represents the EEG signal before denoising;
[0067] Among them, SNR represents the ratio of the effective component to the noise component in the denoised signal. The larger the ratio, the more effective components it contains; RMSE represents the deviation between the denoised signal and the original signal. The smaller the deviation, the more effective components are retained.
[0068] like Figure 4 As shown, the experimental process of this embodiment is as follows:
[0069] 1. The experimenter puts on the EEG cap for the subject and selects an appropriate single-limb motor imagery paradigm according to the subject's post-stroke limb motor function;
[0070] 2. After the training begins, the EEG cap collects real-time EEG data during the experiment. The data communication module transfers the data from the hardware to the software, and the format conversion module converts the data into a format that can be processed by the software.
[0071] 3. The format-converted data is sent to EEG data analysis. After downsampling, filtering, and ICA decomposition, the deep neural network is used to filter out the noise, and finally the filtered noise is removed from the data.
[0072] 4. The noise-removed data is output and stored in real time by the noise-free EEG output module.
[0073] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.
Claims
1. A noise identification method for EEG data using deep neural network, It is characterized in that The following steps are involved: S1. Collect EEG data induced by combined movements of the four joints of shoulder, elbow, wrist and fingers under stroke status through an EEG cap, and divide the movements into single joint training and multi-joint comprehensive training; S2, downsampling, coarse filtering and ICA independent component decomposition of EEG data to obtain IC activation and IC topography of multiple independent components; S3, cropping the IC topographic map, dividing the IC activation into several trails according to a certain time, and setting the overlapping time of adjacent windows; S4. Construct a deep neural network consisting of one-dimensional CNN convolution, two-dimensional CNN convolution and DDN convolution, feed the IC topography into the two-dimensional CNN convolution network, feed the IC activation into the one-dimensional convolution network, and feed the output values of the two convolution networks into the DNN convolution network, and set the hyperparameters; The one-dimensional CNN convolution includes: a first convolution layer, a kernel width of 20, a number of filters of 5, and a step length of 2; a second convolution layer, a kernel width of 6, a number of filters of 16, and a step length of 2; a third convolution layer, a kernel width of 15, a number of filters of 34, and a step length of 1; a fourth convolution layer, a kernel width of 11, a number of filters of 21, and a step length of 1; a fifth convolution layer, a kernel width of 6, a number of filters of 45, and a step length of 1; a sixth convolution layer, a kernel width of 7, a number of filters of 30, and a step length of 1; the first, second, third, fifth, and sixth convolution layers are all connected to a pooling layer with a kernel width of 4 and a step length of 2; the activation function is a PreLU function; and finally, a Flatten layer is connected to reduce the dimension to obtain a feature column vector activated by IC; The two-dimensional CNN convolution includes: the first convolution layer of the IC map, the kernel size is 20*20, the number of filters is 40, and the step length is 2*2; the second convolution layer of the IC map, the kernel size is 5*5, the number of filters is 50, and the step length is 2*2; the third convolution layer of the IC map, the kernel size is 6*6, the number of filters is 14, and the step length is 1*1; the fourth convolution layer of the IC map, the kernel size is 9*9, the number of filters is 35, and the step length is 2*2; the fifth convolution layer of the IC map, the kernel size is 12*12, the number of filters is 12*12, and the step length is 1*1. The number of filters is 21, and the step size is 1*1; the sixth convolutional layer of the IC map has a kernel size of 16*16, a number of filters of 45, and a step size of 2*2; the seventh convolutional layer of the IC map has a kernel size of 8*8, a number of filters of 30, and a step size of 2*2; the second, third, fifth, and seventh convolutional layers of the IC map are all connected to a pooling layer of size 2*2 and a step size of 1*1; the activation function is the PreLU function; finally, the Flatten layer is connected to reduce the dimension to obtain the feature column vector of the IC topographic map; The DDN convolution includes a first fully connected layer, a second fully connected layer and a third fully connected layer, the first fully connected layer has an output dimension of 135, the second fully connected layer has an output dimension of 210 and the third fully connected layer has an output dimension of 28; S5. The output value of the DNN convolutional network is classified by the softmax classifier, and the EEG data is screened and removed according to the classification results. The signal-to-noise ratio and root mean square difference of the EEG data after noise removal are calculated to evaluate the denoising effect.
2. The method for noise identification of EEG data using a deep neural network according to claim 1, It is characterized in that The single joint training stimulates the brain motor function area to generate EEG data through seven actions including shoulder flexion 30°, shoulder extension 30°, elbow flexion 90°, maximum elbow flexion, wrist left rotation, wrist right rotation, and finger fisting; The multi-joint comprehensive training stimulates the brain motor function area to generate EEG data through eight kinds of actions including forearm pronation, forearm supination, upper limb flexion 90°, upper limb extension and lifting to touch the waist, upper limb abduction 90°, upper limb flexion 180°, shoulder touching experiment, and finger-nose experiment.
