A motor imagery decoding method based on multi-subject data merging adversarial training

Abstract

The invention discloses a motor imagery decoding method based on multi-subject data merging adversarial training. The decoding method adopts multi-subject data merging adversarial training of a motor imagery classification model, alternately uses forward training and reverse training to train the motor imagery classification model, so that the model focuses on feature extraction of motor imagery tasks, has better robustness across time periods and across subjects, solves the problem of low accuracy when multi-subject data is merged to train the motor imagery classification model, effectively improves the training effect when multi-subject data is merged, and is more practical in real applications such as stroke rehabilitation; at the same time, the invention broadens the acquisition source of model training data, reduces the amount of data that needs to be collected for a single subject, and can alleviate the difficulties in collecting EEG data and the problem of lack of training samples; at the same time, compared with the traditional one-to-one model training method, the classification accuracy of the invention is higher.

Description

Technical Field

[0001] The present invention belongs to the technical field of brain-computer interface motor imagery paradigm, and in particular relates to a motor imagery decoding method based on multi-subject data merging adversarial training. Background Art

[0002] The motor imagery paradigm (MI) in the brain-computer interface (BCI) is to judge the user's movement intention by analyzing EEG signals and detecting and identifying the activation effects of different brain regions, thereby realizing direct communication and control between the human brain and external devices. Since it does not require external stimulation but reflects the user's autonomous movement awareness, it is of great significance in the fields of motor function compensation and motor function rehabilitation. However, due to the low signal-to-noise ratio of EEG signals, the lack of training samples, and the great influence of time, scene and individual changes on classification accuracy, the EEG-based brain-computer interface decoupling decoding algorithm still faces some challenges.

[0003] At present, motor imagery classification models rarely use multiple subject data to merge and train a single model without cross-subject requirements. Most of the time, subject data and models are trained one-to-one, that is, one subject data is used to train one model. Only when there is a cross-subject requirement, all subject data are used for training. The reason is that in general, using multiple subject data to merge and train will lead to an increase in the difference in training data, thereby significantly reducing the final average accuracy. Although the one-to-one training method has a high accuracy rate, it requires a high amount of data for a single subject, and the difficulty of collecting EEG data leads to a lack of training data. Some experiments have shown that when using multiple subject data to merge and train a motor imagery classification model, the extracted features are directly used for identity recognition, which has a very strong separability and can achieve a high identity recognition accuracy. Therefore, we can think that this is the embodiment of the large differences between the data subjects, which causes the model to not only extract the features of motor imagery classification during feature extraction, but also extract many features related to the subject identity. To address this problem, this method proposes a motor imagery decoding method based on multi-subject data merging adversarial training, which improves the training effect of multi-subject data merging and can become a means of EEG data expansion. Summary of the invention

[0004] The purpose of the present invention is to provide a motor imagery decoding method based on multi-subject data merging adversarial training in view of the deficiencies in the prior art.

[0005] The present invention provides a motor imagery decoding method based on multi-subject data merging adversarial training, which comprises the following steps:

[0006] Step 1: Collect EEG signals of multiple subjects who perform motor imagery tasks to obtain an EEG signal data set; the EEG signals in the EEG signal data set carry motor imagery type labels and identity labels; and after preprocessing the collected data set, divide it into a training set and a validation set.

[0007] Step 2: Build a motor imagery classification model

[0008] The motor imagery classification model includes three modules: feature extraction model, motor imagery classifier and identity recognition classifier. The feature extraction model adopts convolutional neural network. The output features of the feature extraction model are expanded through the Flatten layer and then input into the motor imagery classifier and the identity recognition classifier respectively. Both the motor imagery classifier and the identity recognition classifier adopt a fully connected layer structure.

[0009] Step 3: Use the training set to alternately perform forward training and reverse training on the motor imagery classification model.

[0010] The process of forward training is as follows:

[0011] The training set is input into the motor imagery classification model to obtain the classification probabilities of motor imagery and identity recognition; the cross entropy of the two classifiers is calculated according to the two classification probabilities, and the parameters of the feature extraction model and the two classifiers are updated. Among them, the cross entropy of the motor imagery classifier updates the parameters of the motor imagery classifier and the feature extraction model; the cross entropy of the identity recognition classifier updates the parameters of the identity recognition classifier.

[0012] The process of reverse training is as follows:

[0013] The training set is input into the motor imagery classification model to obtain the classification probabilities of motor imagery and identity recognition; the cross entropy of the motor imagery classifier and the negative entropy of the identity recognition classifier are calculated respectively according to the two classification probabilities; the parameters of the motor imagery classifier and the feature extraction model are jointly updated by the obtained cross entropy and negative entropy.

[0014] Step 4: Verify the motor imagery classification model obtained from each reverse training through the validation set, and select the motor imagery classification model with the highest accuracy as the final motor imagery classification model.

[0015] Step 5: Collect the subject's EEG signal, and input the obtained EEG signal into the final motor imagery classification model to obtain the subject's motor imagery type.

[0016] Preferably, in step 3, the parameters of the motor imagery classifier and the feature extraction model are updated by multiplying the negative entropy of the identity recognition classifier by the proportional coefficient α and adding the result to the cross entropy of the motor imagery classifier.

[0017] Preferably, the value of the proportionality coefficient α is 0.5.

[0018] Preferably, if the subject is not one of the subjects in step 1, EEG signals of the subject performing the motor imagery task are collected, and the obtained EEG signals are added to the EEG signal dataset; steps 2, 3 and 4 are re-executed based on the updated EEG signal dataset; and the obtained motor imagery classification model is used to identify the subject's motor imagery.

[0019] Preferably, the feature extraction model includes a temporal convolution layer, a spatial convolution layer and a depth-separable convolution layer. A BN batch normalization layer is provided after the temporal convolution layer; a BN batch normalization layer, an ELU activation function, a pooling layer and a Dropout layer are provided after the spatial convolution layer and the depth-separable convolution layer.

[0020] Preferably, the number of convolution kernels F1 of the temporal convolution layer is 8, and the size is (1, 64); the number of convolution kernels of the spatial convolution layer is 16, and the size is (number of EEG channels, 1); the number of convolution kernels F2 of the depthwise separable convolution layer is 16, and the size is (1, 16); the size of the pooling layer after the spatial convolution layer is (1, 4); the size of the pooling layer after the depthwise separable convolution layer is (1, 8); and the dropout parameters are all set to 0.25.

[0021] Preferably, the motor imagery classifier and the identity recognition classifier are followed by a Softmax activation function for outputting classification probabilities.

[0022] Preferably, in step 1, the process of preprocessing the data set is as follows:

[0023] (1) Bandpass filter the data in the range of 0.5 Hz to 100 Hz.

[0024] (2) Downsample the EEG data to 100 Hz.

[0025] Preferably, in step 1, there are four types of motor imagery, namely, left hand, right hand, feet and tongue motor imagery.

[0026] Preferably, in step 2, the dimension of the output feature of the motor imagery classifier is equal to the number of motor imagery categories; the dimension of the output feature of the identity recognition classifier is equal to the number of subjects participating in the collection of the EEG signal data set.

[0027] Preferably, in step 5, the learning rate of each module is monitored separately and adjusted dynamically. The initial learning rate of each module is set to 0.001, the learning rate decay value is 0.6, the learning strategy is to trigger decay if the loss does not decrease for 50 consecutive times, the minimum learning rate is 0.00001, and early stopping is triggered if the loss of any module decreases for 500 consecutive times when it is at the minimum learning rate.

[0028] The present invention has the following beneficial effects:

[0029] 1. The present invention adopts multi-subject data merging to train the motor imagery classification model adversarially, and alternately uses forward training and reverse training to train the motor imagery classification model, which solves the problem of low accuracy when multi-subject data is combined to train the motor imagery classification model, and effectively improves the training effect when multi-subject data is combined.

[0030] 2. The present invention broadens the sources of model training data, reduces the amount of data that needs to be collected from a single subject, and can alleviate the difficulties in collecting EEG data and the problem of lack of training samples; at the same time, compared with the traditional one-to-one model training method, the classification accuracy of the present invention is higher.

[0031] 3. The present invention adopts an adversarial decoupling training method, uses the negative entropy loss of identity recognition as a measure of the difference between subjects in EEG data, decouples identity information from EEG data, and enables the model to focus on feature extraction of motor imagery tasks. It has better robustness across time periods and across subjects, and improves the practicality of the motor imagery classification model in real-world applications such as stroke rehabilitation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is the overall workflow diagram of Example 1 of the present invention.

[0033] Figure 2 Schematic diagram of the distribution of sampling electrodes for the data set used in Example 1 of the present invention; wherein, Figure (a) is a schematic diagram of the international 10-20 standard electrode position, and Figure (b) is a schematic diagram of the EOG three-channel position.

[0034] Figure 3 This is a schematic diagram of a data set division method according to Embodiment 1 of the present invention.

[0035] Figure 4 This is a schematic diagram of forward training according to Embodiment 1 of the present invention.

[0036] Figure 5 This is a schematic diagram of reverse training according to Embodiment 1 of the present invention.

[0037] Figure 6 This is a curve chart showing how the loss function value and accuracy of Example 1 of the present invention change with the number of training iterations.

[0038] Figure 7 The figure is a graph showing the change in the loss function value and accuracy of comparative example 1 of the present invention with the number of training iterations. DETAILED DESCRIPTION

[0039] The present invention will be further described below in conjunction with the accompanying drawings.

[0040] Example 1

[0041] like Figure 1 As shown, a motor imagery decoding method based on multi-subject data merging adversarial training includes the following steps:

[0042] Step 1: EEG (electroencephalogram) signal data preprocessing.

[0043] The BCI Competition IV 2a dataset was used to train the motor imagery classification model for verification and testing. The motor imagery classification model consists of three modules: feature extraction model, motor imagery classifier, and identity recognition classifier. The sampling electrode distribution of the dataset is as follows: Figure 2 As shown in the figure, Figure (a) is the electrode device that meets the international 10-20 standard, and Figure (b) is the position of the three channels of EOG (electrooculogram). The dataset consists of EEG data of 9 subjects. During the experiment, the subjects were required to perform four different motor imagery tasks, namely left hand (category 1), right hand (category 2), feet (category 3) and tongue (category 4) motor imagery. Each subject conducted two sessions of experiments on different dates, namely Session 1 and Session 2. Each session can be subdivided into 288 trials. The process of each test is as follows: at the beginning of the test (t = 0s), a fixed cross is displayed on the black screen, and the subject is prompted; two seconds later (t = 2s), an arrow with a duration of 1.25s appears on the screen, pointing to the left, right, down or up, corresponding to four different imagination tasks, namely left hand movement, right hand movement, foot movement and tongue movement, and the subject needs to imagine the movement corresponding to the arrow; each subject needs to complete the imagination task until the cross on the screen disappears (t = 6s); after a short break, the subject takes the test again, and the average time for each test is about 8 seconds. The data of multiple subjects are marked with identity tags to form double-label data of motion imagination tags and identity tags, and a data set is established.

[0044] The data set has 25 channels, of which 22 channels are EEG data and 3 channels are EOG data. The data of the three EOG channels are not included in the classification. The data set samples the signal at 250Hz (250 samples per second) and uses a bandpass filter to filter the signal from 0.5Hz to 100Hz.

[0045] First, delete the three EOG data in the data, then slice the remaining EEG data according to the trial and add identity labels (9 subjects, labels are 0-8). Cut the sample from the beginning of the motor imagery prompt to the end of the motor imagery in each trial, that is, take the data length of 4s (1000 sampling points) from 2s to 6s during the test, and downsample to 100Hz (400 sampling points). Figure 3 As shown, the processed data set is divided into Session 1 and Session 2; all the data in Session 1 are merged and shuffled, 20% of the data in Session 1 is taken as the validation set, and the remaining 80% is taken as the training set; Session 2 is used as the test set.

[0046] Step 2: Construct an EEG signal feature extraction model.

[0047] The convolutional neural network EEGNet-8,2 is used as the feature extraction model. The feature extraction model includes a temporal convolution layer, a spatial convolution layer, and a depthwise separable convolution layer, which can reduce the dimension and extract the features of the EEG signal. A BN batch normalization layer is set after the temporal convolution layer, and a BN batch normalization layer, an ELU activation function, a pooling layer, and a Dropout layer are set after the spatial convolution layer and the depthwise separable convolution layer. The output features of the feature extraction model are expanded through the Flatten layer. The number of convolution kernels F1 of the temporal convolution layer is 8, and the size is (1, 64); the number of convolution kernels of the spatial convolution layer is 16, and the size is (number of EEG channels, 1); the number of convolution kernels F2 of the depthwise separable convolution layer is 16, and the size is (1, 16); the size of the pooling layer after the spatial convolution layer is (1, 4); the size of the pooling layer after the depthwise separable convolution layer is (1, 8); the dropout parameter is set to 0.25.

[0048] Step 3: Build a multi-task classification module.

[0049] A motor imagery classifier and an identity recognition classifier are constructed respectively. Both the motor imagery classifier and the identity recognition classifier are one-layer fully connected layer structures. The input format of the classifier is the length of the EEGNet-8,2 output feature after expansion. The dimension of the motor imagery classifier output feature is equal to the number of motor imagery categories; the dimension of the identity recognition classifier output feature is equal to the number of subjects participating in the EEG signal data set collection; the classifier converts the input EEG features into classification probabilities through the Softmax activation function, and uses the cross entropy loss function to calculate the loss. In addition to calculating the cross entropy loss, the identity recognition classifier must also introduce the negative entropy loss function to calculate the negative entropy value as the loss during reverse training.

[0050] Step 4: Use cross entropy to forward train the model.

[0051] like Figure 4 As shown, the data in the training set is input into the feature extraction model to obtain the compressed and extracted EEG features, and then input into the motor imagery classifier and the identity recognition classifier respectively, and the cross entropy loss of the motor imagery classification and the identity recognition classification is calculated. The cross entropy loss obtained by the motor imagery classifier is back-propagated, the gradient is calculated, and the parameters of the motor imagery classifier and the feature extraction model are updated. After the parameters are updated, the gradient is reset.

[0052] The features extracted by the trained motor imagery classification model are still highly separable in the identity recognition task and can achieve a high accuracy rate of identity recognition. The features extracted by the feature extraction model in the training set vary greatly among subjects, which makes it easy for the identity recognition classifier to distinguish the identity of the subject, that is, the identity information is coupled in the motor imagery features. While updating the parameters of the feature extraction model and the motor imagery classifier, it is necessary to train the identity recognition classifier to recognize the features extracted by the feature extraction model, back-propagate the cross entropy loss obtained by the identity recognition classifier, calculate the gradient, and update the parameters of the identity recognition classifier. After the parameters are updated, the gradient is reset.

[0053] Each completion of the above process is considered as the completion of an iteration of forward training. The iteration of forward training is the iteration of the motor imagery classification model that is not affected by any other factors, and the identity recognition classifier is used to evaluate the amount of identity information contained in the EEG features obtained by the input feature extraction model of the current batch training set.

[0054] Step 5: Introduce the negative entropy reverse training model.

[0055] like Figure 5 As shown in the figure, a reverse training iteration is performed after the forward training iteration. The EEG features obtained by the feature extraction model are input into the motor imagery classifier and the identity recognition classifier, and the respective classification probabilities are output. The cross entropy of the motor imagery classification and the negative entropy of the identity recognition classification are calculated. Negative entropy is used as a measure of the difference between subjects or the amount of identity information. The more uncertain the identity recognition classification result is, the more average the output classification probability is, and the smaller the calculated negative entropy value is. The negative entropy value is multiplied by a proportional coefficient α as a loss function, and then added to the cross entropy obtained by the motor imagery classifier for back propagation, the gradient is calculated, and the parameters of the motor imagery classifier and the feature extraction model are updated. The gradient is reset after the parameters are updated. Since negative entropy is used as a loss function, it is more difficult for the identity recognition classifier to classify the EEG features extracted by the updated feature extraction model. The operation of minimizing the negative entropy of the identity classifier and the operation of minimizing the cross entropy are mutually antagonistic, thereby achieving the desired adversarial training effect.

[0056] During the reverse training iteration, the identity recognition classifier will participate in the back propagation of the model with the evaluated "amount of identity information" in the form of negative entropy to affect the learning ability of the feature extraction model, reduce the feature extraction model's attention to identity-related features, and focus more on extracting relevant features of the current motor imagination task, thereby achieving the effect of decoupling identity information.

[0057] In this embodiment, the value of the proportionality coefficient α is 0.5.

[0058] Step 6: Select the final motor imagery classification model

[0059] A complete adversarial training iteration includes the forward training iteration of step 4 and the reverse training iteration of step 5. The motor imagery classification model is subjected to multiple adversarial training iterations using batch training. A training set of a certain batch size is input into the motor imagery classification model. The motor imagery classification model is forward propagated, loss is calculated, back-propagated, and parameter updates are performed. The motor imagery classification model obtained after each complete adversarial training is verified through the validation set, and the motor imagery classification model with the highest accuracy is selected as the final motor imagery classification model.

[0060] During the adversarial training iteration process, the learning rate of each module is monitored separately and adjusted dynamically. The initial learning rate of each module is set to 0.001, the learning rate decay value is 0.6, the learning strategy is to trigger decay if the loss does not decrease for 50 consecutive times, the minimum learning rate is 0.00001, and early stopping is triggered if the loss of any module does not decrease for 500 consecutive times at the minimum learning rate.

[0061] Figure 6 The graph of the loss function value and accuracy rate changing with the number of training iterations. The vertical axis on the left is the loss function value, and the vertical axis on the right is the accuracy rate. LossMI is the cross entropy loss curve for motor imagery classification, LossID is the cross entropy loss curve for identity recognition classification, and ACC is the accuracy curve of the validation set during training. Figure 6 It can be seen that the LossID curve has been steadily rising since about 30 iterations. This is the optimization effect of introducing negative entropy loss for reverse training. The ACC curve rises with the LossID curve and has a good positive correlation, which shows the effectiveness of introducing negative entropy for reverse training.

[0062] Step 7: Test the final motor imagery classification model

[0063] The final motor imagery classification model selected in step 6 was tested using the test set. The test results are shown in Table 1. The average accuracy and average F1 score of the final motor imagery classification model are both greater than 75%.

[0064] Table 1 Experimental results of the motor imagery classification model of Example 1 tested with multiple subject data

[0065] Test No. Acuracy F1-score (F1 score%) 1 80.21 80.27 2 57.64 55.26 3 90.28 90.16 4 76.39 76.32 5 73.61 73.32 6 64.93 64.74 7 80.56 80.52 8 86.11 86.14 9 73.61 73.79 Mean ± SD % 75.93±9.52 75.61±10.05

[0066] Step 8: Collect the EEG signals of the subject and input the obtained EEG signals into the final motor imagery classification model to obtain the subject's motor imagery type. If the subject is not one of the subjects in step 1, collect EEG signals of the subject performing the motor imagery task for at least 2 periods of 144 signals each, for a total of 288 data samples, and add the obtained EEG signals to the EEG signal data set; re-execute steps 4, 5 and 6 based on the updated EEG signal data set; and use the obtained motor imagery classification model to identify the subject's motor imagery.

[0067] Comparative Example 1:

[0068] The difference between this comparative example and Example 1 is that the method of training the motor imagery classification model with multiple subject data is different.

[0069] This comparative example uses multi-subject data to perform forward training iterations on the motor imagery classification model. For the sake of comparison, the identity recognition classifier is still added during training for identity classification, but the identity recognition classifier only updates its own parameters in each iteration and will not affect the motor imagery recognition model. The loss function value and accuracy rate change with the number of training iterations are shown in Figure 2. Figure 7 As shown in Figure 2, as the LossMI curve gradually decreases, the LossID curve also shows a clear downward trend and drops to a lower value. The results of testing the selected motor imagery classification model using the test set are shown in Table 2. The measured average accuracy and average F1 score are both less than 70%, and the variance is large.

[0070] Comparative Example 1 verifies that in the motor imagery recognition model, the extracted motor imagery features have strong separability for identity recognition tasks, and there is a lot of identity information mixed in the features, which further illustrates the necessity of decoupling identity information when merging multi-subject data for training.

[0071] Table 2 Experimental results of the motor imagery classification model of Example 1 tested with multiple subjects

[0072] Test No. Acuracy F1-score (F1 score%) 1 78.12 77.95 2 53.82 53.52 3 85.42 85.45 4 68.40 68.04 5 37.50 31.87 6 56.60 56.46 7 78.82 78.85 8 78.82 78.94 9 63.89 64.53 Mean ± SD % 66.82±14.55 66.18±15.89

[0073] Comparative Example 2:

[0074] The difference between this comparative example and Example 1 is that the data set division of the training model is different.

[0075] This comparative example uses only one subject's data to train the motor imagery classification model, and uses the subject's test set to test the final motor imagery classification model, i.e., a one-to-one model for subjects. A total of 9 tests were conducted on 9 subjects. The test results are shown in Table 3, and the average accuracy measured is less than 74%. Compared with comparative example 2, the average accuracy of Example 1 is improved by 2.78%. At the same time, there is insufficient data for training the motor imagery classification model using the method of comparative example 2. Comparative example 2 further illustrates the effectiveness of Example 1.

[0076] Test No. Acuracy 1 79.51 2 61.11 3 88.54 4 71.53 5 71.18 6 59.03 7 71.53 8 80.56 9 75.35 Mean ± SD % 73.15±9.29

Claims

1. A motor imagery decoding method based on multi-subject data merging adversarial training, characterized in that: The following steps are involved: Step 1, collecting EEG signals of multiple subjects performing motor imagery tasks to obtain an EEG signal data set; After preprocessing the collected data set, it is divided into a training set and a validation set; Step 2: Build a motor imagery classification model The motor imagery classification model includes three modules: a feature extraction model, a motor imagery classifier, and an identity recognition classifier; the feature extraction model adopts a convolutional neural network; the output features of the feature extraction model are expanded through a Flatten layer and then input into the motor imagery classifier and the identity recognition classifier respectively; the motor imagery classifier and the identity recognition classifier both adopt a fully connected layer structure; Step 3, using the training set to alternately perform forward training and reverse training on the motor imagery classification model; The process of forward training is as follows: Input the training set into the motor imagery classification model to obtain the classification probabilities of motor imagery and identity recognition; calculate the cross entropy of the two classifiers according to the two classification probabilities, and update the parameters of the feature extraction model and the two classifiers; wherein the cross entropy of the motor imagery classifier updates the parameters of the motor imagery classifier and the feature extraction model; and the cross entropy of the identity recognition classifier updates the parameters of the identity recognition classifier; The process of reverse training is as follows: Input the training set into the motor imagery classification model to obtain the classification probabilities of motor imagery and identity recognition; calculate the cross entropy of the motor imagery classifier and the negative entropy of the identity recognition classifier according to the two classification probabilities; and update the parameters of the motor imagery classifier and the feature extraction model jointly through the obtained cross entropy and negative entropy; Step 4, verify the motor imagery classification model obtained by each reverse training through the verification set, and select the motor imagery classification model with the highest accuracy as the final motor imagery classification model; Step 5: Collect the subject's EEG signal, and input the obtained EEG signal into the final motor imagery classification model to obtain the subject's motor imagery type.

2. The method for motor imagery decoding based on multi-subject data merging adversarial training according to claim 1, characterized in that: In step 3, the parameters of the motor imagery classifier and the feature extraction model are updated by multiplying the negative entropy of the identity recognition classifier by the proportional coefficient α and adding it to the cross entropy of the motor imagery classifier.

3. The method for decoding motor imagery based on multi-subject data merging adversarial training according to claim 2, characterized in that: The proportionality factor α The value of is 0.

5.

4. The method for motor imagery decoding based on multi-subject data merging adversarial training according to claim 1, characterized in that: If the subject is not one of the subjects in step 1, the EEG signals of the subject performing the motor imagery task are collected, and the obtained EEG signals are added to the EEG signal dataset; steps 2, 3 and 4 are re-executed based on the updated EEG signal dataset; and the obtained motor imagery classification model is used to identify the subject's motor imagery.

5. The method for motor imagery decoding based on multi-subject data merging adversarial training according to claim 1, characterized in that: The feature extraction model includes a temporal convolution layer, a spatial convolution layer and a depth-separable convolution layer; a BN batch normalization layer is arranged after the temporal convolution layer; a BN batch normalization layer, an ELU activation function, a pooling layer and a Dropout layer are arranged after the spatial convolution layer and the depth-separable convolution layer; a Softmax activation function for outputting classification probability is arranged after the motor imagery classifier and the identity recognition classifier.

6. The method for motor imagery decoding based on multi-subject data merging adversarial training according to claim 1, characterized in that: The number of convolution kernels F1 of the temporal convolution layer is 8, and the size is (1, 64); the number of convolution kernels of the spatial convolution layer is 16, and the size is (number of EEG channels, 1); the number of convolution kernels F2 of the depthwise separable convolution layer is 16, and the size is (1, 16); the size of the pooling layer after the spatial convolution layer is (1, 4); the size of the pooling layer after the depthwise separable convolution layer is (1, 8); the dropout parameters are all set to 0.

25.

7. The method for motor imagery decoding based on multi-subject data merging adversarial training according to claim 1, characterized in that: In step 1, the process of preprocessing the data set is as follows: (1) Bandpass filter the data in the range of 0.5 Hz to 100 Hz; (2) Downsample the EEG data to 100 Hz.

8. The method for motor imagery decoding based on multi-subject data merging adversarial training according to claim 1, characterized in that: In step 1, there are four types of motor imagery, namely left hand, right hand, feet and tongue motor imagery.

9. The method for motor imagery decoding based on multi-subject data merging adversarial training according to claim 1, characterized in that: In the step 2, the dimension of the output feature of the motor imagery classifier is equal to the number of motor imagery categories; the dimension of the output feature of the identity recognition classifier is equal to the number of subjects participating in the collection of the EEG signal data set.

10. The method for motor imagery decoding based on multi-subject data merging adversarial training according to claim 1, characterized in that: In the step 5, the learning rate of each module is monitored separately and adjusted dynamically; the initial learning rate of each module is set to 0.001, the learning rate decay value is 0.6, the learning strategy is to trigger decay if the loss does not decrease for 50 consecutive times, the minimum learning rate is 0.00001, and early stopping is triggered if the loss of any module does not decrease for 500 consecutive times when it is at the minimum learning rate.

Images