Motor imagery brain-computer interface self-adaptive classification method based on confrontation automatic encoder
By extracting common features of EEG signals in the motor imagination brain-computer interface, the problem of insufficient accuracy caused by non-stationarity of EEG signals in traditional methods is solved, and more efficient feature extraction and classification is achieved, thereby improving the Control accuracy of brain-computer interface.
Patent Information
- Application Number
- CN202410960944.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-29
AI Technical Summary
Traditional sports imagination brain-computer interfaces cannot effectively deal with the non-stationarity of EEG signals during feedback training, resulting in insufficient feature extraction and classification accuracy.
The feedback training method based on adversarial autoencoder is adopted, and common features are extracted from the source domain and the target domain through adversarial autoencoder, the electrocerebral decoding model is iteratively updated, and feature extraction and classification are optimized using multi-objective loss function.
It improves the robustness and versatility of feature extraction of EEG signals, enhances the quality of training data, and realizes efficient motion imagination brain-computer interface control.
Smart Images

Figure CN119046779A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of brain-computer interface, and in particular relates to a motor imagery brain-computer interface adaptive classification method based on an adversarial autoencoder. Background Art
[0002] In recent years, brain-computer interface (BCI) technology has gradually attracted widespread attention. Among them, a common non-invasive method is to use motor imagery (MI) to stimulate EEG signals. Motor imagery refers to the cognitive activity generated when people imagine the movement of a part of the body without actually moving this part. Different motor imagery tasks will produce different EEG signals in different areas of the brain, and different instructions can be obtained by decoding these signals. Since motor imagery EEG signals can be generated spontaneously and do not require external stimulation, they have the advantages of intuitiveness and comfort.
[0003] In order to accurately decode these EEG signals, efficient feature extraction and classification methods are required. However, since EEG signals are non-stationary, that is, their features change over time, adaptive feature extraction and classification algorithms are required. Traditional adaptive techniques, such as covariate shift estimation, can adaptively learn EEG features at different times, but they are limited in high-dimensional feature learning. In addition, the extracted features are not necessarily the most suitable for subsequent classifiers. Therefore, end-to-end adaptive deep learning technology is currently becoming mainstream. Among them, adversarial learning technology, especially the Adversarial Auto-Encoder (AAE) model, has been shown to have superior performance in processing non-stationary signals. At present, there is no research on the application of adversarial auto-encoders in feedback training. Summary of the invention
[0004] In order to overcome the shortcomings of the prior art, the present invention provides a motor imagery brain-computer interface adaptive classification method based on an adversarial auto-encoder, which solves the non-stationary problem of EEG signals through the adversarial auto-encoder (AAE) technology, and improves the classification accuracy of motor imagery EEG signals across time periods. The present invention first uses a public data set to pre-train the initial model of motor imagery EEG decoding, and then performs feedback training in an online experiment, treating the newly collected data as the target domain, and the original decoding model training data as the source domain, extracting common features through the adversarial auto-encoder, and iteratively updating the EEG decoding model. This method not only improves the robustness and versatility of feature extraction, but also can enhance the training data by generating EEG signals, and finally realizes efficient motor imagery brain-computer interface control. The present invention can be applied to brain-computer interface systems, helping in the fields of medical treatment and rehabilitation.
[0005] The technical solution adopted by the present invention to solve the technical problem is as follows:
[0006] Step 1: Use a public dataset to pre-train the initial motor imagery EEG signal decoding model;
[0007] Step 2: Start feedback training of online experiments and collect new EEG signal data;
[0008] Step 3: Define the newly collected EEG signal data as the target domain, and define the training data of the initial motor imagery EEG signal decoding model as the source domain;
[0009] Step 4: Extract common features from the source and target domains through the adversarial autoencoder and iteratively train the motor imagery EEG signal decoding model;
[0010] Step 5: Repeat steps 2 to 4 until motor imagery brain-computer interface control is achieved.
[0011] Preferably, the initial motor imagery EEG signal decoding model is composed of a variational autoencoder and a deep neural network; its composition method is that the hidden representation layer of the variational autoencoder is simultaneously input into the deep neural network and the decoder of the variational autoencoder, the deep neural network is used for classification, and the decoder of the variational autoencoder is used to reconstruct the EEG signal.
[0012] Preferably, the common features extracted from the source domain and the target domain by the adversarial autoencoder are stationary features in the non-stationary signal.
[0013] Preferably, during the pre-training process of the initial motor imagery EEG signal decoding model, the initial motor imagery EEG signal decoding model adopts a multi-objective loss function, and the loss function formula is:
[0014] L VAE =L ae +λL class (1)
[0015] Among them, L ae is the reconstruction error of the variational autoencoder, L class is the classification error, and λ is the proportional coefficient of the loss function.
[0016] Preferably, in step 4, in the iterative training of the motor imagery EEG signal decoding model, a multi-objective loss function is used, and its formula is:
[0017] L AAE =L ae +λ1L class +λ2L adversarial (2)
[0018] Among them, Ladversarial is the loss term for adversarial learning, and λ1 and λ2 are both proportional coefficients.
[0019] The beneficial effects of the present invention are as follows:
[0020] 1. The adversarial autoencoder determines the source of the feature through a discriminator and optimizes the feature extraction network to obtain more robust and general signal features.
[0021] 2. While training the feature extraction network, the adversarial autoencoder can also generate EEG signals to enhance the training data and apply them in subsequent classifier training.
[0022] 3. Compared with ordinary autoencoders, adversarial autoencoders are based on variational autoencoders, whose hidden representation is continuous and thus interpretable. This interpretable hidden representation can be further used as visual feedback to improve the feedback training effect.
[0023] In summary, the feedback training method based on adversarial autoencoder can solve the problem of non-stationarity of EEG signals and improve the efficiency of feedback training. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 It is a schematic diagram of the framework of the method of the present invention.
[0025] Figure 2 This is a schematic diagram of the initial motor imagery EEG signal decoding model pre-trained in the present invention.
[0026] Figure 3 This is a schematic diagram of the iterative training motor imagery EEG signal decoding model of the present invention.
[0027] Figure 4 This is the structure of the convolutional encoder of the variational autoencoder of the present invention.
[0028] Figure 5 This is a feedback training experiment setup for an embodiment of the present invention. DETAILED DESCRIPTION
[0029] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0030] The present invention aims to solve the following problem: the traditional motor imagery brain-computer interface cannot effectively deal with the non-stationarity of the subject's EEG signal in the feedback training method. In order to overcome this problem, the present invention introduces a feedback training method based on an adversarial autoencoder.
[0031] The method first uses an existing public dataset to pre-train the initial model for motor imagery EEG signal decoding. Subsequently, feedback training for online experiments is initiated. Once a round of feedback training for online experiments is completed, the newly collected data is considered the target domain, and the training set data of the original decoding model is considered the source domain. By training the adversarial autoencoder, common features are extracted from the source domain and the target domain. These features are used to iteratively update the EEG signal decoding model. This process continues until the operator can efficiently control the motor imagery brain-computer interface. The framework diagram is shown in the figure. Figure 1 The specific content of the invention is as follows:
[0032] (1) Apply the adversarial autoencoder to the feedback training process. Figure 2 As shown in FIG. 1 , in the present invention, an initial motor imagery EEG decoding model consisting of a variational auto-encoder (VAE) and a deep neural network (DNN) is first pre-trained, and the EEG decoding model is used for classification in the first round of feedback training online experiments. After a round of feedback training online experiments is completed, Figure 3 As shown, iterative training of the EEG decoding algorithm model is performed. The newly collected data is defined as the target domain, and the training set data of the original EEG signal decoding model is defined as the source domain. The adversarial autoencoder is used to extract the stationary features in the non-stationary signals from the source domain and the target domain, and then the EEG signal decoding model is iteratively updated.
[0033] During the pre-training process of the initial model, the initial motor imagery EEG decoding model adopts a multi-objective loss function, and the loss function formula is:
[0034] L VAE =L ae +λL class (1)
[0035] Among them, L ae is the reconstruction error of the variational autoencoder, L class is the classification error, and λ is the proportional coefficient of the loss function.
[0036] In the training of the iterative model, the iterative motor imagery EEG decoding model also adopts a multi-objective loss function, and its formula is:
[0037] L AAE =L ae +λ1L class +λ2L adversarial (2)
[0038] Compared with the pre-trained multi-objective loss function, the loss function of iterative training adds L adversarial term, as the loss term for adversarial learning.
[0039] Example:
[0040] In an embodiment of the present invention, the BCI Competition IV 2a data set is used as a training set for the pre-trained initial motor imagery EEG decoding model. The data lead positions are arranged according to a 10-20 system, with a total of 22 EEG signal leads, a sampling rate of 250Hz, a reference electrode for the left ear mastoid, and a ground electrode for the right ear mastoid. In the implementation case, the EEGLAB toolkit of MATLAB is used to preprocess the raw data, including three stages of artifact removal, re-reference, and band-pass filtering. Specifically, the Independent Components Analysis (ICA) method is used to remove artifact components, and the number of independent components selected is 22, the same as the number of electrodes; the zero reference method in EEGLAB is used for re-reference; the frequency band of the bandpass filter is selected from 8-30Hz.
[0041] The BCI Competition IV 2a dataset was input into the initial motor imagery EEG decoding model. First, data slicing was performed to remove the resting and non-task EEG data in the dataset; the data was cut to obtain the valid task state data in each trial, and according to the different task states, the data segments of the same task state were spliced according to the left hand task, right hand task, both feet task and tongue task, and divided into data segments with a time length of 2 seconds.
[0042] The initial motor imagery EEG decoding model in the embodiment of the present invention is composed of a variational autoencoder and a multi-layer perceptron. The convolutional encoder of the variational autoencoder is used to extract features, and its specific structure is as follows: Figure 4As shown, it consists of a temporal filtering module, a spatial filtering module and a feature integration filtering module. Each filtering module contains a convolutional layer, a BatchNorm layer, a RELU activation function and an average pooling layer. The number of filters in the convolutional layers of the three filtering modules are 16, 32 and 64 respectively, and each pooling layer reduces the size of the feature map to 1 / 4 of the previous one. Finally, a convolutional layer with a filter number of 2 is passed to obtain the mean vector and the variance vector. For the mean vector μ and the variance vector δ, the potential feature representation is obtained according to z=μ+δ·ε, where ε is noise subject to a standard normal distribution. In this implementation case, z is a 64-dimensional vector. The potential feature representation is input into the decoder and the multilayer perceptron of the variational autoencoder respectively. The decoder is used to reconstruct EEG signals from the potential feature representation and use the reconstruction error to train the model. Its structure is opposite to that of the encoder and consists of a feature integration filter module, a spatial filter module and a temporal filter module. The multilayer perceptron classifies EEG signals under different task states and uses the classification error to train the network. The middle layer has 8 neurons and the output is 4 neurons.
[0043] The iterative motor imagery EEG signal decoding model in the embodiment of the present invention is composed of an adversarial autoencoder and a multi-layer perceptron. The difference from the initial motor imagery EEG decoding model is that the potential feature representation is input into the adversarial learning network in addition to the input decoder and the multi-layer perceptron. The adversarial learning network consists of a gradient reversal layer and a domain classifier. The domain classifier is used to determine whether the input potential feature is a source domain feature or a target domain feature. The target domain feature is a potential feature representation obtained by encoding the EEG data of the latest collected stage through a convolution encoder, and the source domain feature is a potential feature representation obtained by encoding the EEG data or pre-training data set collected in the previous stage through a convolution encoder. In the embodiment, the domain classifier is composed of 4 layers of fully connected layers, and the number of neurons in each layer is 32, 16, 4, and 2 respectively. The gradient reversal layer in the adversarial learning network is between the convolution encoder and the domain classifier. Its function is to make the iterative targets of the convolution encoder and the domain classifier opposite during the training process, so that the ability of the domain classifier to judge the source of the feature is enhanced, and the ability of the convolution encoder to extract the common features of the target domain and the source domain is enhanced, so as to achieve the purpose of adversarial learning. In the process of training the adversarial learning network, the parameters of the convolutional decoder and the multi-layer perceptron are frozen, and only the parameters of the convolutional encoder and the adversarial learning network are iterated until the convolutional encoder extracts the common features of the target domain and the source domain, making it difficult for the domain classifier to distinguish.
[0044] During the training or updating process of the motor imagery EEG decoding model, a 10-fold cross-validation model was used to validate the model, and finally the model with the highest accuracy in the validation set was selected as the model used in the online experiment.
[0045] After the initial motor imagery EEG decoding model is trained, the first phase of online experiments begins. Figure 5 As shown, each operator conducted three feedback training experiments within a week. Each training session consisted of two sessions with a 10-30 minute break between the two sessions. Each session contained four runs with a 1 minute break between each run. A total of 24 runs were performed, each with 30 trials. During the training, the operator sat in a comfortable chair and performed motor imagery tasks facing the monitor screen. The data acquisition method was consistent with the pre-training data set, that is, the lead positions were arranged in a 10-20 system, with a total of 22 EEG signal leads and a sampling rate of 250Hz.
[0046] Each trial lasts for 7.5 seconds. At 0-2 seconds, the left side of the screen prompts the operator to perform a motor imagery task. In the implementation case of the present invention, if the task is left-hand motor imagery, the left color block is displayed in red; at 2-6 seconds, real-time feedback on motor imagery is performed. In the implementation case of the present invention, the computer updates the position of the red ball every 0.05 seconds, and by processing and classifying the collected EEG signals of the first 2 seconds, the ball moves in the direction of the classification result, thereby providing feedback to the operator; rest at 6-7.5 seconds.
[0047] Except for the last stage, when all groups of a stage are completed, the collected data of this stage will be used as the input of the iterative motor imagery decoding model for training to keep the decoding model updated. The trained iterative motor imagery decoding model will be used for the next stage of online experiments. Before the EEG data is input, the EEGLAB toolkit of MATLAB is used to preprocess the raw data, including artifact removal, re-reference and bandpass filtering, and data slicing and data splicing are performed to divide it into data segments with a time length of 2 seconds, which is the same as the data processing method during pre-training.
Claims
1. A motor imagery brain-computer interface adaptive classification method based on adversarial autoencoder, characterized in that: The steps include: Step 1: Use a public dataset to pre-train the initial motor imagery EEG signal decoding model; Step 2: Start feedback training of online experiments and collect new EEG signal data; Step 3: Define the newly collected EEG signal data as the target domain, and define the training data of the initial motor imagery EEG signal decoding model as the source domain; Step 4: Extract common features from the source and target domains through the adversarial autoencoder and iteratively train the motor imagery EEG signal decoding model; Step 5: Repeat steps 2 to 4 until motor imagery brain-computer interface control is achieved.
2. The method for adaptive classification of motor imagery brain-computer interface based on adversarial autoencoder according to claim 1, characterized in that: The initial motor imagery EEG signal decoding model is composed of a variational autoencoder and a deep neural network; its composition method is that the hidden representation layer of the variational autoencoder is simultaneously input into the deep neural network and the decoder of the variational autoencoder, the deep neural network is used for classification, and the decoder of the variational autoencoder is used to reconstruct the EEG signal.
3. The method for adaptive classification of motor imagery brain-computer interface based on adversarial autoencoder according to claim 2, characterized in that: The common features extracted from the source domain and the target domain by the adversarial autoencoder are stationary features in the non-stationary signal.
4. The method for adaptive classification of motor imagery brain-computer interface based on adversarial autoencoder according to claim 3, characterized in that: During the pre-training process of the initial motor imagery EEG signal decoding model, the initial motor imagery EEG signal decoding model adopts a multi-objective loss function, and the loss function formula is: THE VAE =L ae +λL class (1) Among them, L ae is the reconstruction error of the variational autoencoder, L class is the classification error, and λ is the proportional coefficient of the loss function.
5. The method for adaptive classification of motor imagery brain-computer interface based on adversarial autoencoder according to claim 4, characterized in that: In step 4, a multi-objective loss function is used in the iterative training of the motor imagery EEG signal decoding model, and its formula is: L AAE =L ae +λ1L class +λ2L adversarial (2) Among them, L adversarial is the loss term for adversarial learning, and λ1 and λ2 are both proportional coefficients.
Citation Information
Cited By
Self-adaptive optimization method and system for online decoding of motor imagery brain-computer interface
CN121561684A
