A motor imagery task classification method combining meta-learning and deep learning

Abstract

The application discloses a motor imagery task classification method combining meta-learning and deep learning. First, the motor imagery electroencephalogram (EEG) of a subject is acquired and preprocessed. The preprocessed EEG is subjected to continuous wavelet transform to obtain an EEG time-frequency energy graph. Then, a motor imagery task classification model is constructed, and a large number of training samples are used to train the motor imagery task classification model by combining meta-learning to obtain a meta-learner. Finally, a small number of training samples are input into the meta-learner to obtain initial weight parameters of the motor imagery task classification model, and the model is initialized. The EEG time-frequency energy graph of a motor imagery task to be classified is input into the initialized motor imagery task classification model to classify the motor imagery task. The motor imagery task classification model is trained by combining meta-learning, so that the model has strong generalization ability, can quickly adapt to the differences between new tasks and individual EEG, and has good classification effect and high classification accuracy on small sample data.

Description

Technical Field

[0001] This invention relates to the field of motor imagery EEG signal decoding technology, specifically a motor imagery task classification method that combines meta-learning and deep learning. Background Technology

[0002] Brain-computer interface (BCI) is an emerging interaction method that primarily converts neural signals from the brain into control signals to drive external devices such as computers, artificial limbs, and wheelchairs. BCI technology is widely used in medicine, education, entertainment, and smart homes. The EEG signal paradigm of motor imagery is the most extensively studied. Motor imagery-based BCIs provide an interface for patients with movement disorders or those in a locked-down state. When a patient imagines moving their limbs (such as waving their hand or moving their feet) without actual movement output, specific brain regions will show varying degrees of activation. By analyzing EEG signals and detecting and identifying the activation effects of different brain regions, the user's intention can be determined, enabling communication and control between the human brain and external devices.

[0003] Deep learning technology is widely used in various classification tasks. Deep learning-based classification models often require a large amount of sample data for training, and their classification accuracy depends on the number of training samples. Furthermore, the model only shows good classification performance when the data is similar to the training samples, resulting in poor generalization performance. However, motor imagery EEG signals are sparse, with a limited number of samples. Training a model on a small sample base will not yield good classification results. In addition, there are slight differences in EEG signals between individuals, so each individual's motor imagery task is a new task for the model. Therefore, the model needs to be able to quickly adapt to new tasks and have good adaptability and high classification accuracy for different motor imagery tasks for different individuals. Therefore, this application proposes a motor imagery task classification method that combines meta-learning and deep learning. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is to provide a method for classifying motion imagination tasks that combines meta-learning and deep learning.

[0005] The present invention solves the aforementioned technical problem by adopting the following technical solution:

[0006] A method for classifying motion imagery tasks that combines meta-learning and deep learning, characterized by the following steps:

[0007] Step 1: Acquire the EEG signals of the subject's motor imagery and preprocess them;

[0008] Step 2: Perform continuous wavelet transform on the preprocessed EEG signal to obtain the time-frequency energy map of the EEG signal;

[0009] Step 3: Construct a classification model for the motion imagery task, and train the model using a large number of training samples using meta-learning to obtain a meta-learner;

[0010] Step 4: Input a small number of training samples into the meta-learner to obtain the initial weight parameters of the motor imagery task classification model and initialize the model; input the EEG signal time-frequency energy map of the motor imagery task to be classified into the initialized motor imagery task classification model to classify the motor imagery task.

[0011] Furthermore, in step 3, the loss optimization equation for the meta-learner is:

[0012]

[0013] In the formula: φ is the initial weight parameter of the motor imagery task classification model, U τ,A (·) is the update operator, L τ,B (·) represents the loss of the motion imagination task τ. Let A and B be the expected function of the loss, and let A and B both represent datasets.

[0014] Furthermore, in step 1, the preprocessing includes channel selection, filtering, baseline correction, artifact removal, and rereference.

[0015] Furthermore, the motor imagery task paradigm is divided into four phases: 0s-2s voice prompt "Ready to begin"; 2s-4s picture prompts and voice prompts, with the prompts indicating whether the subject is using the left or right hand; 2.5s-4s the subject performs motor imagery, with the imagery content matching the picture and voice prompts, and EEG signals are collected; 4s-5s rest.

[0016] Furthermore, the motion imagery task classification model employs the EfficientNet network.

[0017] Compared with the prior art, the beneficial effects of the present invention are:

[0018] 1. Meta-learning refers to learning on many different tasks and then applying the learned knowledge to new tasks. Meta-learners have the ability to adjust the weight parameters of classification models. Combining meta-learning with the training of a motor imagery task classification model gives the model strong generalization ability, enabling it to quickly adapt to new tasks and perform well even with a small number of samples. This reduces sample dependence and makes full use of limited training data to improve classification accuracy. The motor imagery task classification model can quickly adapt to new motor imagery EEG signal samples, accommodate individual differences, and achieve good classification results for different samples.

[0019] 2. A classification model for motor imagery tasks was constructed based on the EfficientNet network. The EfficientNet network is an efficient deep learning network structure. By balancing the depth, width and resolution of the network, it helps to enhance noise suppression and feature extraction for very weak signals such as EEG signals that are easily affected by noise, thereby obtaining clearer and more accurate signal features and further improving classification efficiency and accuracy.

[0020] 3. Deep learning, especially networks with convolutional layers such as EfficientNet, has shown excellent noise suppression and feature extraction capabilities in fields such as image and speech processing.

[0021] 4. Improve system usability: The high accuracy and strong generalization ability of this method make the classification results of motor imagery tasks more reliable and stable, which is very important for the practical application of brain-computer interface technology and can improve the usability and user experience of brain-computer interface systems. Attached Figure Description

[0022] Figure 1 This is an overall flowchart of the present invention;

[0023] Figure 2 Here is a diagram of the EfficientNet network structure;

[0024] Figure 3 The classification accuracy of different models for different sample data. Detailed Implementation

[0025] Specific embodiments are given below with reference to the accompanying drawings. These specific embodiments are only used to describe the technical solution of the present invention in detail and are not intended to limit the scope of protection of this application.

[0026] This invention is a method for classifying motor imagery tasks that combines meta-learning and deep learning (hereinafter referred to as the method, see [link]). Figures 1-3 The process includes the following steps:

[0027] Step 1: Acquire the subject's motor imagery EEG signals and preprocess them to obtain preprocessed EEG signals; preprocessing includes channel selection, filtering, baseline correction, artifact removal, and rereference;

[0028] Step 2: Perform continuous wavelet transform (CWT) on the preprocessed EEG signal to obtain the EEG signal time-frequency energy map; several EEG signal time-frequency energy maps form a dataset. In this embodiment, the size of the EEG signal time-frequency energy map is 96×96 pixels and contains 3 channels; divide the dataset into training set A, test set B and validation set C;

[0029] The continuous wavelet transform uses the Morlet wavelet, and the wavelet function is:

[0030]

[0031] In the formula: W(a,b) are wavelet transform coefficients, x(t) is the preprocessed EEG signal, ψ(·) is the wavelet mother function, a is the scale parameter, and b is the location parameter;

[0032] The basis functions of the Morlet wavelet are:

[0033] ψ(t)=exp(iω0t)exp(-t 2 / twenty two)

[0034] In the formula: ω0 is the center frequency;

[0035] Step 3: Construct a motion imagery task classification model based on the EfficientNet network, and train the motion imagery task classification model by combining meta-learning to obtain the meta-learner;

[0036] The EfficientNet network mainly consists of Conv and MBConv modules. The input EEG signal time-frequency energy map is processed by the Conv and MBConv modules, and then passed through pooling layers and an output layer to obtain the classification results for the motor imagery task. The MBConv module includes two operations: squeezing and excitation. The squeezing operation formula is as follows:

[0037]

[0038] In the formula: Z represents the result of the extrusion operation, F sq (·) represents the squeezing function, and F and W represent the height and width of the feature map;

[0039] The compression formula is:

[0040] S = F ex (Z,W)=σ(g(Z,W))=σ(W2δ(W1Z)) (4)

[0041] In the formula: S is the result of the compression operation, F ex (·) is the activation function, σ(·) is the Sigmoid activation function, g(·) is the gating function, δ(·) is the ReLU activation function, and W1 and W2 are learnable parameters.

[0042] Meta-learning employs the MAML algorithm, which consists of inner-loop optimization and outer-loop optimization. Inner-loop optimization trains the meta-learner, while outer-loop optimization trains the motion imagery task classification model. Since different learning tasks correspond to a set of initial weight parameters for the motion imagery task classification model, the motion imagery tasks in the training set are used as the learning tasks for the meta-learner. The meta-learning task generator randomly samples a learning task from the training set. The meta-learner is iteratively updated to reduce the loss when learning motion imagery tasks. The loss optimization equation is:

[0043]

[0044] In the formula: φ is the initial weight parameter of the motor imagery task classification model, U τ (·) is the update operator, L τ (·) represents the loss of the motion imagination task τ. Let be the expected function of the loss;

[0045] The motion imagery task classification model is initialized based on the initial weight parameters learned by the meta-learner. The meta-learner is trained using test set B. The meta-learner learns and infers how to adjust the model's weight parameters for different learning tasks by taking the motion imagery task as input and the weight parameters of the motion imagery task classification model as output. To improve the generalization ability of the meta-learner, the model weight parameters are initialized using training set A, and the loss is calculated using test set B. The loss optimization equation is:

[0046]

[0047] To further optimize the meta-learner, stochastic gradient descent is used to update the loss until training ends, thus obtaining the meta-learner.

[0048] Step 4: Input the validation set C into the meta-learner, output the initial weight parameters of the motion imagery task classification model, and initialize the model; input the validation set C into the initialized motion imagery task classification model, and output the classification result.

[0049] Example 1

[0050] 3. This embodiment uses the publicly available BCI IV-2a dataset for motor imagery. This dataset contains 9 participants, each of whom performed two sessions on different days. Each session had 6 runs, and each run had 48 trials (12 for left hand, 12 for right hand, 12 for both feet, and 12 for tongue). At the beginning of each session, a baseline signal was recorded for 5 minutes. The recording was divided into 3 parts: the first part was a 2-minute recording of fixation on the cross, the second part was a 1-minute recording with eyes closed, and the third part was a 1-minute recording of eye movements. Experimental paradigm: 0-2 seconds of cross imagery, followed by 1.25 seconds of left, right, down, and up arrow cues, corresponding to the imagined categories of left hand, right hand, both feet, and tongue, respectively; 3-6 seconds of motor imagery; 6 seconds of cross imagery disappearance followed by a short rest.

[0051] A total of 25 electrodes were used to collect EEG signals, including 23 EEG channels and 3 EOG channels. The acquisition frequency was 250 Hz, with a bandpass filter of 0.5-100 Hz. Electrode signals from channels C3, C4, FP5, and FP6 were selected as the raw EEG signals. This signal processing was based on the EfficientNet-B3 model and implemented in Python using the PyTorch DL framework. Training was accelerated using four GTX 2080Ti GPUs.

[0052] Using data from the ninth participant as test data, the classification accuracy reached 91%, indicating that the method has good generalization effect on small samples.

[0053] Example 2

[0054] This embodiment selected four subjects: two healthy men aged 30 and 60, and two healthy women aged 30 and 60. None of the four subjects had experience with EEG.

[0055] Experimental preparation: Before starting the experiment, obtain informed consent from the participants and ensure that they fully understand the purpose and process of the experiment; ensure that the experimental environment is quiet and comfortable to reduce the impact of external interference on the participants.

[0056] Prepare an EEG acquisition system, using the NeuSen W wireless digital EEG acquisition system, and wear it according to the international 10-20 positioning system.

[0057] Data collection paradigm: Before the experiment began, participants were asked to close their eyes, relax their muscles, and remain quiet. Each data collection session consisted of four phases: 0s-2s voice prompt "Ready to begin"; 2s-4s image and voice prompts, with the prompt being either the left or right hand; 2.5s-4s participants performed motor imagery, with the image and voice prompts matching the content of the image and voice prompts, and EEG signals were collected; 4s-5s rest; after each cycle of 24 times, participants rested for ten minutes, and each participant completed three rounds of data collection.

[0058] The EEG signals of each participant were processed, and three models—CNN, EEGNet, and MENet—were used to classify the motor imagery task for each participant. The classification accuracy statistics are shown in [link to statistics]. Figure 3 The MENet model refers to the motion imagery task classification model of this invention. From Figure 3 It can be seen that the motor imagery task classification model of the present invention achieved a high accuracy on the EEG data of each subject, indicating that the model can learn quickly across different tasks, has good generalization performance, and achieves better classification results under the same small sample conditions.

[0059] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A method for classifying motion imagery tasks that combines meta-learning and deep learning, characterized in that, The method includes the following steps: Step 1: Acquire the EEG signals of the subject's motor imagery and preprocess them; Step 2: Perform continuous wavelet transform on the preprocessed EEG signal to obtain the time-frequency energy map of the EEG signal; Step 3: Construct a motion imagery task classification model. Combine meta-learning with a large number of training samples to train the motion imagery task classification model and obtain a meta-learner. Initialize the motion imagery task classification model according to the initial weight parameters learned by the meta-learner. Train the meta-learner using the test set. The meta-learner learns the model's weight parameters by taking the motion imagery task as input and the weight parameters of the motion imagery task classification model as output. The motion imagery task classification model adopts the EfficientNet network. Step 4: Input a small number of training samples into the meta-learner to obtain the initial weight parameters of the motor imagery task classification model and initialize the model; input the EEG signal time-frequency energy map of the motor imagery task to be classified into the initialized motor imagery task classification model to classify the motor imagery task.

2. The method for classifying motion imagery tasks combining meta-learning and deep learning according to claim 1, characterized in that, In step 3, the loss optimization equation for the meta-learner is: ， In the formula: The initial weight parameters for the motion imagery task classification model. To update the operator, Imagine tasks for movement The loss, Let be the expected function of the loss. , Both refer to datasets.

3. The method for classifying motion imagery tasks combining meta-learning and deep learning according to claim 1 or 2, characterized in that, In step 1, preprocessing includes channel selection, filtering, baseline correction, artifact removal, and rereference.

4. The method for classifying motion imagery tasks combining meta-learning and deep learning according to claim 1, characterized in that, The motor imagery task paradigm consists of four phases: 0s-2s voice prompt "Ready to begin"; 2s-4s picture and voice prompts, with the prompts indicating whether the subject is using the left or right hand; 2.5s-4s the subject performs motor imagery, with the image and voice prompts matching the content of the image and voice prompts, and EEG signals are collected; 4s-5s rest.

Images