Mask-based self-supervised and strongly supervised model for motor imagery eeg signal classification

Abstract

The application discloses a kind of based on mask self-supervision and strong supervision model motor imagination electroencephalogram signal classification method, data set formed by several finger motor imagination electroencephalogram signals from a subject is divided into multiple subsets according to batch size, after pre-processing to each subset, random mask is carried out, obtain the mask subset and mask matrix corresponding to each subset;Design neural network comprising mask self-supervision network and strong supervision deep learning classification network, use the mask subset and mask matrix corresponding to subset to train neural network, obtain neural network model after training is completed;When testing, test subset and corresponding mask subset are input into neural network model, and the classification probability of test subset is output;The advantage is that mask self-supervision network can learn the deep feature of finger motor imagination electroencephalogram signal, to achieve the purpose of improving classification effect;Through strong supervision deep learning classification network, using deep learning method, with good generalization, robustness.

Description

Technical Field

[0001] This invention belongs to the field of brain-computer interface technology and relates to a motor imagery EEG signal processing technology, particularly to a motor imagery EEG signal classification method based on masked self-supervised and strongly supervised models, which is especially suitable for the classification of finger motor imagery EEG signals. Background Technology

[0002] Motor imagery is the act of imagining specific movements in the brain without actually performing those movements. Brain-computer interfaces (BCIs) based on motor imagery EEG signals establish a channel between the brain and external devices, enabling tasks such as controlling robots and driving cars. Existing methods for classifying motor imagery EEG signals mainly include machine learning methods (such as support vector machines, linear discriminant analysis, and Naive Bayes classifiers), deep learning methods (such as convolutional neural networks and recurrent neural networks), ensemble learning, and adaptive learning.

[0003] The brain is a highly complex system composed of numerous interconnected neurons, generating electroencephalogram (EEG) signals with chaotic characteristics. Furthermore, due to the physiological structure of the human fingers, there is a high degree of correlation between finger movements, and individual differences exist among data collectors. Therefore, classifying finger movement-related EEG signals is an even more complex problem. For these reasons, machine learning methods, such as the multi-class CSP (Common Spatial Pattern) method proposed by Kato et al., combined with an SVM classifier, struggle to learn deep-level signal features, resulting in unsatisfactory classification performance. Deep learning methods have seen rapid development in recent years and have shown promising applications in motor imagery EEG signal classification (such as left-right hand classification), but they have not yet been applied to finger movement-related EEG signal classification. Summary of the Invention

[0004] This invention addresses the problem of classifying electroencephalogram (EEG) signals related to finger motor imagery by providing a method based on masked self-supervised and strongly supervised models. It designs a masked self-supervised network to learn the deep features of finger motor imagery EEG signals and a strongly supervised deep learning classification network to improve the classification performance of finger motor imagery EEG signals.

[0005] The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models, characterized by including a training phase and a testing phase;

[0006] The steps of the training phase are as follows:

[0007] Step 1.1: Acquire several EEG signals of finger movement imagery from a subject and construct a dataset, denoted as X;

[0008] Step 1.2: Divide the dataset X into multiple subsets according to the set batch size; then preprocess each subset and perform random masking to obtain the mask subset and mask matrix corresponding to each subset;

[0009] Step 1.3: Design a neural network that includes a masked self-supervised network and a strongly supervised deep learning classification network;

[0010] The masked self-supervised network includes an encoder and a decoder. The input of the encoder serves as the input of the masked self-supervised network, and the input of the decoder is connected to the output of the encoder. The output of the decoder serves as the output of the masked self-supervised network. The encoder consists of a first Transformer network block and a first batch normalization layer. The input of the first Transformer network block serves as the input of the encoder, and the input of the first batch normalization layer is connected to the output of the first Transformer network block. The output of the first batch normalization layer serves as the output of the encoder. The decoder consists of a first linear layer, a second Transformer network block, a second batch normalization layer, and a second linear layer. The network consists of layers: the input of the first linear layer serves as the input of the decoder; the input of the second Transformer network block is connected to the output of the first linear layer; the input of the second batch normalization layer is connected to the output of the second Transformer network block; the input of the second linear layer is connected to the output of the second batch normalization layer; and the output of the second linear layer serves as the output of the decoder. The first Transformer network block has a depth of 4 and 4 attention heads, the second Transformer network block has a depth of 2 and 4 attention heads, the first linear layer has 512 neurons, and the second linear layer has 25 × 6 = 150 neurons.

[0011] The strongly supervised deep learning classification network consists of a third batch normalization layer, a fourth batch normalization layer, a first fully connected layer, a second fully connected layer, and a third fully connected layer. The input of the third batch normalization layer serves as the first input of the strongly supervised deep learning classification network, and the input of the fourth batch normalization layer serves as the second input. The input of the first fully connected layer receives the dataset obtained by merging and concatenating the datasets output from the third and fourth batch normalization layers. The input of the second fully connected layer is connected to the output of the first fully connected layer, and the input of the third fully connected layer is connected to the output of the second fully connected layer. The output of the third fully connected layer serves as the output of the strongly supervised deep learning classification network. Specifically, the first fully connected layer has 500 neurons and uses the leaky_relu activation function; the second fully connected layer has 100 neurons and uses the leaky_relu activation function; and the third fully connected layer has 5 neurons and uses the softmax activation function.

[0012] Step 1.4: Train the neural network: for any subset X B X B mask subset X masked Input mask self-supervised network, the encoder input receives X masked The dataset output by the encoder is denoted as X. en X en The dimension is (B, 29, 768); the input of the decoder receives X. en The dataset output by the second batch normalization layer is denoted as X. de X de The dimension of the dataset is (B, 72, 512), and the output dataset of the decoder is denoted as X. pred X pred The dimension is (B, 72, 150); X B and X en Input to a strongly supervised deep learning classification network, the first input of the strongly supervised deep learning classification network receives X B The second input of a strongly supervised deep learning classification network receives X. en The merged dataset is denoted as X1, with dimensions [B, 33072]. The dataset output from the first fully connected layer is denoted as X2, with dimensions [B, 500]. The dataset output from the second fully connected layer is denoted as X3, with dimensions [B, 100]. The dataset output from the third fully connected layer is X... B The classification probability is denoted as P. yc P yc The dimension is [B,5]; after obtaining P ycNext, the loss functions of the masked self-supervised network and the strongly supervised deep learning classification network are calculated, denoted as loss1 and loss2 respectively. Then, the weighted sum of loss1 and loss2 is calculated to obtain the loss function of the entire neural network, denoted as loss; where B represents the batch size, i.e., X. B The number of finger movement imagery EEG signals included, loss1 is calculated using the MSE loss function, and loss2 is calculated using the cross-entropy loss function;

[0013] Step 1.5: Perform gradient descent on the neural network based on the loss, and update the network parameters and weight parameters of the masked self-supervised network and the strongly supervised deep learning classification network respectively; when the loss stops decreasing or the decrease becomes gradual, the training is completed, the neural network model is obtained, and the network parameters are saved.

[0014] The steps of the testing phase are as follows:

[0015] Step 2.1: Acquire several EEG signals of finger movement imagery from any one subject and construct a test dataset, denoted as X. test ;

[0016] Step 2.2: Following the same procedure as in Step 1.2, process the test dataset X in the same way. test The batch size is divided into multiple test subsets; then each test subset is preprocessed and then subjected to random masking to obtain the mask subset and mask matrix corresponding to each test subset.

[0017] Step 2.3: Input each test subset and its corresponding mask subset into the trained neural network model. The neural network model outputs the classification probability of each test subset.

[0018] In step 1.1, each finger movement imagery EEG signal is stored in the form of a data matrix with a dimension of T×N; where T represents the number of sampling points of a finger movement imagery EEG signal and N represents the number of channels.

[0019] In step 1.2, the batch size is set to B, and the number of finger movement imagery EEG signals in dataset X is set to be divisible by B.

[0020] In step 1.2, for any subset X B The preprocessing process is as follows: X B Take a PatchEmbed network block as input, and the PatchEmbed network block outputs B patches. Let X be the set of these B patches. P Then for X P Each patch in the dataset is encoded with a position vector, and the resulting set is denoted as X.P2 ; where X B The dimensions are [B, T, N], where B represents the set batch size, T represents the number of sampling points for a finger movement imagery EEG signal, and N represents the number of channels. The PatchEmbed network block consists of a convolutional layer and a batch normalization layer. The input of the convolutional layer receives X... B The input of the batch normalization layer is connected to the output of the convolutional layer. The output of the batch normalization layer is B patches. The convolutional kernel size is (25, 6), the stride is (25, 6), and the patch dimension is (72, 768). P2 The dimensions are (B, 72, 768).

[0021] In step 1.2, for any subset X B X obtained after preprocessing P2 The random masking process is as follows: given a mask ratio of mask_ratio, X is... P2 Input a random mask function, the random mask function for X P2 The second dimension is randomly masked, and the random masking function outputs X. B The corresponding mask subset and mask matrix are denoted as X. masked and masks, which consist of 0s and 1s; where mask_ratio ∈ (0,1), X masked The dimension of the mask is (B, M, 768), where M represents the number of masks that are not masked, M = 72 (1 - mask_ratio), and the dimension of the masks is (B, 72).

[0022] The given mask ratio, mask_ratio, is set to 0.6, X masked The dimensions are [B, 28, 768], meaning that 60% is occluded.

[0023] In step 1.4, Among them, X pred X is the dataset output by the decoder. target For X B The converted patch representation has dimensions [25, 6], X target The dimension is [B, 72, 150], and masks are the X output of a random masking function. B The corresponding mask matrix, Represents the matrix (X) pred -X target ) 2 The sum of all elements in masks, where γ represents the matrix (X). pred -X target ) 2• The row number of masks, l indicates matrix (X) pred -X target ) 2 • The column number of masks, ∑masks ij The sum of all elements in the matrix mask is the number of elements masked during the random masking process. i represents the row number of the matrix mask, and j represents the column number of the matrix mask.

[0024] In step 1.4, loss2(b) represents X B The loss function corresponding to the finger movement imagery EEG signal in the b-th clause. T represents the number of sampling points for the EEG signal of finger movement imagery, K represents the number of classification categories, K=5 represents 5 fingers, s yc () is the sign function, when the true category of the b-th finger movement imagery EEG signal is equal to k. yc (b) Take 1, when the true category of the b-th finger movement imagination EEG signal is not equal to k. yc (b) Set p to 0 yc (b) represents the probability that the predicted category of the b-th finger movement imagery EEG signal is k.

[0025] In step 1.4, loss = σ1 × loss1 + σ2 × loss2, where σ1 represents the weight of loss1 and σ2 represents the weight of loss2. σ1 and σ2 are configured using a dynamic weighting method.

[0026] Compared with the prior art, the advantages of the present invention are as follows:

[0027] By using a mask encoding and decoding design, the mask self-supervised network can learn the deep features of finger movement imagery EEG signals. Then, a strongly supervised deep learning classification network is used to improve the classification effect. The deep learning method has the advantages of good generalization, robustness and easy deployment. Attached Figure Description

[0028] Figure 1 A schematic diagram of D finger movement imagery EEG signals from a single subject in a 5F finger movement imagery EEG signal dataset.

[0029] Figure 2 The flowchart shows the composition of the neural network designed for the method of this invention, which includes a masked self-supervised network and a strongly supervised deep learning classification network. Detailed Implementation

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0031] This invention proposes a method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models, which includes a training phase and a testing phase.

[0032] The steps of the training phase are as follows:

[0033] Step 1.1: Obtain several EEG signals of finger movement imagery from a subject and construct a dataset, denoted as X.

[0034] In practice, a finger motor imagery EEG dataset can be directly selected, such as the 5F finger motor imagery EEG dataset from The Largest SCP Data of Motor-Imagery, an internationally standardized EEG database. This dataset was collected from 8 subjects, with data acquired at a sampling rate of 200Hz or 1000Hz. Each dataset contains 21 channels of finger motor imagery EEG signals. Each subject completed a corresponding mental imagery by viewing prompts from a graphical user interface (eGUI). A motion signal (represented by a number from 1 to 5) appeared directly above the finger to be moved, lasting for 1 second. During this time, the subject performed the indicated mental imagery once, followed by a 1.5-2.5 second pause to end the experiment. Depending on the subject's preference, the image was recalled as the corresponding finger bending upwards or downwards. Each motion signal required a response. After performing the displayed motion, the subject remained still until the next motion signal appeared. At the beginning of each recording session, a 2.5-minute initial relaxation period was given to allow subjects to relax and adapt to the recording session conditions. Each recording session lasts 50 to 55 minutes, and motor imagery EEG signals are continuously recorded throughout the process.

[0035] In this embodiment, in step 1.1, each finger movement imagery EEG signal is stored in the form of a data matrix with a dimension of T×N; where T represents the number of sampling points of a finger movement imagery EEG signal, and in this embodiment, T is taken as approximately 600, and N represents the number of channels, and in this embodiment, N = 21. Figure 1 A schematic diagram of D finger motor imagery EEG signals from a single subject within a 5F finger motor imagery EEG signal dataset is provided. The set of D finger motor imagery EEG signals is denoted as X, where X = {x1, ..., x...} i ,...,x D In this embodiment, D is taken to be around 950, and x1 represents the data matrix of the first finger movement imagery EEG signal in X. i This represents the data matrix of the i-th finger movement imagery EEG signal in X, x D Let X represent the data matrix of the D-th finger movement imagery EEG signal, where 1 ≤ i ≤ D.

[0036] Step 1.2: Divide the dataset X into multiple subsets according to the set batch size; then preprocess each subset and perform random masking to obtain the mask subset and mask matrix corresponding to each subset.

[0037] In this embodiment, in step 1.2, the batch size is set to B, and the number of finger movement imagery EEG signals in the dataset X is set to be divisible by B. In this embodiment, B = 32.

[0038] In this embodiment, in step 1.2, for any subset X B The preprocessing process is as follows: X B Take a PatchEmbed network block as input, and the PatchEmbed network block outputs B patches. Let X be the set of these B patches. P Then for X P Each patch in the dataset is encoded with a position vector, and the resulting set is denoted as X. P2 ; where X B The dimensions are [B, T, N], where B represents the set batch size, T represents the number of sampling points for a finger movement imagery EEG signal, and N represents the number of channels. The PatchEmbed network block consists of a convolutional layer and a batch normalization layer. The input of the convolutional layer receives X... B The input of the batch normalization layer is connected to the output of the convolutional layer. The output of the batch normalization layer is B patches. The convolutional kernel size is (25, 6), the stride is (25, 6), and the patch dimension is (72, 768). P2 The dimensions are (B, 72, 768). Here, the PatchEmbed network block is an existing network block; adding position vector encoding to a patch is a prior art technique.

[0039] In this embodiment, in step 1.2, for any subset X B X obtained after preprocessing P2 The random masking process is as follows: given a mask ratio of mask_ratio, X is... P2 Input a random mask function, the random mask function for X P2 The second dimension is randomly masked, and the random masking function outputs X. B The corresponding mask subset and mask matrix are denoted as X. masked The mask consists of 0s and 1s, and is used to calculate the masked portion of the data by multiplying it with the original dataset matrix; where mask_ratio ∈ (0,1), X maskedThe dimension is (B, M, 768), where M represents the number of unmasked objects, M = 72(1 - mask_ratio). Given a mask ratio of mask_ratio of 0.6, X... masked The dimensions of the mask are [B, 28, 768], meaning 60% is masked, and the dimensions of the masks are (B, 72). If mask_ratio is 0, it means no masking is performed; if mask_ratio is 1, it means full masking. Here, the random masking function is the existing function.

[0040] Step 1.3: As Figure 2 As shown, the design includes a masked self-supervised network and a strongly supervised deep learning classification network. The masked self-supervised network learns the deep features of the brain signals of finger movement imagination to improve the classification effect of the subsequent strongly supervised deep learning classification network, which is a classifier.

[0041] The mask self-supervised network consists of an encoder and a decoder. The input of the encoder serves as the input of the mask self-supervised network, and the input of the decoder is connected to the output of the encoder. The output of the decoder serves as the output of the mask self-supervised network. The encoder consists of a first Transformer network block and a first batch normalization layer. The network consists of a first Transformer network block (BN) with the input of the first Batch Normalization layer serving as the encoder's input. The input of the first Batch Normalization layer is connected to the output of the first Transformer network block, and the output of the first Batch Normalization layer serves as the encoder's output. The decoder consists of a first linear layer, a second Transformer network block, a second Batch Normalization layer, and a second linear layer. The input of the first linear layer serves as the decoder's input, the input of the second Transformer network block is connected to the output of the first linear layer, the input of the second Batch Normalization layer is connected to the output of the second Transformer network block, and the input of the second linear layer is connected to the output of the second Batch Normalization layer. The output of the second linear layer serves as the decoder's output. The first Transformer network block has a depth of 4 and 4 attention heads, the second Transformer network block has a depth of 2 and 4 attention heads, the first linear layer has 512 neurons, and the second linear layer has 25 × 6 = 150 neurons. Here, the Transformer network block is an existing network block.

[0042] The strongly supervised deep learning classification network consists of a third batch normalization layer, a fourth batch normalization layer, a first fully connected layer, a second fully connected layer, and a third fully connected layer. The input of the third batch normalization layer serves as the first input of the strongly supervised deep learning classification network, and the input of the fourth batch normalization layer serves as the second input. The input of the first fully connected layer receives the dataset obtained by merging and concatenating the datasets output from the third and fourth batch normalization layers. The input of the second fully connected layer is connected to the output of the first fully connected layer, and the input of the third fully connected layer is connected to the output of the second fully connected layer. The output of the third fully connected layer serves as the output of the strongly supervised deep learning classification network. Specifically, the first fully connected layer has 500 neurons and uses the leaky_relu activation function, the second fully connected layer has 100 neurons and uses the leaky_relu activation function, and the third fully connected layer has 5 neurons (5 fingers) and uses the softmax activation function.

[0043] Step 1.4: Train the neural network: for any subset X B X B mask subset X masked Input mask self-supervised network, the encoder input receives X masked The dataset output by the encoder is denoted as X. en X en The dimension is (B, 29, 768); the input of the decoder receives X. en The dataset output by the second batch normalization layer is denoted as X. de X de The dimension of the dataset is (B, 72, 512), and the output dataset of the decoder is denoted as X. pred X pred The dimension is (B, 72, 150); X B and X en Input to a strongly supervised deep learning classification network, the first input of the strongly supervised deep learning classification network receives X B The second input of a strongly supervised deep learning classification network receives X. en The merged dataset is denoted as X1, with dimensions [B, 33072]. The dataset output from the first fully connected layer is denoted as X2, with dimensions [B, 500]. The dataset output from the second fully connected layer is denoted as X3, with dimensions [B, 100]. The dataset output from the third fully connected layer is X... B The classification probability is denoted as P. yc P yc The dimension is [B,5]; after obtaining P ycNext, the loss functions of the masked self-supervised network and the strongly supervised deep learning classification network are calculated, denoted as loss1 and loss2 respectively. Then, the weighted sum of loss1 and loss2 is calculated to obtain the loss function of the entire neural network, denoted as loss; where B represents the batch size, i.e., X. B The number of finger movement imagery EEG signals included is calculated using the MSE loss function and the cross-entropy loss function.

[0044] In this embodiment, in step 1.4, Among them, X pred X is the dataset output by the decoder. target For X B The converted patch representation has dimensions [25, 6], X target The dimensions are [B, 72, 150], h × w = 72. T represents the number of sampling points for the EEG signal of imagining finger movement, N represents the number of channels, 25 × 6 = 150, symbol This is the floor function operator, where masks is the output of the random masking function X. B The corresponding mask matrix, Represents the matrix (X) pred -X target ) 2 The sum of all elements in masks, where γ represents the matrix (X). pred -X target ) 2 • The row number of masks, l indicates matrix (X) pred -X target ) 2 • The column number of masks, ∑masks ij The sum of all elements in the matrix mask is the number of elements masked during the random masking process. i represents the row number of the matrix mask and j represents the column number of the matrix mask. Therefore, loss1 only calculates the loss function for the masked part.

[0045] In this embodiment, in step 1.4, loss2(b) represents X B The loss function corresponding to the finger movement imagery EEG signal in the b-th clause. T represents the number of sampling points for the EEG signal of finger movement imagery, K represents the number of classification categories, K=5 represents 5 fingers, s yc () is the sign function, when the true category of the b-th finger movement imagery EEG signal is equal to k. yc(b) Take 1, when the true category of the b-th finger movement imagination EEG signal is not equal to k. yc (b) Set p to 0 yc (b) represents the probability that the predicted category of the b-th finger movement imagery EEG signal is k.

[0046] In this embodiment, in step 1.4, loss = σ1 × loss1 + σ2 × loss2, where σ1 represents the weight of loss1 and σ2 represents the weight of loss2. σ1 and σ2 are configured using a dynamic weighting method. The overall loss function of the neural network is a weighted sum of the loss function of the masked self-supervised network and the loss function of the strongly supervised deep learning classification network. The weights of the loss function of the masked self-supervised network and the loss function of the strongly supervised deep learning classification network are configured using a dynamic weighting method. The weights can be trained along with the neural network and change dynamically.

[0047] Step 1.5: Perform gradient descent on the neural network based on the loss, and update the network parameters and weight parameters of the masked self-supervised network and the strongly supervised deep learning classification network respectively; when the loss stops decreasing or the decrease becomes gradual, the training is completed, the neural network model is obtained, and the network parameters are saved.

[0048] The steps of the testing phase are as follows:

[0049] Step 2.1: Acquire several EEG signals of finger movement imagery from any one subject and construct a test dataset, denoted as X. test .

[0050] Step 2.2: Following the same procedure as in Step 1.2, process the test dataset X in the same way. test The batch size is divided into multiple test subsets; then each test subset is preprocessed and then subjected to random masking to obtain the mask subset and mask matrix corresponding to each test subset.

[0051] Step 2.3: Input each test subset and its corresponding mask subset into the trained neural network model. The neural network model outputs the classification probability of each test subset.

[0052] To verify the feasibility and effectiveness of the method of the present invention, experiments were conducted on the method of the present invention.

[0053] The experimental environment was Windows 10, using an RTX 4070 GPU for training, and the PyTorch deep learning framework. The learning rate was 0.0001, the mask rate was 0.6, the training epochs were 50, and the experiment was repeated 100 times with the average result taken.

[0054] Eight subjects were selected, and at least one test dataset was obtained for each subject. Using steps 2.2 and 2.3 of the method of this invention, the classification probability of each test subset of each test dataset was obtained. The average classification probability of all test subsets of a test dataset was taken as the classification probability of the test dataset. Table 1 shows the experimental results.

[0055] Table 1 shows the experimental results of testing on test datasets from 8 subjects.

[0056] A-160408 19.43％ B-160309 44.18％ B-160311 45.11％ C-160429 58.06％ E-160321 54.24％ E-160415 68.59％ E-160429 58.74％ F-160210 31.91％ G-160413 42.88％ G-160428 41.73％ H-160804 37.47％ I-160719 55.70％ I-160723 47.48％ average 46.58％

[0057] Observing Table 1, it can be seen that the method of this invention achieves a higher classification accuracy compared to the method proposed by Kato et al. In the SCP dataset, the official methods used, such as Support Vector Machines and Event-Related Potentials (ERPs), did not exceed 60% in the 5-finger classification accuracy, with an average accuracy of only 43%. In contrast, the method of this invention achieves a classification accuracy exceeding 60% on the test dataset corresponding to E-160415 in the 5-finger classification task, generally outperforming the classification accuracy published by the official SCP. The method of this invention exhibits better generalization performance.

Claims

1. A method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models, characterized in that... Includes training and testing phases; The steps of the training phase are as follows: Step 1.1: Acquire several EEG signals of finger movement imagery from a subject and construct a dataset, denoted as X; Step 1.2: Divide the dataset X into multiple subsets according to the set batch size; then preprocess each subset and perform random masking to obtain the mask subset and mask matrix corresponding to each subset; Step 1.3: Design a neural network that includes a masked self-supervised network and a strongly supervised deep learning classification network; The masked self-supervised network includes an encoder and a decoder. The input of the encoder serves as the input of the masked self-supervised network, and the input of the decoder is connected to the output of the encoder. The output of the decoder serves as the output of the masked self-supervised network. The encoder consists of a first Transformer network block and a first batch normalization layer. The input of the first Transformer network block serves as the input of the encoder, and the input of the first batch normalization layer is connected to the output of the first Transformer network block. The output of the first batch normalization layer serves as the output of the encoder. The decoder consists of a first linear layer, a second Transformer network block, a second batch normalization layer, and a second linear layer. The network consists of layers: the input of the first linear layer serves as the input of the decoder; the input of the second Transformer network block is connected to the output of the first linear layer; the input of the second batch normalization layer is connected to the output of the second Transformer network block; the input of the second linear layer is connected to the output of the second batch normalization layer; and the output of the second linear layer serves as the output of the decoder. The first Transformer network block has a depth of 4 and 4 attention heads, the second Transformer network block has a depth of 2 and 4 attention heads, the first linear layer has 512 neurons, and the second linear layer has 25 × 6 = 150 neurons. The strongly supervised deep learning classification network consists of a third batch normalization layer, a fourth batch normalization layer, a first fully connected layer, a second fully connected layer, and a third fully connected layer. The input of the third batch normalization layer serves as the first input of the strongly supervised deep learning classification network, and the input of the fourth batch normalization layer serves as the second input. The input of the first fully connected layer receives the dataset obtained by merging and concatenating the datasets output from the third and fourth batch normalization layers. The input of the second fully connected layer is connected to the output of the first fully connected layer, and the input of the third fully connected layer is connected to the output of the second fully connected layer. The output of the third fully connected layer serves as the output of the strongly supervised deep learning classification network. Specifically, the first fully connected layer has 500 neurons and uses the leaky_relu activation function; the second fully connected layer has 100 neurons and uses the leaky_relu activation function; and the third fully connected layer has 5 neurons and uses the softmax activation function. Step 1.4: Train the neural network: for any subset X B X B mask subset X masked Input mask self-supervised network, the encoder input receives X masked The dataset output by the encoder is denoted as X. en X en The dimension is (B, 29, 768); the input of the decoder receives X. en The dataset output by the second batch normalization layer is denoted as X. de X de The dimension of the dataset is (B, 72, 512), and the output dataset of the decoder is denoted as X. pred X pred The dimension is (B, 72, 150); X B and X en Input to a strongly supervised deep learning classification network, the first input of the strongly supervised deep learning classification network receives X B The second input of a strongly supervised deep learning classification network receives X. en The merged dataset is denoted as X1, with dimensions [B, 33072]. The dataset output from the first fully connected layer is denoted as X2, with dimensions [B, 500]. The dataset output from the second fully connected layer is denoted as X3, with dimensions [B, 100]. The dataset output from the third fully connected layer is X... B The classification probability is denoted as P. yc P yc The dimension is [B,5]; after obtaining P yc Next, the loss functions of the masked self-supervised network and the strongly supervised deep learning classification network are calculated, denoted as loss1 and loss2 respectively. Then, the weighted sum of loss1 and loss2 is calculated to obtain the loss function of the entire neural network, denoted as loss; where B represents the batch size, i.e., X. B The number of finger movement imagery EEG signals included, loss1 is calculated using the MSE loss function, and loss2 is calculated using the cross-entropy loss function; Step 1.5: Perform gradient descent on the neural network based on the loss, and update the network parameters and weight parameters of the masked self-supervised network and the strongly supervised deep learning classification network respectively; when the loss stops decreasing or the decrease becomes gradual, the training is completed, the neural network model is obtained, and the network parameters are saved. The steps of the testing phase are as follows: Step 2.1: Acquire several EEG signals of finger movement imagery from any one subject and construct a test dataset, denoted as X. test ; Step 2.2: Following the same procedure as in Step 1.2, process the test dataset X in the same way. test The batch size is divided into multiple test subsets; then each test subset is preprocessed and then subjected to random masking to obtain the mask subset and mask matrix corresponding to each test subset. Step 2.3: Input each test subset and its corresponding mask subset into the trained neural network model. The neural network model outputs the classification probability of each test subset.

2. The method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models according to claim 1, characterized in that... In step 1.1, each finger movement imagery EEG signal is stored in the form of a data matrix with a dimension of T×N; where T represents the number of sampling points of a finger movement imagery EEG signal and N represents the number of channels.

3. The method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models according to claim 1, characterized in that... In step 1.2, the batch size is set to B, and the number of finger movement imagery EEG signals in dataset X is set to be divisible by B.

4. The method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models according to claim 3, characterized in that... In step 1.2, for any subset X B The preprocessing process is as follows: X B Take a PatchEmbed network block as input, and the PatchEmbed network block outputs B patches. Let X be the set of these B patches. P Then for X P Each patch in the dataset is encoded with a position vector, and the resulting set is denoted as X. P2 ; where X B The dimensions are [B, T, N], where B represents the set batch size, T represents the number of sampling points for a finger movement imagery EEG signal, and N represents the number of channels. The PatchEmbed network block consists of a convolutional layer and a batch normalization layer. The input of the convolutional layer receives X... B The input of the batch normalization layer is connected to the output of the convolutional layer. The output of the batch normalization layer is B patches. The convolutional kernel size is (25, 6), the stride is (25, 6), and the patch dimension is (72, 768). P2 The dimensions are (B, 72, 768).

5. The method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models according to claim 4, characterized in that... In step 1.2, for any subset X B X obtained after preprocessing P2 The random masking process is as follows: given a mask ratio of mask_ratio, X is... P2 Input a random mask function, the random mask function for X P2 The second dimension is randomly masked, and the random masking function outputs X. B The corresponding mask subset and mask matrix are denoted as X. masked and masks, which consist of 0s and 1s; where mask_ratio ∈ (0,1), X masked The dimension of the mask is (B, M, 768), where M represents the number of masks that are not masked, M = 72 (1 - mask_ratio), and the dimension of the masks is (B, 72).

6. The method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models according to claim 5, characterized in that... The given mask ratio, mask_ratio, is set to 0.6, X masked The dimensions are [B, 28, 768], meaning that 60% is occluded.

7. The method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models according to claim 1, characterized in that... In step 1.4, Among them, X pred X is the dataset output by the decoder. target For X B The converted patch representation has dimensions [25, 6], X target The dimension is [B, 72, 150], and masks are the X output of a random masking function. B The corresponding mask matrix, Represents the matrix (X) pred -X target ) 2 The sum of all elements in masks, where γ represents the matrix (X). pred -X target ) 2 • line number of masks Represents the matrix (X) pred -X target ) 2 • The column number of masks, Σmasks ij The sum of all elements in the matrix mask is the number of elements masked during the random masking process. i represents the row number of the matrix mask, and j represents the column number of the matrix mask.

8. The method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models according to claim 7, characterized in that... In step 1.4, loss2(b) represents X B The loss function corresponding to the finger movement imagery EEG signal in the b-th clause. T represents the number of sampling points for the EEG signal of finger movement imagery, K represents the number of classification categories, K=5 represents 5 fingers, s yc () is the sign function, when the true category of the b-th finger movement imagery EEG signal is equal to k. yc (b) Take 1, when the true category of the b-th finger movement imagination EEG signal is not equal to k. yc (b) Set p to 0 yc (b) represents the probability that the predicted category of the b-th finger movement imagery EEG signal is k.

9. The method for classifying motor imagery EEG signals based on masked self-supervised and strongly supervised models according to claim 8, characterized in that... In step 1.4, loss = σ1 × loss1 + σ2 × loss2, where σ1 represents the weight of loss1 and σ2 represents the weight of loss2. σ1 and σ2 are configured using a dynamic weighting method.

Images