Motor imagery classification system and method

Abstract

The present invention discloses a motor imagery classification system, comprising a data preprocessing module, a feature extraction and fusion module, and a classification module; wherein the data preprocessing module is used to extract 3D spatial spectrum features in EEG signals; the feature extraction and fusion module uses double-branch real-valued convolution and complex-valued convolution to respectively extract spatial features and frequency features in 3D spatial spectrum features; then the extracted spatial features and frequency features are fused; and the classification module performs classification according to the fusion features output by the feature fusion module. The present invention improves the decoding performance by overall improving the extraction of multivariate features of EEG signals. The present invention also provides a motor imagery classification method.

Description

Technical Field

[0001] The invention belongs to the field of electroencephalogram signal processing, and in particular relates to a motor imagery classification system and method. Background Art

[0002] Brain-computer interfaces (BCIs) provide an alternative pathway between humans and external devices, allowing users to use their intentions to control prostheses, which acts like "decoding". It is currently widely used in the field of rehabilitation training. The motor imagery (MI for short) task is one of the most commonly used BCI paradigms. When humans mentally simulate movements in response to given prompts, the sensorimotor rhythm (SMR) can be regulated. When the subject imagines the movement of a specific body part, the corresponding area of ​​the brain is activated. As a result, the area will experience a desynchronization of the μ rhythm, which is an idle state rhythm based on about 10Hz. This process is called event-related desynchronization (ERD). ERD can be detected in electroencephalogram (EEG) recordings and further decoded to detect the user's intentions. The processed signal can then be sent to control external devices.

[0003] Feature extraction is particularly critical in the process of decoding EEG signals in brain-computer interface systems. Currently, a large number of inventions are aimed at improving the classification performance of MI-based BCI. The traditional common spatial pattern (CSP) is a popular feature extraction algorithm. CSP is based on the simultaneous diagonalization of two task-related covariance matrices, which minimizes the variance of one type of spatial filter signal while maximizing the variance of another type. However, its frequency band is usually selected manually or heuristically, which leads to poor classification accuracy. In addition, due to the differences between subjects, such as the frequency range, time and spatial location of brain regions induced by EEG, the generalization of the model is weakened.

[0004] With the application of deep learning in the field of BCI, most EEG signals are currently transformed into images for classification, but no attention is paid to preserving the spatial and spectral structure of the multivariate features of EEG. For example, the target frequency component is determined by short-time Fourier transform (STFT), but the time resolution or frequency resolution cannot be obtained at the same time, and the limited window width will cause certain spectrum leakage; wavelet transform is a time-frequency analysis method that can perform multi-scale analysis. However, most researchers use the statistical characteristics of wavelet coefficients as EEG features, which requires manual screening and is based on certain prior knowledge. The pattern of motor imagery signals is not fixed, which will lead to abnormal changes in EEG signals, resulting in low efficiency in identifying motor imagery. Summary of the invention

[0005] Purpose of the invention: In view of the problems existing in the prior art, the present invention proposes a motor imagery classification system that generates spatial spectrum multivariate features representing the spatial spectrum of EEG signals; uses two-branch real-valued convolution and complex-valued convolution to extract complementary frequency information and spatial information respectively; and introduces a squeeze and excitation (SE) mechanism to improve the representation of the spatial-spectral information contained in the temporal spectrum features.

[0006] Technical solution: To achieve the above-mentioned purpose, the present invention provides a motor imagery classification system, including a data preprocessing module, a feature extraction and fusion module, and a classification module; the feature extraction and fusion module and the classification module are based on a deep neural network;

[0007] Among them, the data preprocessing module is used to extract the 3D spatial spectrum features in the EEG signal;

[0008] The feature extraction and fusion module uses dual-branch real-valued convolution and complex-valued convolution to extract the spatial features and frequency features in the 3D spatial spectrum features respectively; then the extracted spatial features and frequency features are fused;

[0009] The classification module performs classification based on the fused features output by the feature fusion module.

[0010] Furthermore, the data and processing module includes a data set division submodule, a filtering submodule and a 3D spatial spectrum feature generation submodule; wherein,

[0011] The data partitioning submodule randomly divides the collected EEG signals into training set, validation set and test set;

[0012] The filtering submodule obtains a filter group from the initial set, and obtains an optimized filter group from the filter group based on mutual information; and uses the optimized filter group to filter the collected electroencephalogram signal;

[0013] The 3D spatial spectrum feature generation submodule forms 3D spatial spectrum features by stacking and calculating the Pearson product-moment correlation coefficients of rows and columns in the filtered EEG signal.

[0014] Furthermore, the feature extraction and fusion module includes a complex-valued domain extraction submodule, a real-valued domain extraction submodule and a fusion submodule.

[0015] The complex-valued domain extraction submodule includes Fourier transform and complex-valued convolution. The 3D spatial spectrum features extract complex-valued domain information through fast Fourier transform, and then it is input into the fusion submodule after complex-valued convolution; the real-valued domain extraction submodule includes real-valued convolution, and the 3D spatial spectrum features are input into the fusion submodule through real-valued convolution; the fusion submodule fuses the signals output by the complex-valued domain extraction submodule and the real-valued domain extraction submodule.

[0016] Furthermore, the real-valued convolution consists of three 3×3×3 3D convolution layers, and the complex-valued convolution is the complex domain form of the real-valued convolution.

[0017] The present invention also provides a motor imagery classification method, which mainly comprises the following steps:

[0018] Step 1: Extract 3D spatial spectrum features from EEG signals;

[0019] Step 2: First, extract the spatial features and frequency features in the 3D spatial spectrum features respectively, and then fuse the extracted spatial features and frequency features;

[0020] Step 3: Classify according to the fusion features output by the feature fusion module.

[0021] Furthermore, the method for extracting 3D spatial spectrum features in EEG signals is:

[0022] Obtaining a filter bank from the initial set, and obtaining an optimized filter bank from the filter bank based on the mutual information;

[0023] The 3D spatial spectrum features are formed by stacking and calculating the Pearson product-moment correlation coefficients of rows and columns in the filtered EEG signal.

[0024] Furthermore, the method for extracting the spatial features in the 3D spatial spectrum features in step 2 is to input the 3D spatial spectrum features into a real-valued convolution for spatial feature extraction, wherein the real-valued convolution is a three-layer 3×3×3 3D convolution layer.

[0025] Furthermore, the method for extracting frequency features from 3D spatial spectrum features in step 2 is to extract complex-valued domain information from the 3D spatial spectrum features through fast Fourier transform, and then extract frequency features through complex-valued convolution.

[0026] The present invention also provides a computer-readable medium storing software, wherein the software includes instructions that can be executed by one or more computers, and the instructions, through such execution, enable the one or more computers to perform operations, and the operations include a process based on the above-mentioned motor imagery classification method.

[0027] The present invention also provides a computer system, comprising:

[0028] one or more processors;

[0029] A memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors are caused to perform operations, wherein the operations include a process based on the above-mentioned motor imagery classification method.

[0030] Working principle: The present invention proposes to generate spatial spectrum multivariate features of EEG signals represented by spatial spectrum; due to the dependency between dimensions, dual-branch real-valued convolution and complex-valued convolution are used to extract complementary frequency information and spatial information respectively to improve the robustness of the model; and then a squeeze and excitation (SE) mechanism is introduced to improve the representation of spatial-spectral information contained in the temporal spectrum features, thereby improving the extraction of multivariate features of EEG signals as a whole to improve the decoding performance.

[0031] Beneficial effects: Compared with the prior art, the multi-branch complex-valued feature fusion network model proposed in the present invention can effectively retain the spatial spectrum feature representation of task-related information, and the complex-valued convolution (CVNN for short) has improved performance, faster learning speed, and better generalization effect;

[0032] After the features are fused, the squeeze and excitation (SE) mechanism is used to selectively enhance some useful feature maps and suppress some useless feature maps using global information, thereby improving the training speed and performance of the network;

[0033] The system provided by the present invention proves that in MI, the fusion of complex-valued convolution and real-valued convolution can reasonably utilize the frequency domain and spatial domain information of EEG to improve the performance of the MI-BCI system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a schematic diagram of the motor imagery classification system provided in Example 1;

[0035] Figure 2 Schematic diagram of feature extraction and fusion module;

[0036] Figure 3 is a schematic diagram of the classification module;

[0037] Figure 4 A flowchart of the motor imagery classification method provided in Example 2;

[0038] Figure 5 Comparison chart of classification accuracy decoded by different algorithms for each subject;

[0039] Figure 6 This is a comparison chart of the confusion matrix of the present invention and the other three decoding models. DETAILED DESCRIPTION

[0040] The following will be combined with the accompanying drawings in the examples of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Embodiment 1:

[0042] like Figure 1 As shown, this embodiment provides a motor imagery classification system, which mainly includes a data preprocessing module, a feature extraction and fusion module, a classification module and an evaluation module; the feature extraction and fusion module and the classification module are based on a deep neural network.

[0043] Among them, the data preprocessing module is used to extract the 3D spatial spectrum features in the EEG signals; the feature extraction and fusion module first extracts the spatial features and frequency features in the 3D spatial spectrum features respectively, and then fuses the extracted spatial features and frequency features; the classification module classifies according to the fusion features output by the feature fusion module; the evaluation module is used to evaluate the performance of the classification module.

[0044] The data preprocessing module includes a data set partitioning submodule, a filtering submodule and a 3D spatial spectrum feature generation submodule.

[0045] The data partitioning submodule randomly divides the collected EEG signals into a training set, a validation set, and a test set; the size ratio of the training set, the validation set, and the test set is 8:1:1. The training set, the validation set, and the test set are normalized separately. In this embodiment, the minimum-maximum normalization is used for scaling. During the process of sub-dataset partitioning, it is ensured that the model does not contain any information from the test set when training with the training set or searching for hyperparameters with the validation set.

[0046] The filtering submodule obtains a filter group from the collected original electroencephalogram signal, and obtains an optimized filter group from the filter group based on mutual information; and then uses the optimized filter group to filter the collected original electroencephalogram signal.

[0047] The 3D spatial spectrum feature generation submodule forms 3D spatial spectrum features by stacking and calculating the Pearson product-moment correlation coefficients of rows and columns in the filtered EEG signal.

[0048] like Figure 2 As shown in FIG. 1 , the feature extraction and fusion module uses a dual-branch 3D convolution block to process 3D spatial spectrum features, which includes a complex-valued domain extraction submodule, a real-valued domain extraction submodule, and a fusion submodule.

[0049] The complex-valued domain extraction submodule includes Fourier transform and complex-valued convolution. The 3D spatial spectrum feature extracts the complex-valued domain information through fast Fourier transform, and then it is input into the fusion submodule after complex-valued convolution. The real-valued domain extraction submodule includes real-valued convolution. The 3D spatial spectrum feature is input into the fusion submodule through real-valued convolution. The fusion submodule fuses the signals output by the complex-valued domain extraction submodule and the real-valued domain extraction submodule. The real-valued convolution consists of three 3×3×3 3D convolution layers, which convolve the channel, time, and frequency band of the 3D spatial spectrum feature to obtain the spatial feature. The complex-valued convolution is the complex domain form of the real-valued convolution module. By adding Fourier transform before the complex-valued convolution, the frequency information can be extracted, which supplements the spatial information extracted by the real-valued convolution.

[0050] like Figure 3 As shown in the figure, the classification module passes the fused features output by the feature extraction and fusion module through the squeeze and excitation (SE) block in sequence, and then uses global average pooling to convert the H×W×C input into a 1×1×C output, and then compresses the output between 0 and 1 through two fully connected layers, and then adds the residual with the fused features output by the feature extraction and fusion module; finally, it is composed of two fully connected layers plus a softmax layer, and a Dropout layer is connected after each fully connected layer for classification.

[0051] The evaluation module is used to evaluate whether the performance of the classification module has been improved. All data sets except the test set are used to learn the network parameters, and the test set is used to infer the performance indicators of the learned model.

[0052] Embodiment 2:

[0053] like Figure 4 As shown, this embodiment provides a classification method based on a motor imagery classification system. This embodiment selects the BCI IV2a data set to train the evaluation system; the BCI IV2a data set is a four-category motor imagery task, including the left hand, right hand, foot and tongue, and records the data collected by 22 EEG (electroencephalogram signal hereinafter referred to as EEG) channels of 9 healthy subjects, with a sampling frequency of 250Hz. The data set contains two complete experiments recorded on two different dates for each subject. Each experiment includes multiple motor imagery tasks, and each motor imagery task starts at t=0s, and then when t=2s, a 1.25-second arrow prompt appears, prompting the subject to perform the corresponding motor imagery task: left hand, right hand, feet, tongue. The subject performs the motor imagery task for 3-6 seconds; each motor imagery task includes a process of prompting, imagination, and feedback.

[0054] The main steps include:

[0055] Step 1: The data preprocessing module extracts the 3D spatial spectrum features in the EEG signal. Specifically includes:

[0056] Step 101: The collected EEG signals are randomly divided into a training set, a validation set, and a test set; the size ratio of the training set, the validation set, and the test set is 8:1:1. The training set, the validation set, and the test set are scaled using min-max normalization, and during the process of dividing the sub-datasets, it is ensured that the model does not contain any information from the test set when using the training set for training or using the validation set for searching hyperparameters.

[0057] Step 102: Obtain a filter bank from the collected original EEG signal, and obtain an optimized filter bank from the filter bank based on mutual information. Then, the collected EEG signal is filtered using the optimized filter bank to obtain an optimized EEG signal.

[0058] The collected original EEG signal is: X s represents the set of original EEG signals collected in the sth trial, y s It represents the category of motor imagery performed in the sth trial, S is the total number of trials, and each trial completes a motor imagery process.

[0059] The initial filter group set is 8Hz to 30Hz, the entire range includes the α band (8Hz ~ 13Hz), the β band (14Hz ~ 30Hz), the range of each individual bandpass filter is 4Hz, the step size is 1Hz, and then the mutual information between the filters is calculated, the calculated mutual information is sorted in descending order, and N is selected from high to low. f Bandpass filters; thus finding a more accurate optimized frequency band; generally start with N f For 5.

[0060] First, the original EEG signal is filtered using the bandpass filter in the initial filter group set to obtain the first filtered signal Z 1 , using the local average reference function with electrode Cz as the reference, the first filtered signal Z 1 Perform normalization processing to obtain the second filtered signal Z 2 .

[0061] Mutual information I(f i ,Y) is the second filtered signal Z calculated by CSP (Common Space Pattern Algorithm, referred to as CSP) 2 The eigenvector F is obtained, f i represents the feature vector extracted from the bandpass filter of the i-th frequency band;

[0062] F = log(var(W T Z 2 )); I(f i ,Y)=H(Y)-H(Y|f i )

[0063] Where W is the spatial filter obtained by CSP; W T is the transpose of W;

[0064]

[0065] H(Y), H(Y|f i ) are the representation entropy and conditional entropy calculated by Bayes’ rule, respectively, where f i,s is the characteristic value of the EEG signal in the ith frequency band of the sth motor imagery task; the N spectrums with the highest mutual information are obtained by sorting the above mutual information in descending order. f N bandpass filters. f The band-pass filters are the optimized filter group; Y represents the total number of motor imagery types in this embodiment.

[0066] Step 103: The 3D spatial spectrum feature generation submodule forms a 3D spatial spectrum feature by stacking and calculating the Pearson product-moment correlation coefficient of the row and column of each EEG signal in the optimized EEG signal set obtained in step 102.

[0067] The Pearson product-moment correlation coefficient matrix is ​​used to compress the filtered EEG signal and obtain information-rich spatial features. The Pearson product-moment correlation coefficient matrix is ​​a normalized form of the covariance matrix, which is a measure of the joint variability of two random variables, thereby obtaining the spatial distribution of the spatial and temporal correlations recorded in the filtered EEG signal:

[0068]

[0069] Among them, Z 3c To optimize the EEG signal Z 3 The row vector, Z 3r To optimize the EEG signal Z 3 Column vector of 3 The row / column dimensions of To optimize the EEG signal Z 3 The column mean of To optimize the EEG signal Z 3 The row mean of ; To optimize the EEG signal Z 3 The standard deviation of the column vector, To optimize the EEG signal Z 3 The standard deviation of the row vector; Z 3c,j To optimize the EEG signal Z 3 The jth row element, ; Z 3r,j To optimize the EEG signal Z 3 The j-th column element, r∈R E＝E, R represents the real number space, and E is the number of channels.

[0070] Finally, the Pearson product-moment correlation coefficient matrix of multiple frequency bands is stacked to form a 3D spatial spectrum feature. The 3D spatial spectrum feature M is expressed as M = [r 1 ,r 2 ,r 3 …,r Nf ]; Among them, r 1 is the spatial distribution of spatial and temporal correlations recorded in the EEG signal after filtering by the first optimized filter, and so on, r Nf For the Nth f The spatial distribution of spatial and temporal correlations recorded in the EEG signal after filtering with an optimized filter.

[0071] Step 2: The feature extraction and fusion module first extracts the spatial features and frequency features in the 3D spatial spectrum features respectively, and then fuses the extracted spatial features and frequency features. It includes a complex value domain extraction submodule, a real value domain extraction submodule, and a fusion submodule.

[0072] The complex-valued domain extraction submodule includes Fourier transform and complex-valued convolution. The 3D spatial spectrum feature extracts the complex-valued domain information through fast Fourier transform, and then it is input into the fusion submodule after complex-valued convolution; the real-valued domain extraction submodule includes real-valued convolution, and the 3D spatial spectrum feature is input into the fusion submodule through real-valued convolution; the fusion submodule fuses the signals output by the complex-valued domain extraction submodule and the real-valued domain extraction submodule. The real-valued convolution consists of three layers of 3=3×3 3D convolution layers, which convolve the channel, time, and frequency band of the 3D spatial spectrum feature to obtain the spatial feature; the complex-valued convolution is the complex domain form of the real-valued convolution module. By adding Fourier transform before the complex-valued convolution, the frequency information can be extracted, which supplements the spatial information extracted by the real-valued convolution.

[0073] Step 3: The classification module performs classification based on the fusion features output by the feature fusion module.

[0074] The classification module passes the fusion features output by the feature extraction and fusion module through the squeeze and excitation (SE) block in sequence, and then uses global average pooling to convert the H×W×C input into a 1×1×C output, and then compresses the output between 0 and 1 through two fully connected layers, and then adds the residual with the fusion features output by the feature extraction and fusion module; finally, it is composed of two fully connected layers plus a softmax layer, and a Dropout layer is connected after each fully connected layer for classification. Among them, H, W and C are the length, width and height of the three-dimensional vector of the input signal respectively. The classification module automatically obtains the importance of each feature channel through learning, and then enhances useful features and suppresses features that are not very useful for the current task according to this importance.

[0075] Among them, the extrusion process:

[0076]

[0077] Among them, M m is the Pearson product-moment correlation coefficient matrix of the EEG signal after filtering by the mth optimized filter in the 3D spatial spectrum feature M, E is the dimension of the Pearson product-moment correlation coefficient matrix, i = 1, 2, ..., E; e = 1, 2, ..., E; F sq () is the squeeze and excitation (SE) block function;

[0078] During the squeezing process, the input data eventually becomes a column vector. The length of this vector is equal to the number of channels. Then the activation operation is performed to automatically learn the importance of each feature, amplify the features that have a greater impact on the classification results, and suppress useless features:

[0079] s=F ex (u,W)=σ(g(u,W))=σ(W 2 ELU(W 1 u c ));

[0080] In the formula, σ is the activation function, are the weights of the two fully connected layers, W 1 The dimension reduction layer with a dimension reduction ratio of r, W 2 It is a layer of data dimension increase with the same proportion; it has more nonlinearity and can better fit the complex correlation between channels; it greatly reduces the number of parameters and calculations. Fex(.) is a composite function, that is, after two network layers, the activation function σ is compounded with a g() function inside, ELU(.) is the activation function, and r is the dimensionality reduction ratio factor.

[0081] The MI decoding method of the present invention makes the average accuracy of the four categories of MI reach 85%, as shown in Table 1 and Figure 5 As shown, compared with FBCSP, EEGNet, and WT-CNN in previous studies, the performance was improved by 15%, 13%, and 9%, respectively, indicating that the present invention fully utilizes the multivariate features of EEG signals and the performance is significantly improved.

[0082] Table 1:

[0083]

[0084]

[0085] As shown in Table 2, the average Kappa value of the present invention is 0.76±0.03.

[0086] Table 2:

[0087]

[0088] In addition, the confusion matrix is ​​used to explore the effectiveness of the features extracted by the proposed method, such as Figure 6 The experimental results of the test set are shown in Figure 1, where the values ​​on the diagonal of the confusion matrix are the correctly predicted samples for the classification task of each motion image.

[0089] The present invention also provides a computer system, comprising: one or more processors; a memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors perform operations, wherein the operations include a process based on the above-mentioned motor imagery classification method.

[0090] It should be understood that the above-mentioned examples of the present invention based on the above-mentioned motor imagery classification method can be in any computer system including data storage and data processing, and the above-mentioned computer system can be at least one electronic processing system or electronic device including a processor and a memory, such as a PC, whether it is a personal PC, a commercial PC, or a PC for graphics processing, or a server-level PC. These PCs can realize wired and / or wireless data transmission, especially image data, by having a data interface and / or a network interface.

[0091] In other embodiments, the computer system may also be a server, especially a cloud server, having data storage, processing and network communication functions.

[0092] An exemplary computer system generally includes at least one processor, a memory, and a network interface connected by a system bus. The network interface is used to communicate with other devices / systems.

[0093] The processor is used to provide system calculations and control.

[0094] The memory includes non-volatile memory and cache.

[0095] Non-volatile memory generally has a massive storage capacity and can store operating systems and computer programs. These computer programs may include operable instructions that, when executed by one or more processors, enable one or more processors to execute the process based on the above-mentioned motor imagery classification method of the aforementioned embodiment of the present invention.

[0096] In necessary or reasonable implementations, the aforementioned computer system, whether a PC device or a server, may also include more or fewer components than those shown in the diagram, or a combination, or use different hardware, software or other components or different deployment methods.

Claims

1. A motor imagery classification system, characterized in that: It includes data preprocessing module, feature extraction and fusion module, and classification module; the feature extraction and fusion module and classification module are based on deep neural network; Among them, the data preprocessing module is used to extract the 3D spatial spectrum features in the EEG signal; The feature extraction and fusion module uses dual-branch real-valued convolution and complex-valued convolution to extract the spatial features and frequency features in the 3D spatial spectrum features respectively; then the extracted spatial features and frequency features are fused; The classification module performs classification based on the fused features output by the feature fusion module.

2. The motor imagery classification system according to claim 1, characterized in that: The data and processing module includes a data set division submodule, a filtering submodule and a 3D spatial spectrum feature generation submodule; wherein, The data partitioning submodule randomly divides the collected EEG signals into training set, validation set and test set; The filtering submodule obtains a filter group from the initial set, and obtains an optimized filter group from the filter group based on mutual information; and uses the optimized filter group to filter the collected electroencephalogram signal; The 3D spatial spectrum feature generation submodule forms 3D spatial spectrum features by stacking and calculating the Pearson product-moment correlation coefficients of rows and columns in the filtered EEG signal.

3. The motor imagery classification system according to claim 1, characterized in that: The feature extraction and fusion module includes a complex-valued domain extraction submodule, a real-valued domain extraction submodule and a fusion submodule. The complex-valued domain extraction submodule includes Fourier transform and complex-valued convolution. The 3D spatial spectrum features extract complex-valued domain information through fast Fourier transform, and then it is input into the fusion submodule after complex-valued convolution; the real-valued domain extraction submodule includes real-valued convolution, and the 3D spatial spectrum features are input into the fusion submodule through real-valued convolution; the fusion submodule fuses the signals output by the complex-valued domain extraction submodule and the real-valued domain extraction submodule.

4. The motor imagery classification system according to claim 3, characterized in that: The real-valued convolution consists of three layers of 3×3×3 3D convolution layers, and the complex-valued convolution is the complex domain form of the real-valued convolution.

5. A motor imagery classification method, characterized in that: The main steps include: Step 1: Extract 3D spatial spectrum features from EEG signals; Step 2: First, extract the spatial features and frequency features in the 3D spatial spectrum features respectively, and then fuse the extracted spatial features and frequency features; Step 3: Classify according to the fusion features output by the feature fusion module.

6. The motor imagery classification method according to claim 5, characterized in that: The method for extracting 3D spatial spectrum features in EEG signals is: Obtaining a filter bank from the initial set, and obtaining an optimized filter bank from the filter bank based on the mutual information; The 3D spatial spectrum features are formed by stacking and calculating the Pearson product-moment correlation coefficients of rows and columns in the filtered EEG signal.

7. The motor imagery classification method according to claim 5, characterized in that: The method for extracting spatial features from the 3D spatial spectrum features in step 2 is to input the 3D spatial spectrum features into real-valued convolution for spatial feature extraction, wherein the real-valued convolution is a three-layer 3×3×3 3D convolution layer.

8. The motor imagery classification method according to claim 5, characterized in that: The method for extracting frequency features from 3D spatial spectrum features in step 2 is to extract complex-valued domain information from the 3D spatial spectrum features through fast Fourier transform, and then extract frequency features through complex-valued convolution.

9. A computer-readable medium storing software, characterized in that: The software includes instructions that can be executed by one or more computers, and the instructions, through such execution, cause the one or more computers to perform operations, wherein the operations include a process based on the above-mentioned motor imagery classification method as described in any one of claims 5-8.

10. A computer system, characterized in that: include: one or more processors; A memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors perform operations, wherein the operations include a process according to any one of claims 5 to 8 based on the above-mentioned motor imagery classification method.

Images