Decoding model construction method and decoding method for motor imagery brain-computer interface signals

Abstract

The invention belongs to the related technical field of accurate decoding of motor imagery brain-computer interface signals, and particularly relates to a decoding model construction method and decoding method of motor imagery brain-computer interface signals, comprising the steps of constructing a knowledge data fusion network KDFNet which comprises a time domain convolution module, a space domain convolution module, a feature engineering module and a classification module; the time domain convolution module comprises m time domain convolution kernels, different time domain convolution kernels correspond to different frequency bands, and each time domain convolution kernel is used for performing time domain convolution on each channel signal; the spatial domain convolution module comprises m * n spatial domain convolution kernels, and the output of one time domain convolution kernel is correspondingly subjected to further spatial domain convolution processing by n spatial domain convolution kernels; the feature engineering module is used for obtaining spectrum power knowledge features based on the output of each airspace convolution; the FBCSP model is trained in a supervised mode to obtain parameter values of a classifier in the model, and the parameter values are used for initializing parameters of a full connection layer in a classification module; and the KDFNet is trained in a supervised manner. According to the invention, decoding can be realized efficiently.

Description

Technical Field

[0001] The present invention belongs to the technical field related to the accurate decoding of motor imagery brain-computer interface signals, and more specifically, relates to a method for constructing a decoding model and a decoding method for motor imagery brain-computer interface signals. Background Art

[0002] A brain-computer interface establishes a direct communication path between the human brain and external devices (such as computers, wheelchairs, robots, etc.). Electroencephalogram (EEG) records the electrical activities on the scalp of the brain. Due to its convenience and low cost, it is the most widely used input signal in non-invasive brain-computer interfaces. Brain-computer interfaces based on EEG have been applied to robot control, language decoding, stroke rehabilitation, and consciousness assessment, etc.

[0003] Motor imagery (MI) refers to the user imagining the movement of a certain part of a limb (or muscle) without actual motor output. During this process, it will cause changes in the sensory-motor rhythms (SMRs) in the corresponding areas of the cerebral cortex, mainly involving the μ rhythm (8 - 12 Hz) and the β rhythm (14 - 30 Hz), and ERD and ERS phenomena will occur in the corresponding brain regions of the cerebral cortex. For example, when imagining the movement of the left hand, the energy of the two rhythm signals in the right sensorimotor area decreases, and the energy of the two rhythm signals in the left sensorimotor area increases; when imagining the movement of the right hand, the energy of the two rhythm signals in the left sensorimotor area decreases, and the energy of the two rhythm signals in the right sensorimotor area increases. Therefore, based on this phenomenon, by detecting the SMR pattern in a specific area of the cerebral cortex, it can be used to identify the body part imagined by the user, thereby generating various control instructions.

[0004] A motor imagery brain-computer interface means that when a person performs a specific imagination task, it generates a specific pattern of EEG. Through operations such as preprocessing of the specific pattern of EEG, feature extraction, and feature classification, the decoding of EEG is realized, and the activation conditions of different brain regions are detected and recognized to judge the user's intention, and then direct communication and control between the human brain and external devices are realized. Common motor imagery parts are: left hand, right hand, both feet, and tongue.

[0005] Traditional electroencephalogram (EEG) signal analysis methods usually rely on prior knowledge of ERD / ERS and follow three steps: signal processing, feature extraction, and feature classification. The purpose of signal processing is to improve the signal-to-noise ratio of EEG signals. Since ERD / ERS appears in specific frequency bands, corresponding band-pass filters are usually applied to MI EEG. Basic feature extraction techniques focus on time-domain or frequency-domain analysis to extract discriminative features. Due to the spatial characteristics of MI EEG, the Common Spatial Pattern (CSP) and its variants are widely used for feature extraction in MI EEG classification. It converts the original multi-channel EEG signals into more separable spatial patterns by designing spatial filters to maximize the variance ratio of filtered signals of different classes. Initially, CSP was proposed for binary classification problems, and it has been reported to be extended to multi-classification problems. Further, Filter Bank CSP (FBCSP) was proposed, which divides EEG signals into multiple frequency bands through band-pass filters, extracts CSP features from each frequency band, and then selects the most useful features for classification. After CSP filtering, the average energy of the signal in each channel is usually calculated as a feature for classification. Then, various classifiers can be used, such as Logistic Regression (LR), Support Vector Machine (SVM), and Linear Discriminant Analysis (LDA). Although traditional EEG signal analysis methods incorporate some prior knowledge, the significant differences in the data distributions of EEG signals from different devices and subjects often limit their generalization performance on different datasets.

[0006] Convolutional neural networks (CNNs) have also achieved good results in EEG signal decoding. It has been reported that ShallowCNN and DeepCNN are proposed for raw EEG classification. ShallowCNN is inspired by FBCSP, and network modules such as temporal convolution, spatial convolution, log-variance calculation, and classifier in it correspond to specific calculation steps in FBCSP. DeepCNN is similar but contains more convolutional and pooling layers. There is also a report introducing a more compact EEGNet, which uses separable convolution techniques to reduce model parameters and shows good performance in various brain-computer interface tasks.

[0007] Deep learning methods integrate feature extraction and classification into an end-to-end neural network, aiming to directly learn the optimal classification model from raw EEG data. Convolutional neural networks (CNNs) are one of the most popular deep learning models for EEG decoding and show excellent performance in various MI EEG classification tasks. However, these models require a large amount of labeled EEG data to achieve high classification accuracy, and the collection of EEG data is time-consuming and unfriendly. Summary of the Invention

[0008] In view of the above deficiencies or improvement requirements of the prior art, the present invention provides a method for constructing a decoding model and a decoding method for motor imagery brain-computer interface signals, aiming to propose a decoding model construction method with high generalization performance and capable of efficiently realizing the decoding of motor imagery brain-computer interface signals.

[0009] To achieve the above object, according to one aspect of the present invention, there is provided a method for constructing a decoding model for motor imagery brain-computer interface signals, including:

[0010] Construct a knowledge data fusion network KDFNet, including a time-domain convolution module, a spatial-domain convolution module, a feature engineering module, and a classification module. Among them, the time-domain convolution module contains m time-domain convolution kernels of size 1×l, where m is the number of frequency band divisions of the motor imagery brain-computer interface signals actually required. Different time-domain convolution kernels correspond to different frequency bands, and each time-domain convolution kernel is used to perform time-domain convolution on each channel signal; the spatial-domain convolution module contains m*n spatial-domain convolution kernels of size C×1. The output of one time-domain convolution kernel corresponds to further spatial-domain convolution processing by n spatial-domain convolution kernels to obtain features containing frequency-domain knowledge and spatial knowledge corresponding to the frequency band; the feature engineering module is used to calculate the feature of the logarithm of variance for the output of each spatial-domain convolution and splice the results as the spectral power knowledge feature; the classification module is used to perform motor imagery classification based on the spectral power knowledge feature.

[0011] Supervisedly train the FBCSP model to obtain the parameter values of the classifier in the model; the number of band-pass filters in the FBCSP model, the length of each band-pass filter, and the corresponding frequency bands are the same as the time-domain convolution kernels; the number of common spatial pattern filters corresponding to each frequency band is the same as the number of spatial-domain convolution kernels corresponding to that frequency band; use the parameter values of the classifier to initialize the parameters of the fully connected layer in the classification module so that KDFNet obtains classification knowledge; supervisedly train KDFNet to obtain additional knowledge from the training data and complete the construction of the decoding model for motor imagery brain-computer interface signals.

[0012] Further, the feature engineering module is specifically used to calculate the logarithm of variance feature of the features output by each spatial-domain convolution and splice all the logarithms of variance as the spectral power knowledge feature.

[0013] Further, the parameter values of each common spatial pattern filter are also used to initialize the parameters of the corresponding spatial-domain convolution kernel so that KDFNet obtains spatial-domain prior knowledge.

[0014] Further, the parameter values of each band-pass filter are used to initialize the parameters of the corresponding frequency-domain convolution kernel so that KDFNet obtains frequency-domain prior knowledge.

[0015] According to another aspect of the present invention, there is provided a method for decoding motor imagery brain-computer interface signals, which uses the decoding model constructed by the decoding model construction method described above to achieve decoding.

[0016] According to another aspect of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described above and / or the steps of the method described above are implemented.

[0017] According to another aspect of the present invention, there is provided a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is run by a processor, the device where the storage medium is located is controlled to execute the steps of the method described above and / or the steps of the method described above.

[0018] According to another aspect of the present invention, there is provided a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps of the method described above and / or the steps of the method described above are implemented.

[0019] Generally speaking, compared with the prior art through the above technical solutions conceived by the present invention, the technical solutions provided by the present invention mainly have the following beneficial effects:

[0020] 1. The present invention proposes a construction method of a Knowledge Data Fusion Network (KDFNet), which builds a bridge between CSP and CNN, and combines prior knowledge with EEG data. Specifically, first, a Knowledge Data Fusion Network KDFNet is constructed, and an FBCSP model is constructed and trained based on the hyperparameters of each module in the Knowledge Data Fusion Network KDFNet, so as to initialize the parameters of the fully connected layer of the classification module in KDFNet with the parameter values of the classifier in the FBCSP model, so that KDFNet obtains classification knowledge. Then, KDFNet is supervised and trained, and is further improved by gradient descent of the classification loss to obtain additional knowledge from the training data, realizing the fusion of knowledge and data. It is a decoding model construction method with high generalization performance and can efficiently implement the decoding of motor imagery brain-computer interface signals. The present invention verifies through experimental tests that the KDFNet model has significant and stable performance in MI classification and can be used as a decoding method for practical EEG-based brain-computer interface MI.

[0021] 2. The present invention further proposes to use logarithmic variance activation to capture power spectrum information, which helps the neural network capture spectral power features related to motor imagery and improve the decoding accuracy.

[0022] 3. The present invention further proposes to initialize the time-domain convolutional layer and the space-domain convolutional layer by using a band-pass filter and a common spatial pattern filter respectively, so as to mine the prior knowledge of the signal in the space domain and the prior knowledge in the frequency domain, enabling the KDFNet model to integrate the sensorimotor rhythm and the event-related desynchronization and synchronization patterns, and enabling the training of a decoding model with high generalization performance and high efficiency through fewer training samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flowchart of a method for constructing a decoding model of motor imagery brain-computer interface signals provided by an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of the knowledge source, knowledge integration, and knowledge representation of KDFNet provided by an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of the KDFNet structure provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] In order to make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0027] Embodiment 1

[0028] A method for constructing a decoding model of motor imagery brain-computer interface signals, as Figure 1 shown, includes:

[0029] Construct a knowledge data fusion network KDFNet, including a time-domain convolutional module, a space-domain convolutional module, a feature engineering module, and a classification module. Among them, the time-domain convolutional module contains m time-domain convolutional kernels of size 1×l, where m is the number of frequency band divisions of the motor imagery brain-computer interface signals actually required. Different time-domain convolutional kernels correspond to different frequency bands, and each time-domain convolutional kernel is used to perform time-domain convolution on each channel signal; the space-domain convolutional module contains m*n space-domain convolutional kernels of size C×1. The output of one time-domain convolutional kernel is further processed by n space-domain convolutional kernels for space-domain convolution to obtain features containing frequency-domain knowledge and spatial knowledge corresponding to the frequency band; the feature engineering module is used to calculate the features for characterizing the spectral power for the output of each space-domain convolution and splice the results as the spectral power knowledge features; the classification module is used to perform motor imagery classification based on the spectral power knowledge features.

[0030] Supervised training is performed on the FBCSP model to obtain the parameter values of the classifier in the model; the number of band-pass filters in the FBCSP model, the length of each band-pass filter, and the corresponding frequency band and time-domain convolution kernel; the number of common spatial pattern filters corresponding to each frequency band is the same as the number of spatial-domain convolution kernels corresponding to that frequency band;

[0031] The parameter values of the classifier are used to initialize the parameters of the fully connected layer in the classification module, so that KDFNet can obtain classification knowledge; supervised training is performed on KDFNet to obtain additional knowledge from the training data, and the construction of the decoding model for motor imagery brain-computer interface signals is completed.

[0032] This embodiment is a method for the deep fusion of CSP and CNN, which designs a CNN model enhanced by CSP and a CNN initialization method introducing expert knowledge, and is an efficient decoding method that deeply combines expert knowledge and neural networks in MI classification of brain-computer interfaces.

[0033] As a preferred implementation, the feature engineering module is specifically used to calculate the logarithmic variance features of the features output by each spatial-domain convolution and splice all the logarithmic variances as the spectral power knowledge features.

[0034] As a preferred implementation, the parameter values of each common spatial pattern filter are also used to initialize the parameters of the corresponding spatial-domain convolution kernel, so that KDFNet can obtain spatial-domain prior knowledge.

[0035] As a preferred implementation, the parameter values of each band-pass filter are used to initialize the parameters of the corresponding frequency-domain convolution kernel, so that KDFNet can obtain frequency-domain prior knowledge.

[0036] Combined Figure 2 Introduce the knowledge source, knowledge fusion and knowledge representation of KDFNet.

[0037] The knowledge source of KDFNet is the knowledge-driven traditional machine learning process in MI classification, that is, the classic FBCSP process. Each module uses the corresponding prior knowledge:

[0038] Time-domain filtering stage: Provide prior knowledge of ERD / ERS frequency distribution through multiple band-pass filters.

[0039] Frequency-domain filtering stage: Use CSP filtering to provide prior knowledge of ERD / ERS spatial distribution in each frequency band.

[0040] Feature engineering stage: Extract energy-based logarithmic variance features from the data after spatial-domain filtering.

[0041] Classification stage: Use an interpretable traditional classifier to classify features.

[0042] For the integration of knowledge, the overall structure of KDFNet is consistent with the knowledge-driven FBCSP process, as Figure 3 shown. It mainly consists of four parts: a time-domain convolution module, a spatial-domain convolution module, a feature engineering module, and a classification module. In a specific implementation, time-domain and spatial-domain filtering are implemented using convolutional layers, and prior knowledge is integrated by initializing the time-domain and spatial-domain convolution kernels with a band-pass filter and a CSP filter, respectively. In the feature engineering module, KDFNet uses the calculation of logarithmic variance along the time dimension to incorporate prior knowledge of power-spectrum-related features. The classification module consists of a fully connected layer and a softmax layer for classification, and prior knowledge is integrated by initializing the parameters of the fully connected layer to those in a traditional classifier.

[0043] For the representation of knowledge in KDFNet, a specific implementation:

[0044] (a) Encode frequency-domain knowledge into the time convolution kernel.

[0045] (b) Spatial-domain knowledge is reflected in the spatial convolution kernel.

[0046] (c) Feature-related knowledge is represented by the calculation of logarithmic variance.

[0047] (d) The knowledge of the classifier is embodied in the parameters of the fully connected layer.

[0048] Therefore, the construction of KDFNet mainly includes two steps: 1) Obtain prior knowledge from the traditional FBCSP process; 2) Construct and train the network.

[0049] Next, the structures of each module will be introduced separately.

[0050] 1. Time-domain convolution module

[0051] To integrate frequency-domain knowledge, according to the time-domain convolution theorem, time convolution is a natural method for band-pass filtering. A fixed-size CNN convolution kernel slides over the input signal, and the output is calculated through dot product operations, which is similar to FIR filtering. Therefore, by embedding a band-pass FIR filter in the time convolution layer of KDFNet, the frequency characteristics of EEG signals can be effectively captured. Considering that ERD / ERS mainly occurs in the range of 8 - 32 Hz, similar to FBCSP, multiple convolution kernels are used to extract information in different frequency bands. As Figure 3As shown, the frequency range is divided into sub-bands with a width of 4 Hz, resulting in m = 6 temporal convolutional kernels. The size of each convolutional kernel is 1×l, and each kernel corresponds to a feature map. Therefore, the output data format of the input is m×C×(T - l + 1), where C is the number of channels of the EEG signal and T is the number of time sampling points of the EEG signal for each channel. To accelerate convergence and enhance generalization, two-dimensional batch normalization is applied along the feature map dimension.

[0052] 2. Spatial Convolution Module

[0053] The spatial convolution module enables the model to have the ability to capture spatial discriminative features, which is very important for MI classification. This module uses depth convolution, where each feature map filtered in the time domain is processed by a different spatial filter. To integrate prior knowledge, we first train CSP filters on m frequency bands on the training data. n CSP filters are obtained for each frequency band, and then we use them to initialize the spatial convolutional kernels. This initialization ensures that the spatial convolutional kernels start from a good position and can be further optimized during the training process to better fit the data, while leveraging prior knowledge and data.

[0054] 3. Feature Engineering Module

[0055] In most CNN networks, classical activation functions such as GELU and ELU are used for non-linear transformation, and average pooling or max pooling is used to reduce the feature dimension for classification. However, different MI (motor imagery) classes have different spectral power patterns. Operations based on variance, which capture the spectral power of time series, may be more suitable for MI classification. Therefore, we use a log variance layer that calculates the log variance of the time series to extract energy-based discriminative features (spectral power knowledge feature output). After calculating the log variance features of the feature output containing frequency domain knowledge and spatial domain knowledge (by calculating the log variance of the feature maps after temporal convolution and spatial convolution to obtain the log variance features), we obtain m×n features, and then concatenate these features and process them using one-dimensional batch normalization.

[0056] 4. Classification Module

[0057] The features extracted by the log variance layer are input into a fully connected layer for classification. The fully connected layer has m×n input units (feature dimension) and k output units (number of classification categories). Usually, the parameters of the fully connected layer are randomly initialized. To [reason not provided in the original, so it's hard to translate accurately here], we use the parameters of the traditional classifier in the FBCSP (filter bank common spatial pattern) process to initialize them, so that the classification layer has a better start. Finally, the output of the fully connected layer is passed through a softmax layer to obtain class probabilities.

[0058] In this embodiment, KDFNet includes two supervised training stages: the traditional FBCSP classification learning stage and the network learning stage. In the traditional FBCSP classification process, first, m FIR filters with a length of l in the frequency band are calculated. Then, n CSP filters are trained on the frequency-filtered signals of each frequency band. The spatially filtered signals are used to calculate the log-variance eigenvectors as the input for training the LR classifier. In the KDFNet training stage, the convolutional kernels in the time-domain and spatial-domain filtering blocks are initialized with FIR and CSP filters respectively. The parameters of the fully connected layer are initialized with the parameters in the LR classifier. Then, KDFNet is trained by minimizing the cross-entropy loss so that it can learn from the data and supplement the prior knowledge.

[0059] Experimental verification shows that on the MI dataset, the method proposed in the present invention has a higher classification accuracy compared with traditional classifications (CSP-LR, FBCLP-LR, MDRM) and deep models (without prior knowledge: CNN+LSTM, HybridNet, LMDA-Net, EEGConformer, with prior knowledge: EEGNet, ShallowCNN, FBCNet, CSP-Retraining, FBCSP-Retraining). As shown in Table 1, Table 1 shows the classification accuracies of different methods on different users of the MI dataset. The highest result for each user is bolded, and the second-highest result is underlined.

[0060] Table 1

[0061]

[0062] In summary, the present invention proposes KDFNet, which integrates knowledge-driven and data-driven methods into EEG-based MI classification. By using a band-pass FIR filter to initialize the time convolutional kernel, the prior knowledge of the sensorimotor rhythm frequency band is combined. Similarly, the spatial distribution knowledge of ERD and ERS is used to initialize the spatial convolutional kernel with a CSP filter. Then, differentiable log-variance calculation is applied to capture the power spectrum information. In addition, the parameters of the traditional LR classifier trained on these knowledge-based features are used to initialize the parameters of the KDFNet classification layer. Finally, KDFNet is fine-tuned on the labeled training data.

[0063] In this embodiment, KDFNet combines the prior knowledge from different stages of the motor imagery brain-computer interface system. In the time-domain filtering stage, the temporal convolutional kernel is initialized by a band-pass finite impulse response (FIR) filter, incorporating frequency knowledge into it. In the spatial-domain filtering stage, the common spatial pattern (CSP) filter is used to initialize the spatial convolutional kernel, integrating spatial knowledge. For feature engineering, the knowledge of handcrafted features is introduced through the calculation of the power spectrum based on logarithmic variance. Finally, in the classification stage, the parameters of the classification layer are initialized with a traditional classifier. After embedding the prior knowledge, KDFNet further performs data-driven end-to-end optimization on the training EEG. In summary, KDFNet integrates prior knowledge in multiple stages and adapts the data, thus obtaining better MI brain-computer decoding performance.

[0064] It should be noted that several preferred methods such as logarithmic variance features, spatial-domain prior knowledge, and frequency-domain prior knowledge can be combined arbitrarily and then jointly applied with classification knowledge to construct the decoding model.

[0065] Embodiment 2

[0066] A decoding method for motor imagery brain-computer interface signals uses the decoding model constructed by the above-mentioned decoding model construction method to achieve decoding.

[0067] The related technical solutions are the same as those in Embodiment 1 and will not be elaborated here.

[0068] Embodiment 3

[0069] This application also relates to an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0070] The electronic device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The so-called processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The memory can be used to store computer programs and / or modules. The processor, by running or executing the computer programs and / or modules stored in the memory, and by calling the data stored in the memory, realizes various functions of the electronic device.

[0071] The related technical solutions are the same as above and will not be elaborated here.

[0072] Embodiment 4

[0073] This application also relates to a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0074] Specifically, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

[0075] The related technical solutions are the same as above and will not be elaborated here.

[0076] Embodiment 5

[0077] The embodiment of this application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps of the method in the above embodiment of this application.

[0078] The related technical solutions are the same as above and will not be elaborated here.

[0079] Those skilled in the art can easily understand that the above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a decoding model for motor imagery brain-computer interface signals, characterized in that: include: A knowledge data fusion network KDFNet is constructed, including a time domain convolution module, a spatial domain convolution module, a feature engineering module and a classification module, wherein the time domain convolution module contains m time domain convolution kernels of size 1×1, m is the number of frequency band divisions of the motor imagery brain-computer interface signal actually required, different time domain convolution kernels correspond to different frequency bands, and each time domain convolution kernel is used to perform time domain convolution on each channel signal; the spatial domain convolution module contains m*n spatial domain convolution kernels of size C×1, and the output of a time domain convolution kernel is further processed by spatial domain convolution by n spatial domain convolution kernels to obtain the features of the corresponding frequency band containing frequency domain knowledge and spatial knowledge; the feature engineering module is used to calculate the features for characterizing the spectral power of the output of each spatial domain convolution and splice the results as spectral power knowledge features; the classification module is used to classify motor imagery based on spectral power knowledge features; The FBCSP model is supervisedly trained to obtain the parameter values ​​of the classifier in the model; the number of bandpass filters in the FBCSP model, the length of each bandpass filter and the corresponding frequency band and the simultaneous domain convolution kernel; the number of co-spatial pattern filters corresponding to each frequency band is the same as the number of spatial domain convolution kernels corresponding to the frequency band; the parameters of the fully connected layer in the classification module are initialized using the parameter values ​​of the classifier so that KDFNet obtains classification knowledge; KDFNet is supervisedly trained to obtain additional knowledge from the training data and complete the construction of the decoding model of the motor imagery brain-computer interface signal.

2. The decoding model construction method according to claim 1, characterized in that: The feature engineering module is specifically used to calculate the logarithmic variance feature of the feature output by each spatial domain convolution and concatenate all the logarithmic variances as the spectral power knowledge feature.

3. The decoding model construction method according to claim 1, characterized in that: The parameter values ​​of the common spatial mode filters in the FBCSP model are also obtained, and the parameter values ​​of each common spatial mode filter are used to initialize the parameters of the corresponding spatial convolution kernel, so that KDFNet can obtain spatial domain prior knowledge.

4. The decoding model construction method according to any one of claims 1 to 3, characterized in that: The parameter values ​​of the bandpass filters in the FBCSP model are also obtained, and the parameter values ​​of each bandpass filter are used to initialize the parameters of the corresponding frequency domain convolution kernel, so that KDFNet can obtain frequency domain prior knowledge.

5. A method for decoding motor imagery brain-computer interface signals, characterized in that: Decoding is achieved by using a decoding model constructed by the decoding model construction method as described in any one of claims 1 to 4.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 and / or the steps of the method according to claim 5 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the steps of the method according to any one of claims 1 to 4 and / or the steps of the method according to claim 5.

8. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 4 and / or the steps of the method according to claim 5 are implemented.

Images