A finger movement imagery decoding method based on channel-adaptive convolution kernel and information fusion network

Abstract

The application discloses a finger motor imagery decoding method based on a channel adaptive convolution kernel and an information fusion network. The application introduces a channel adaptive kernel module CAK to cope with the channel difference problem of the convolution kernel, adopts two branches to solve the difference problem existing between different electrode channels, one is an EEG-CAK branch taking original EEG signals as input, and the other is a COV-CAK branch taking a covariance matrix as input. The feature vectors output by the EEG-CAK and COV-CAK branches are spliced into one feature vector; the feature vector passes through two fully connected layers, and the number of neurons in the second layer is equal to the category number of finger motor imagery. The CAK module skillfully determines effective convolution kernels for all channels in the CNN framework at the same time, and fuses the information extracted by them. In addition, the CAK is a general module and can be applied to other EEG tasks. The decoding scheme for finger motor imagery disclosed by the application achieves the best decoding accuracy at present.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence-brain-computer interface, and particularly relates to a finger motor imagery decoding method, in particular to a finger motor imagery decoding method based on channel adaptive convolution kernel and information fusion network. BACKGROUND

[0002] The brain-computer interface based on motor imagery has important application prospects, such as rehabilitation training. The complex function of fingers is an important feature that distinguishes humans from other animals. Therefore, the finger motor imagery decoding method is very important for practical applications such as rehabilitation training.

[0003] The method based on deep learning can automatically extract deep features from electroencephalogram (EEG) signals, so it is paid much attention in motor imagery decoding. Convolutional neural network (CNN) is a commonly used deep learning model. Most of the methods based on CNN use time-domain convolution structure, that is, a convolution kernel with a size of '1*Kernel_Size' is used to convolve the EEG signal, but the motor imagery decoding accuracy realized by the above method is not particularly high.

[0004] Therefore, the application proposes a finger motor imagery decoding method based on channel adaptive convolution kernel and information fusion network. By introducing a channel adaptive kernel (CAK) module, effective convolution kernels are determined for all channels in the framework of CNN, and the information extracted by them is fused. The application pays attention to the channel difference problem of the convolution kernel, and different effective convolution kernels (i.e. channel-dependent convolution kernels) are adapted through different channels to fully utilize the differences between channels. SUMMARY

[0005] The application aims at the deficiencies of the prior art, and provides a finger motor imagery decoding method based on channel adaptive convolution kernel and information fusion network.

[0006] In a first aspect, the application provides a finger motor imagery decoding method based on channel adaptive convolution kernel and information fusion network, which specifically includes the following steps:

[0007] Step S1, acquiring EEG data of finger motor imagery, and pre-processing the EEG data to obtain a multi-channel EEG signal , wherein C is the number of electrode channels of EEG, and T is the number of sampling points; then calculating the multi-channel EEG signal Covariance matrix ;

[0008] Step S2, constructing an EEG channel adaptive kernel branch (i.e. EEG-CAK branch) with the multi-channel EEG signal The depth information is extracted as input;

[0009] The EEG-CAK branch comprises a first CAK module, a first spatial convolution module, a first average pooling module, a deep convolution module, a point convolution module, a second average pooling module and a first Flatten layer connected in sequence.

[0010] The first CAK module identifies effective convolution kernels for each channel of the multi-channel EEG signal and assigns weights; then multiplies the feature maps corresponding to different convolution kernels of each channel by the respective weights and adds them to obtain a mixed feature map of multiple channels.

[0011] The first spatial convolution module compresses the mixed feature map of multiple channels output by the first CAK module into one channel to summarize the information of multiple channels.

[0012] The first average pooling module is used to compress the feature map output by the first spatial convolution module to reduce the amount of calculation.

[0013] The deep convolution module extracts deep-level information of the feature map output by the first average pooling module while reducing the amount of calculation.

[0014] The point convolution module performs point convolution on the feature map output by the deep convolution module.

[0015] The second average pooling module is used to compress the feature map output by the point convolution module again.

[0016] The first Flatten layer expands the feature map output by the second average pooling module to form a feature vector.

[0017] Step S3, a covariance channel adaptive kernel branch (i.e., COV-CAK branch) is constructed, and the covariance matrix of the multi-channel EEG signal is input to extract potential effective inter-channel structural relationship information;

[0018] The COV-CAK branch comprises a second CAK module, a second spatial convolution module, a third average pooling module and a second Flatten layer connected in sequence.

[0019] The second CAK module identifies effective convolution kernels for each channel of the input and assigns weights; then multiplies the feature maps corresponding to different convolution kernels of each channel by the respective weights and adds them to obtain a mixed feature map of multiple channels.

[0020] ​The second spatial domain convolution module compresses the mixed feature maps of multiple channels output by the second CAK module into one channel to summarize the information of multiple channels.

[0021] The third average pooling module is used for compressing the feature maps output by the second spatial domain convolution module to reduce the amount of calculation.

[0022] The second Flatten layer expands the feature maps output by the third average pooling module to form a feature vector.

[0023] In step S4, the feature vectors output by the EEG-CAK branch and the COV-CAK branch are spliced, and then sequentially pass through two serially connected fully connected layers and a softmax function to obtain a finger motor imagery classification result.

[0024] In a second aspect, the present application provides a finger motor imagery decoding system, comprising:

[0025] A data acquisition module acquires EEG data of finger motor imagery, and pre-processes the EEG data to obtain a multi-channel EEG signal Then, the multi-channel EEG signal A covariance matrix is calculated ;

[0026] An EEG-CAK branch module extracts information from the multi-channel EEG signal ;

[0027] A COV-CAK branch module extracts potential effective inter-channel structural relationship information from the covariance matrix of the multi-channel EEG signal ;

[0028] A classification module splices the output results of the EEG-CAK branch module and the COV-CAK branch module, and then sequentially passes through two serially connected fully connected layers and a softmax function to obtain a finger motor imagery classification result.

[0029] In a third aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein when the computer program is executed in a computer, the computer executes the method.

[0030] In a fourth aspect, the present application provides a computing device comprising a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method is implemented.

[0031] The present application has the following beneficial effects:

[0032] The application introduces a CAK module in the framework of a convolutional neural network (CNN), adaptively identifies effective convolution kernels for each channel, and further aggregates the features extracted by the effective convolution kernels, solves the channel-dependent convolution kernel problem in EEG signal decoding, significantly improves the ability of the model to learn and process the relationship between channels, especially the relationship between distant channels, and can more accurately decode the EEG signals of finger motor imagery, and improve the accuracy of decoding.

[0033] In summary, the application is the first attempt to apply the channel adaptive kernel concept to EEG signal decoding, especially in dealing with channel-dependent convolution kernels, providing a completely new solution. By adapting different effective convolution kernels for different channels, the application makes full use of the differences between channels and enhances the feature extraction capability of the model; at the same time, the effective convolution kernels are determined for all channels, reducing the computational load and improving the training and inference efficiency of the model.

[0034] The method of the application is not only suitable for finger motor imagery decoding, but also can be extended to other application scenarios that need to decode EEG signals, and has wide applicability and potential clinical application value. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the application, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0036] Figure 1 The module flowchart of the method of the application is shown in the figure.

[0037] Figure 2 The structure diagram of the channel adaptive kernel module is shown in the figure. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application.

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the application clearer, the technical solutions of the application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0040] The terms "comprising" and "having" and any variations thereof herein are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or apparatus that comprises a list of steps or units is not necessarily limited to the listed steps or units, but can optionally include additional steps or units not expressly listed or can also include steps or units inherent to such process, method, product or apparatus.

[0041] The embodiment of the present application provides a finger motor imagery decoding method based on a channel adaptive convolution kernel and an information fusion network.

[0042] A key of the convolutional neural network is a kernel-based convolution operation. Due to the limited range of the convolution operation, although the receptive field can be expanded, actually the CNN has weak ability to learn the relationship between channels. Therefore, a challenge problem of the CNN is to learn and process the relationship between channels, especially the relationship between long-distance channels. In fact, the relationship information between channels is very valuable for decoding the EEG signal.

[0043] For the channel difference problem of the convolution kernel, the present application introduces a channel adaptive kernel module, which skillfully determines effective convolution kernels for all channels in the framework of the CNN and fuses the information extracted by them. In addition, the CAK is a general module and can be applied to other EEG tasks.

[0044] In order to make the technical scheme and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings. As shown in the drawings, Figure 1 The present application discloses a finger motor imagery decoding method based on a channel adaptive convolution kernel and an information fusion network, comprising the following steps:

[0045] Step 1, acquiring the EEG data of finger motor imagery, and pre-processing the EEG data to obtain a multi-channel EEG signal , wherein C is the number of electrode channels of the EEG, and T is the number of sampling points; then calculating the covariance matrix of the multi-channel EEG signal .

[0046] Step 2, constructing an EEG-CAK branch, taking the multi-channel EEG signal as input, extracting information, including:

[0047] The EEG-CAK branch comprises a first CAK module, a first spatial convolution module, a first average pooling module, a deep convolution module, a point convolution module, a second average pooling module and a first Flatten layer connected in series.​

[0048] The first CAK module is a multi-channel EEG signal Each channel of the CAK module identifies the effective convolution kernel and assigns weights (i.e., calculates weights for candidate convolution kernels of different sizes); then multiplies the feature maps corresponding to different convolution kernels of each channel by their respective weights and adds them together to obtain a mixed feature map of multiple channels. The specific execution process of the CAK module in this branch is based on For three candidate convolution kernel sizes, perform Split operation, Fuse operation and Select operation respectively (such as Figure 2 shown).

[0049] The first spatial convolution module compresses the mixed feature maps of multiple channels output by the first CAK module into one channel to aggregate the information of multiple channels. Specifically, the module uses 16 C×1 convolution kernels to compress the data of C channels into one channel.

[0050] The first average pooling module is used to compress the feature map output by the first spatial convolution module to reduce the amount of calculation, where the size of the pooling kernel is .

[0051] The depth convolution module extracts deep information from the feature map output by the first average pooling module while reducing the amount of computation. This module performs depth convolution on the feature map using 16 1×16 convolution kernels.

[0052] The point convolution module performs point convolution on the feature map output by the depth convolution module. This module is implemented using 16 1×1 convolution kernels.

[0053] The second average pooling module is used to re-compress the feature map output by the point convolution module, where the size of the pooling kernel is .

[0054] The first Flatten layer expands the feature map output by the second average pooling module to form a feature vector with a length of 160.

[0055] In order to avoid overfitting during training, the Dropout operation is used after each average pooling, randomly dropping 50% of the features.

[0056] Step 3: Construct the COV-CAK branch to use multi-channel EEG signals It uses the covariance matrix as input to mine the potential effective inter-channel structural relationship contained in the covariance matrix to make up for the shortcomings of the EEG-CAK branch.

[0057] The COV-CAK branch includes a second CAK module, a second spatial convolution module, a third average pooling module, and a second Flatten layer connected in series;

[0058] The second CAK module is input Each channel of the CAK module identifies the effective convolution kernel and assigns weights; then the feature maps corresponding to the different convolution kernels of each channel are multiplied by their respective weights and added together to obtain a mixed feature map of multiple channels. The specific execution process of the CAK module in this branch is based on For three candidate convolution kernel sizes, perform Split operation, Fuse operation and Select operation respectively (such as Figure 2 shown).

[0059] The second spatial convolution module compresses the mixed feature maps of multiple channels output by the second CAK module into one channel to aggregate the information of multiple channels. Specifically, the module uses 10 convolution kernels of size C×1 to compress the information of multiple channels.

[0060] The third average pooling module is used to compress the feature map output by the second spatial convolution module to reduce the amount of calculation. The size of the pooling kernel is Similarly, Dropout is used after average pooling to prevent overfitting.

[0061] The second Flatten layer expands the feature map output by the third average pooling module to form a feature vector with a length of 20.

[0062] In step 4, the feature vectors of length 160 and 20 output by the EEG-CAK branch and the COV-CAK branch are concatenated to obtain a feature vector of length 180. This is then passed through two serially connected fully connected layers and a softmax function to obtain the finger movement imagery classification results. The first of the two serially connected layers contains 10 neurons, and the second contains 5 neurons, corresponding to the five finger categories.

[0063] The first CAK module and the second CAK module have the same structure, both including Split operation, Fuse operation and Select operation, such as Figure 2 As shown; specifically:

[0064] (1) Split operation:

[0065] Assume that the input data is , C is the number of channels, M=X or , that is, when the EEG signal is input, N is the number of sampling points T, and when the covariance matrix is ​​input, N is equal to the number of channels C.

[0066] The split operation is to pass the input through parallel first, second and third time domain convolutions to obtain three groups of feature maps , i = 1, 2, 3;

[0067] The first, second and third time domain convolutions have different kernel sizes.

[0068] Each time domain convolution uses a padding operation to ensure that each convolution obtains feature maps of equal size, and is processed by Batch Normalization and a ReLU function.

[0069] The feature map set is represented as follows:

[0070]

[0071] wherein represents a convolution operation, and represents the size of the convolution kernel.

[0072] (2) Fuse operation:

[0073] The three groups of feature maps obtained by the split operation are added element by element to obtain a feature map set U.

[0074]

[0075] Then each feature map in U is wherein F represents the total number of feature maps in each group, and the channel is averaged and pooled to obtain the collection information of each channel in .

[0076]

[0077] wherein represents the tth data point of channel c in feature map .

[0078] Each feature map can generate a feature vector ; the feature vectors compose the feature map .

[0079] (3) Select operation, extracts the weight of different convolution kernels on each channel from the feature map S:

[0080] The Select operation includes three parallel branches with the same structure but different initialization parameters; each branch first uses a 1x1 convolution kernel to perform point convolution on S, compressing the information on F feature maps in S to one feature map, and then passes through two fully connected layers to obtain weight information .

[0081]

[0082] Its denotes a point convolution operation, denote the weight parameters of the two fully connected layers, respectively.

[0083] The weight information is updated adaptively during network training , and is standardized using a softmax function. Finally, the weight vector .

[0084]

[0085] where denotes the natural exponential function;

[0086] The weight vector is multiplied element by element with the corresponding feature map and summed to obtain the fused feature map set .

[0087]

[0088] Through these three steps, CAK can adaptively select an effective convolution kernel size for each channel and further fuse the signals extracted by the effective convolution kernel of each channel. In other words, CAK selects an effective kernel for each channel from the candidate convolution kernel and assigns a weight, and then fuses.

[0089] To further verify the effectiveness and superiority of the present application, the publicly available 5-finger motor imagery dataset [1] was used as experimental data. The experiment used a subject-dependent mode, and each session was individually subjected to 10-fold cross-validation experiments. All samples in each session were randomly divided into approximately equal 10 parts, 90% of the data was used to train the model each time, and the remaining 10% of the data was used for testing. Finally, the average decoding accuracy of 10 rounds was calculated.

[0090] The parameters of model training in the experiment are as follows: the batch size is set to 100, the learning rate is set to 0.005, the training epoch is set to 500, and the Adam optimizer is used. The software configuration of the experiment is Python 3.10, Pytorch 2.0.0, the hardware configuration is Intel Core i7-12700H 2.3GHz CPU, 16 GB RAM, and NVIDIA RTX 3060 GPU.

[0091] The decoding accuracy of the method of the present application and the existing decoding progress results on the same data [1,2,3] , and the decoding accuracy of the reproduced known deep learning model EEGNet [4] , AMCNN-DGCN [5] , FBNCNN [6] , MBEEGNet [7] and MBK-CNN [8] are shown in the table. The details of each method are referred to the corresponding literature. MBEEGNet [7] and MBK-CNN [8] come from motor imagery classification, AMCNN-DGCN [5] and FBN-CNN [6] come from fatigue detection.

[0092] Table 1 Comparison of decoding accuracy with other methods (%)

[0093] Trial of the subject [1] [2] AMCNN-DGCN FBNCNN [3] EEGNet MBEEGNet MBK-CNN The method A1 # 59.3 49.43 51.31 64.57 71.11 70.07 77.06 76.22 A2 # 20.8 26.64 28.60 20.27 18.57 26.76 27.46 28.25 B1 # 43.1 33.92 46.02 45.84 46.04 47.82 48.75 50.53 B2 # 42.5 42.57 45.67 44.26 52.72 55.94 51.02 57.10 B3 # 44.1 37.95 45.89 45.50 48.49 50.67 47.96 53.28 B4 # 53.9 45.05 46.08 55.66 60.79 59.54 57.35 62.35 C1 # 61.4 43.11 50.41 59.22 69.63 70.14 71.71 69.42 C2 # 61.7 46.32 49.52 63.11 72.26 71.52 72.15 76.20 E1 # 52.3 46.54 50.52 56.90 63.81 66.73 65.28 70.18 E2 # 66.5 51.26 50.10 68.28 76.97 76.87 77.81 80.23 E3 # 53.0 42.61 51.38 54.86 67.03 66.08 66.80 68.81 F1 # 38.6 39.30 49.54 55.50 61.45 59.56 64.27 66.55 F2 # 34.1 32.37 48.62 45.49 46.77 50.21 51.14 52.19 F3 # 45.4 38.58 50.26 55.56 62.46 62.67 59.03 64.13 G1 # 41.6 39.01 47.66 44.79 56.24 55.08 53.48 57.30 G2 # 42.4 39.99 49.94 44.29 51.30 54.54 50.88 57.07 H1 # 34.0 33.19 51.29 34.90 39.80 42.41 42.50 45.45 I1 # 51.4 46.41 50.06 55.05 59.44 63.41 66.95 65.17 I2 # 41.7 41.97 48.32 48.21 52.29 55.84 60.13 59.92 Mean 43 46.7 40.85 47.96 50.64 56.69 58.20 58.51 61.07 Standard deviation 10.0 11.0 6.10 4.93 10.88 13.28 11.46 12.40 11.95

[0094] The comparison results show that the decoding accuracy of the method of the present application is 61.07%, which is better than the best result of 50.64% [3] on the same data. Compared with FBN-CNN which takes PLI functional network (not from original EEG) as input, the method of the present application improves by 13.11%. The average decoding accuracy of the method of the present application is improved by 4.38%, 2.87% and 2.56% compared with EEGNet, MBEEGNet and MBK-CNN, respectively. Two-sample paired T test shows that the difference is significant (P<0.05).

[0095] [1]M. Kaya, M. K. Binli, E. Ozbay et al. A largeelectroencephalographic motor imagery dataset for electroencephalographicbrain computer interfaces, Scientific data, vol. 5, pp. 1-16, 2018.

[0096] [2] R. N. Azizah, H. Zakaria, and B. R. Hermanto. Channels selectionfor pattern recognition of five fingers motor imagery electroencephalographysignals, Journal of Physics: Conference Series, vol. 2312, no. 1, p. 012019,2022.

[0097] [3] K. Yang, R. Li, J. Xu, L. Zhu, W. Kong, and J. Zhang. DSFE:Decoding eeg-based finger motor imagery using feature-dependent frequency,feature fusion and ensemble learning, IEEE Journal of Biomedical and HealthInformatics, vol. 28(8), pp. 4625-4635, 2024.

[0098] [4] V. J. Lawhern, A. J. Solon, N. R. Waytowich et al. EEGNet: acompact convolutional neural network for eeg-based brain–computer interfaces,Journal of neural engineering, vol. 15, no. 5, p. 056013, 2018.

[0099] [5] H. Wang, L. Xu, and et al. Linking attention-based multiscale CNN with dynamical GCN for driving fatigue detection, IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-11, 2021.

[0100] [6] J. Chen, S. Wang, E. He, and et al. Two-dimensional phase lag index image representation of electroencephalography for automated recognition of driver fatigue using convolutional neural network, Expert Systems with Applications, vol. 191, p. 116339, 2022.

[0101] [7] G. A. Altuwaijri and G. Muhammad. A multibranch of convolutional neural network models for electroencephalogram-based motor imagery classification, Biosensors, vol. 12, no. 1, p. 22, 2022.

[0102] [8] J. Shin and W. Chung. Multi-band CNN with band-dependent kernels and amalgamated cross entropy loss for motor imagery classification, IEEE Journal of Biomedical and Health Informatics, vol. 27, no. 9, pp. 4466-4477, 2023.

[0103] The above is the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can also make a number of improvements and refinements, these improvements and refinements are also considered to be within the scope of the present application.

Claims

1. A finger movement imagery decoding method based on channel-adaptive convolution kernel and information fusion network, characterized in that The specific steps include: Step S1: Obtain EEG data of finger movement imagination and preprocess the EEG data to obtain multi-channel EEG signals , where C is the number of EEG electrode channels and T is the number of sampling points; then for the multi-channel EEG signal Calculate the covariance matrix ; Step S2: Construct EEG channel adaptive kernel branch to multi-channel EEG signal As input, depth information is extracted; the EEG channel adaptive kernel branch includes a first CAK module, a first spatial convolution module, a first average pooling module, a depth convolution module, a point convolution module, a second average pooling module, and a first Flatten layer connected in series in sequence; Step S3: Construct a covariance channel adaptive kernel branch, using the covariance matrix of the multi-channel EEG signal As input, it extracts potentially effective inter-channel structural relationship information; the covariance channel adaptive kernel branch includes a second CAK module, a second spatial convolution module, a third average pooling module, and a second Flatten layer connected in series in sequence; Step S4: concatenate the feature vectors output by the EEG channel adaptive kernel branch and the covariance channel adaptive kernel branch, and then pass them through two series-connected fully connected layers and a softmax function to obtain the finger movement imagery classification result.

2. The method according to claim 1, characterized in that The first CAK module is a multi-channel EEG signal Identify the effective convolution kernel for each channel and assign weights; then multiply the feature maps corresponding to different convolution kernels of each channel by their respective weights and add them together to obtain a mixed feature map of multiple channels; The first spatial convolution module compresses the mixed feature maps of multiple channels output by the first CAK module into one channel to summarize the information of the multiple channels; The first average pooling module is used to compress the feature map output by the first spatial convolution module to reduce the amount of calculation; The depth convolution module extracts the deep information of the feature map output by the first average pooling module while reducing the amount of calculation; The point convolution module performs point convolution on the feature map output by the depth convolution module; The second average pooling module is used to re-compress the feature map output by the point convolution module; The first Flatten layer expands the feature map output by the second average pooling module to form a feature vector.

3. The method according to claim 1, characterized in that The second CAK module is input Identify the effective convolution kernel for each channel and assign weights; then multiply the feature maps corresponding to different convolution kernels of each channel by their respective weights and add them together to obtain a mixed feature map of multiple channels; The second spatial convolution module compresses the mixed feature maps of multiple channels output by the second CAK module into one channel to summarize the information of the multiple channels; The third average pooling module is used to compress the feature map output by the second spatial convolution module to reduce the amount of calculation; The second Flatten layer expands the feature map output by the third average pooling module to form a feature vector.

4. The method according to claim 1, characterized in that The first CAK module and the second CAK module have the same structure, both including Split operation, Fuse operation and Select operation, specifically: (1) Split operation: Assume that the input data is , C is the number of channels; when input M=X, N=T, when input When N=C, the input data M is subjected to the first time domain convolution, the second time domain convolution, and the third time domain convolution in parallel to obtain three sets of feature maps. , i=1,2,3; The convolution kernel sizes of the first time domain convolution, the second time domain convolution, and the third time domain convolution are different; (2) Fuse operation: Add the three sets of feature maps obtained by the Split operation element by element to obtain the feature map set U; Then each feature map in U , F represents the total number of feature maps in each group, and the average pooling is performed by channel to obtain Aggregate information for each channel ; Each feature map produces a feature vector ; Eigenvector Composition feature map ; (3) Select operation: Extract the weights of different convolution kernels on each channel from the feature map S.

5. The method according to claim 4, characterized in that: The Select operation includes three parallel branches with the same structure but different initialization parameters; each branch first uses a 1×1 convolution kernel to perform point convolution on the feature map S, compressing the information on F feature maps in the feature map S into one feature map, and then passes through two fully connected layers to obtain weight information. ; Adaptively update weight information during network training , and use the softmax function for normalization; finally, the weight vector ; ； in represents the natural exponential function; The weight vector And the corresponding feature map Multiply and sum element by element to get a fused feature map set ; 。 6. A finger movement imagery decoding system implementing the method according to any one of claims 1 to 5, characterized in that include: Data acquisition module, which acquires EEG data of finger movement imagination and preprocesses the EEG data to obtain multi-channel EEG signals ; Then multi-channel EEG signal Calculate the covariance matrix ; EEG channel adaptive kernel branch module, multi-channel EEG signal Extract information; Covariance channel adaptive kernel branch module, multi-channel EEG signal covariance matrix Extract potentially effective inter-channel structural relationship information; The classification module splices the output results of the EEG channel adaptive kernel branch module and the covariance channel adaptive kernel branch module, and then passes them through two series-connected fully connected layers and a softmax function to obtain the finger movement imagination classification results.

7. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 5.

8. A computing device comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 5 is implemented.

Images