Finger motor imagery decoding method based on channel adaptive convolution kernel and information fusion network

Abstract

The invention discloses a finger motor imagery decoding method based on a channel adaptive convolution kernel and an information fusion network. A channel adaptive kernel module CAK is introduced to deal with the channel difference problem of a convolution kernel, and two branches are adopted to solve the difference problem between different electrode channels, one branch is an EEG-CAK branch taking an original EEG signal as input, and the other branch is a COV-CAK branch taking a covariance matrix as input. The feature vectors output by the EEG-CAK branch and the COV-CAK branch are spliced into one feature vector; the feature vector passes through two full-connection layers, and the number of neurons in the second layer is equal to the number of categories of finger motor imagery. The CAK module skillfully determines effective convolution kernels for all channels at the same time in a CNN frame, and fuses the extracted information. In addition, the CAK is a general module and can be applied to other EEG tasks. According to the decoding scheme of the finger motor imagery disclosed by the invention, the optimal decoding accuracy at present is obtained.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence-brain-computer interface, and specifically relates to a finger movement imagery decoding method, specifically a finger movement imagery decoding method based on channel adaptive convolution kernel and information fusion network. Background Art

[0002] Brain-computer interfaces based on motor imagery have important application prospects, such as rehabilitation training. The complex functions of fingers are an important feature that distinguishes humans from other animals. Therefore, the solution method of finger motor imagery is very important for practical applications such as rehabilitation training.

[0003] Deep learning-based methods can automatically extract deep features from electroencephalogram (EEG) signals, so they are highly valued in motor imagery solutions. Convolutional neural networks (CNNs) are frequently used deep learning models. Most CNN-based methods use a time-domain convolution structure, that is, they use a convolution kernel of size "1*Kernel_Size" to convolve the EEG signal, but the motor imagery decoding accuracy achieved by the above methods is not particularly high.

[0004] Therefore, the present invention proposes a finger movement imagery decoding method based on channel adaptive convolution kernel and information fusion network. By introducing the Channel Adaptive Kernel (CAK) module, the effective convolution kernels are cleverly determined for all channels at the same time within the framework of CNN, and the information extracted by them is fused. The present invention pays attention to the channel difference problem of convolution kernels, and fully utilizes the differences between channels by adapting different effective convolution kernels (i.e., channel-dependent convolution kernels) to different channels. Summary of the invention

[0005] The purpose of the present invention is to provide a finger movement imagery decoding method based on channel adaptive convolution kernel and information fusion network in view of the deficiencies of the prior art.

[0006] In a first aspect, the present invention provides a finger movement imagery decoding method based on a channel adaptive convolution kernel and an information fusion network, which specifically comprises the following steps: Step S1: Obtain EEG data of finger movement imagination and preprocess the EEG data to obtain a multi-channel EEG signal , where C is the number of EEG electrode channels and T is the number of sampling points; then for multi-channel EEG signals Calculate the covariance matrix ; Step S2: construct an EEG channel adaptive kernel branch (ie, EEG-CAK branch) to As input, extract depth information; The EEG-CAK branch includes a first CAK module, a first spatial convolution module, a first average pooling module, a depth convolution module, a point convolution module, a second average pooling module, and a first Flatten layer which are connected in series in sequence; The first CAK module is a multi-channel EEG signal The effective convolution kernel is identified for each channel and a weight is assigned; then the feature maps corresponding to different convolution kernels of each channel are multiplied by their respective weights and added together to obtain a mixed feature map of multiple channels.

[0007] The first spatial convolution module compresses the mixed feature maps of multiple channels output by the first CAK module into one channel to summarize the information of the multiple channels.

[0008] The first average pooling module is used to compress the feature map output by the first spatial convolution module to reduce the amount of calculation.

[0009] The deep convolution module extracts deep information of the feature map output by the first average pooling module while reducing the amount of calculation.

[0010] The point convolution module performs point convolution on the feature map output by the depth convolution module.

[0011] The second average pooling module is used to re-compress the feature map output by the point convolution module.

[0012] The first Flatten layer expands the feature map output by the second average pooling module to form a feature vector.

[0013] Step S3: construct a covariance channel adaptive kernel branch (i.e., COV-CAK branch) to obtain the covariance matrix of the multi-channel EEG signal. As input, extract potentially effective inter-channel structural relationship information; The COV-CAK branch includes a second CAK module, a second spatial convolution module, a third average pooling module, and a second Flatten layer connected in series in sequence; The second CAK module is input The effective convolution kernel is identified for each channel and a weight is assigned; then the feature maps corresponding to different convolution kernels of each channel are multiplied by their respective weights and added together to obtain a mixed feature map of multiple channels.

[0014] The second spatial convolution module compresses the mixed feature maps of multiple channels output by the second CAK module into one channel to summarize the information of multiple channels.

[0015] The third average pooling module is used to compress the feature map output by the second spatial convolution module to reduce the amount of calculation.

[0016] The second Flatten layer expands the feature map output by the third average pooling module to form a feature vector.

[0017] Step S4: concatenate the feature vectors output by the EEG-CAK branch and the COV-CAK branch, and then pass them through two series-connected fully connected layers and a softmax function to obtain the finger movement imagery classification result.

[0018] In a second aspect, the present invention provides a finger movement imagery decoding system, comprising: Data acquisition module, which acquires EEG data of finger movement imagination and preprocesses the EEG data to obtain multi-channel EEG signals ; Then multi-channel EEG signal Calculate the covariance matrix ; EEG-CAK branch module, multi-channel EEG signal Extract information; COV-CAK branch module, multi-channel EEG signal covariance matrix Extract potentially effective inter-channel structural relationship information; The classification module concatenates the output results of the EEG-CAK branch module and the COV-CAK branch module, and then passes them through two series-connected fully connected layers and a softmax function to obtain the finger movement imagination classification results.

[0019] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method described.

[0020] In a fourth aspect, the present invention provides a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the described method is implemented.

[0021] The present invention has the following beneficial effects: The present invention introduces a CAK module within the framework of a convolutional neural network (CNN), adaptively identifies effective convolution kernels for each channel, and further aggregates the features extracted by the effective convolution kernels, thereby solving the channel-dependent convolution kernel problem in EEG signal decoding, significantly improving the model's ability to learn and process the relationship between channels, especially the relationship between long-distance channels, and being able to more accurately decode the EEG signals of finger movement imagination, thereby improving the accuracy of decoding.

[0022] In summary, the present invention is the first attempt to apply the concept of channel adaptive kernel to EEG signal decoding, especially in dealing with the problem of channel-dependent convolution kernel, providing a new solution. By adapting different effective convolution kernels to different channels, the present invention makes full use of the differences between channels and enhances the feature extraction capability of the model; at the same time, the effective convolution kernel is determined for all channels, which reduces the amount of calculation and improves the training and reasoning efficiency of the model.

[0023] The method of the present invention is not only suitable for decoding finger movement imagery, but can also be extended to other application scenarios that require decoding of EEG signals. It has wide applicability and potential clinical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] FIG1 is a schematic diagram of a module flow chart of the method of the present invention; FIG2 is a schematic diagram of the structure of the channel adaptive core module. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0029] The embodiment of the present invention provides a finger movement imagery decoding method based on channel adaptive convolution kernel and information fusion network. In view of the shortcomings of the prior art, the inventor of this case has proposed the technical solution of the present invention after long-term research and extensive practice, which is mainly based on at least: A key to convolutional neural networks is the kernel-based convolution operation. Since the convolution operation has a limited range, although it can expand the receptive field, CNN is actually weak in learning the relationship between channels. Therefore, a challenging problem faced by CNN is to learn and process the relationship between channels, especially the relationship between long-distance channels. In fact, the relationship information between channels is very valuable for decoding EEG signals.

[0030] For the channel difference problem of convolution kernel, the present invention introduces a channel adaptive kernel module, which cleverly determines effective convolution kernels for all channels at the same time within the framework of CNN and fuses the information extracted by them. In addition, CAK is a general module that can be applied to other EEG tasks.

[0031] In order to make the purpose, technical solution and advantages of the present invention more clear, the present application is further described in detail below with reference to the accompanying drawings. As shown in FIG1 , the present invention discloses a finger movement imagery decoding method based on a channel adaptive convolution kernel and an information fusion network, comprising the following steps:

[0032] Step 1: Obtain EEG data of finger movement imagination and preprocess the EEG data to obtain multi-channel EEG signals , where C is the number of EEG electrode channels and T is the number of sampling points; then for multi-channel EEG signals Calculate the covariance matrix .

[0033] Step 2: Construct the EEG-CAK branch to use multi-channel EEG signals For input, extract information including: The EEG-CAK branch includes a first CAK module, a first spatial convolution module, a first average pooling module, a depth convolution module, a point convolution module, a second average pooling module, and a first Flatten layer which are connected in series in sequence; The first CAK module is a multi-channel EEG signal Identify valid convolution kernels for each channel and assign weights (i.e., calculate weights for candidate convolution kernels of different sizes); then multiply the feature maps corresponding to different convolution kernels of each channel by their respective weights and add them together to obtain a mixed feature map of multiple channels. The specific execution process of the CAK module in this branch is For the three candidate convolution kernel sizes, the Split operation, the Fuse operation and the Select operation are performed respectively (as shown in Figure 2).

[0034] The first spatial convolution module compresses the mixed feature maps of multiple channels output by the first CAK module into one channel to summarize the information of multiple channels. Specifically, the module uses 16 C×1 convolution kernels to compress the data of C channels into 1 channel.

[0035] The first average pooling module is used to compress the feature map output by the first spatial convolution module to reduce the amount of calculation, where the size of the pooling kernel is .

[0036] The deep convolution module extracts the deep information of the feature map output by the first average pooling module, while reducing the amount of calculation. The module uses 16 1×16 convolution kernels to perform deep convolution on the feature map.

[0037] The point convolution module performs point convolution on the feature map output by the depth convolution module. The module is implemented using 16 1×1 convolution kernels.

[0038] The second average pooling module is used to re-compress the feature map output by the point convolution module, where the size of the pooling kernel is .

[0039] The first Flatten layer expands the feature map output by the second average pooling module to form a feature vector with a length of 160.

[0040] In order to avoid overfitting during training, the Dropout operation was used after each average pooling to randomly drop 50% of the features.

[0041] Step 3: Construct the COV-CAK branch to use multi-channel EEG signals It uses the EEG-CAK branch as input to mine the potentially effective inter-channel structural relationship contained in the covariance matrix, making up for the shortcomings of the EEG-CAK branch.

[0042] The COV-CAK branch includes a second CAK module, a second spatial convolution module, a third average pooling module, and a second Flatten layer connected in series in sequence; The second CAK module is input Each channel of the CAK module identifies the effective convolution kernel and assigns weights; then the feature maps corresponding to the different convolution kernels of each channel are multiplied by their respective weights and added together to obtain a mixed feature map of multiple channels. The specific execution process of the CAK module in this branch is For the three candidate convolution kernel sizes, the Split operation, the Fuse operation and the Select operation are performed respectively (as shown in Figure 2).

[0043] The second spatial convolution module compresses the mixed feature maps of multiple channels output by the second CAK module into one channel to summarize the information of multiple channels. Specifically, the module uses 10 convolution kernels of size C×1 to compress the information of multiple channels.

[0044] The third average pooling module is used to compress the feature map output by the second spatial convolution module to reduce the amount of calculation. The size of the pooling kernel is Similarly, Dropout operation is used after average pooling to prevent overfitting.

[0045] The second Flatten layer expands the feature map output by the third average pooling module to form a feature vector with a length of 20.

[0046] Step 4: Concatenate the feature vectors of length 160 and 20 output by the EEG-CAK branch and the COV-CAK branch to obtain a feature vector of length 180. Then, pass through two series-connected fully connected layers and softmax function to obtain the finger movement imagery classification results. The first of the two series-connected fully connected layers contains 10 neurons, and the second fully connected layer contains 5 neurons corresponding to the 5 finger categories.

[0047] The first CAK module and the second CAK module have the same structure, both including Split operation, Fuse operation and Select operation, as shown in FIG2 ; specifically: (1) Split operation: Assume that the input data is , C is the number of channels, M = X or , that is, when the EEG signal is input, N is the number of sampling points T, and when the covariance matrix is ​​input, N is equal to the number of channels C.

[0048] The split operation is to process the input through the first time domain convolution, the second time domain convolution, and the third time domain convolution in parallel to obtain three sets of feature maps. , i=1,2,3; The convolution kernel sizes of the first time domain convolution, the second time domain convolution, and the third time domain convolution are different; Each time domain convolution uses a padding operation to ensure that each convolution obtains feature maps of equal size, and is processed by Batch Normalization and ReLU functions.

[0049] Feature map collection The representation is as follows: in represents the convolution operation, Indicates the size of the convolution kernel.

[0050] (2) Fuse operation: The three groups of feature maps obtained by the Split operation are added element by element to obtain the feature map set U.

[0051] Then each feature map in U , where F represents the total number of feature maps in each group, and average pooling is performed by channel to obtain Aggregate information for each channel in .

[0052] in Representation feature map The t-th data point of channel c.

[0053] Each feature map can generate a feature vector ; Eigenvector Composition feature map .

[0054] (3) Select operation, extract the weights of different convolution kernels on each channel from the feature map S: The Select operation includes three parallel branches with the same structure but different initialization parameters; each branch first uses a 1×1 convolution kernel to perform point convolution on S to compress the information on the F feature maps in S into one feature map, and then passes through two fully connected layers to obtain the weight information .

[0055] That represents the point convolution operation, Represent the weight parameters of the two fully connected layers respectively.

[0056] Adaptively update weight information during network training , and use the softmax function for normalization. Finally, we get the weight vector .

[0057] in represents the natural exponential function; The weight vector And the corresponding feature map Multiply and sum each element to get a fused feature map set .

[0058] Through these three steps, CAK is able to adaptively select the effective convolution kernel size for each channel and further fuse the signals extracted by the effective convolution kernel of each channel. In other words, CAK selects effective kernels for each channel from the candidate convolution kernels and assigns weights, and then fuses them.

[0059] In order to further verify the effectiveness and superiority of the present invention, a public 5-finger motor imagery dataset was used. [1] The experimental data is the experimental data. The experiment adopts the subject-dependent mode, and each session conducts a 10-fold cross-validation experiment. All samples of each session are randomly divided into 10 approximately equal parts, and 90% of the data is used to train the model each time, and the remaining 10% of the data is used for testing. Finally, the average decoding accuracy of 10 rounds is calculated.

[0060] The parameters of model training in the experiment are Batch size set to 100, Learning rate set to 0.005, training Epoch set to 500, and Adam optimizer is used. The software configuration of the experiment is Python 3.10, Pytorch2.0.0, and the hardware configuration is Intel Core i7-12700H 2.3GHz CPU, 16 GB RAM, NVIDIA RTX 3060GPU.

[0061] The decoding accuracy of the method of the present invention is comparable to the existing decoding progress results on the same data [1,2,3] , and the well-known deep learning model EEGNet [4] , AMCNN-DGCN [5] , FBNCNN [6] ,MBEEGNet [7] and MBK-CNN [8] The comparison of decoding accuracy is shown in the table. For details of each method, please refer to the corresponding literature. [7] and MBK-CNN [8] From motor imagery classification, AMCNN-DGCN [5] and FBN-CNN [6] From fatigue detection.

[0062] Table 1 Comparison of decoding accuracy with other methods (%) The number of trials [1] [2] AMCNN-DGCN FBNCNN [3] EEGNet MBEEGNet MBK-CNN This method A1 # 59.3 49.43 51.31 64.57 71.11 70.07 77.06 76.22 A2 # 20.8 26.64 28.60 20.27 18.57 26.76 27.46 28.25 B1 # 43.1 33.92 46.02 45.84 46.04 47.82 48.75 50.53 B2 # 42.5 42.57 45.67 44.26 52.72 55.94 51.02 57.10 B3 # 44.1 37.95 45.89 45.50 48.49 50.67 47.96 53.28 B4 # 53.9 45.05 46.08 55.66 60.79 59.54 57.35 62.35 C1 # 61.4 43.11 50.41 59.22 69.63 70.14 71.71 69.42 C2 # 61.7 46.32 49.52 63.11 72.26 71.52 72.15 76.20 E1 # 52.3 46.54 50.52 56.90 63.81 66.73 65.28 70.18 E2 # 66.5 51.26 50.10 68.28 76.97 76.87 77.81 80.23 E3 # 53.0 42.61 51.38 54.86 67.03 66.08 66.80 68.81 F1 # 38.6 39.30 49.54 55.50 61.45 59.56 64.27 66.55 F2 # 34.1 32.37 48.62 45.49 46.77 50.21 51.14 52.19 F3 # 45.4 38.58 50.26 55.56 62.46 62.67 59.03 64.13 G1 # 41.6 39.01 47.66 44.79 56.24 55.08 53.48 57.30 G2 # 42.4 39.99 49.94 44.29 51.30 54.54 50.88 57.07 H1 # 34.0 33.19 51.29 34.90 39.80 42.41 42.50 45.45 I1 # 51.4 46.41 50.06 55.05 59.44 63.41 66.95 65.17 I2 # 41.7 41.97 48.32 48.21 52.29 55.84 60.13 59.92 average value 43 46.7 40.85 47.96 50.64 56.69 58.20 58.51 61.07 Standard Deviation 10.0 11.0 6.10 4.93 10.88 13.28 11.46 12.40 11.95 The comparison results show that the decoding accuracy of the method of the present invention is 61.07%, which is 50.64% better than the best result on the same data. [3]Compared with FBN-CNN with PLI functional network (rather than raw EEG) as input, the proposed method improves by 13.11%. The average decoding accuracy of the proposed method is 4.38%, 2.87% and 2.56% higher than that of EEGNet, MBEEGNet and MBK-CNN respectively. The two-sample paired T test shows that the difference is significant (P<0.05).

[0063] [1] M. Kaya, MK Binli, E. Ozbay et al. A largeelectroencephalographic motor imagery dataset for electroencephalographicbrain computer interfaces, Scientific data, vol. 5, pp. 1-16, 2018. [2] RN Azizah, H. Zakaria, and BR Hermanto. Channels selection for pattern recognition of five fingers motor imagery electroencephalographysignals, Journal of Physics: Conference Series, vol. 2312, no. 1, p. 012019,2022. [3] K. Yang, R. Li, J. Xu, L. Zhu, W. Kong, and J. Zhang. DSFE: Decoding eeg-based finger motor imagery using feature-dependent frequency, feature fusion and ensemble learning, IEEE Journal of Biomedical and HealthInformatics, vol. 28(8), pp. 4625-4635, 2024. [4] V. J. Lawhern, A. J. Solon, N. R. Waytowich et al. EEGNet: acompact convolutional neural network for eeg-based brain–computer interfaces,Journal of neural engineering, vol. 15, no. 5, p. 056013, 2018. [5] H. Wang, L. Xu, and et al. Linking attention-based multiscale CNNwith dynamical GCN for driving fatigue detection, IEEE Transactions onInstrumentation and Measurement, vol. 70, pp. 1-11, 2021. [6] J. Chen, S. Wang, E. He, and et al. Two-dimensional phase lagindex image representation of electroencephalography for automatedrecognition of driver fatigue using convolutional neural network, ExpertSystems with Applications, vol. 191, p. 116339, 2022. [7] G. A. Altuwaijri and G. Muhammad. A multibranch of convolutionalneural network models for electroencephalogram-based motor imageryclassification, Biosensors, vol. 12, no. 1, p. 22, 2022. [8] J. Shin and W. Chung. Multi-band CNN with band-dependent kernels and amalgamated cross entropy loss for motor imagery classification, IEEEJournal of Biomedical and Health Informatics, vol. 27, no. 9, pp. 4466-4477, 2023. The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A finger movement imagery decoding method based on channel adaptive convolution kernel and information fusion network, characterized in that The specific steps include: Step S1: Obtain EEG data of finger movement imagination and preprocess the EEG data to obtain a multi-channel EEG signal , where C is the number of EEG electrode channels and T is the number of sampling points; then for multi-channel EEG signals Calculate the covariance matrix ; Step S2: construct EEG channel adaptive kernel branches to As input, extract depth information; Step S3: construct a covariance channel adaptive kernel branch, using the covariance matrix of the multi-channel EEG signal As input, extract potentially effective inter-channel structural relationship information; Step S4, concatenate the feature vectors output by the EEG channel adaptive kernel branch and the covariance channel adaptive kernel branch, and then pass them through two series-connected fully connected layers and a softmax function in sequence to obtain the finger movement imagery classification result.

2. The method according to claim 1, characterized in that: The EEG channel adaptive kernel branch includes a first CAK module, a first spatial convolution module, a first average pooling module, a depth convolution module, a point convolution module, a second average pooling module, and a first Flatten layer which are connected in series in sequence.

3. The method according to claim 2, characterized in that: The first CAK module is a multi-channel EEG signal Identify the effective convolution kernel for each channel and assign weights; then multiply the feature maps corresponding to different convolution kernels of each channel by their respective weights and add them together to obtain a mixed feature map of multiple channels; The first spatial convolution module compresses the mixed feature maps of multiple channels output by the first CAK module into one channel to summarize the information of the multiple channels; The first average pooling module is used to compress the feature map output by the first spatial convolution module to reduce the amount of calculation; The deep convolution module extracts the deep information of the feature map output by the first average pooling module, while reducing the amount of calculation; The point convolution module performs point convolution on the feature map output by the depth convolution module; The second average pooling module is used to re-compress the feature map output by the point convolution module; The first Flatten layer expands the feature map output by the second average pooling module to form a feature vector.

4. The method according to claim 2, characterized in that: The covariance channel adaptive kernel branch includes a second CAK module, a second spatial convolution module, a third average pooling module, and a second Flatten layer which are connected in series in sequence.

5. The method according to claim 4, characterized in that: The second CAK module is input Identify the effective convolution kernel for each channel and assign weights; then multiply the feature maps corresponding to different convolution kernels of each channel by their respective weights and add them together to obtain a mixed feature map of multiple channels; The second spatial convolution module compresses the mixed feature maps of multiple channels output by the second CAK module into one channel to summarize the information of the multiple channels; The third average pooling module is used to compress the feature map output by the second spatial convolution module to reduce the amount of calculation; The second Flatten layer expands the feature map output by the third average pooling module to form a feature vector.

6. The method according to claim 4, characterized in that: The first CAK module and the second CAK module have the same structure, both including Split operation, Fuse operation and Select operation, specifically: (1) Split operation: Assume that the input data is , C is the number of channels; when input M=X, N=T, when input When N=C, the input data M is subjected to the first time domain convolution, the second time domain convolution, and the third time domain convolution in parallel to obtain three sets of feature maps. , i=1,2,3; The convolution kernel sizes of the first time domain convolution, the second time domain convolution, and the third time domain convolution are different; (2) Fuse operation: Add the three sets of feature maps obtained by the Split operation element by element to obtain the feature map set U; Then each feature map in U , F represents the total number of feature maps in each group, and average pooling is performed by channel to obtain Aggregate information for each channel in ; Each feature map produces a feature vector ; Eigenvector Composition feature map ; (3) Select operation: Extract the weights of different convolution kernels on each channel from the feature map S.

7. The method according to claim 6, characterized in that: The Select operation includes three parallel branches with the same structure but different initialization parameters; each branch first uses a 1×1 convolution kernel to perform point convolution on the feature map S, compresses the information on the F feature maps in the feature map S into one feature map, and then passes through two fully connected layers to obtain the weight information ; Adaptively update weight information during network training , and use the softmax function for normalization; finally, we get the weight vector ; in represents the natural exponential function; The weight vector And the corresponding feature map Multiply and sum each element to get a fused feature map set ; 。 8. A finger movement imagery decoding system implementing the method according to any one of claims 1 to 7, characterized in that include: Data acquisition module, which acquires EEG data of finger movement imagination and preprocesses the EEG data to obtain multi-channel EEG signals ; Then multi-channel EEG signal Calculate the covariance matrix ; EEG channel adaptive kernel branch module, multi-channel EEG signal Extract information; Covariance channel adaptive kernel branch module, multi-channel EEG signal covariance matrix Extract potentially effective inter-channel structural relationship information; The classification module concatenates the output results of the EEG channel adaptive kernel branch module and the covariance channel adaptive kernel branch module, and then passes them through two series-connected fully connected layers and a softmax function to obtain the finger movement imagery classification results.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.

10. A computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 7 is implemented.

Images