Motor imagery decoding method based on CSP multi-scale convolutional neural network

Abstract

The application discloses a motor imagery decoding method based on a CSP multi-scale convolutional neural network, comprising the following steps: normalizing EEG brain electrical signal data, then performing band-pass filtering processing, cutting a plurality of filtering frequency bands, and performing CSP common space spatial filtering processing; assigning weights to the data of each frequency band; automatically extracting the signals with the assigned weights, filtering with different fineness, pooling and convolution, splicing the features extracted by multiple different scale convolutions in the same dimension to form a feature map; reassigning feature weights to the feature maps of different frequency bands, extracting deeper time sequence features by using a gated recurrent network, and then performing feature splicing; and performing full connection network classification processing, so that the number of finally output neurons is equal to the number of classification categories. The motor imagery decoding method can greatly improve the filtering effect, and effectively improve the accuracy of motor imagery decoding classification and the robustness of the classification model.

Description

Technical Field

[0001] This invention relates to the field of motion visualization technology, and in particular to a motion visualization decoding method based on CSP multi-scale convolutional neural networks. Background Technology

[0002] With the development of neuroscience, brain-computer interface (BCI) technology is also continuously advancing in fields such as medicine. In motor imagery within BCI technology, a typical approach involves designing specific motor imagery paradigms to guide subjects in imagining limb movements through corresponding visual images. Then, an EEG cap is used to acquire the corresponding brainwaves, and finally, algorithms are used to decode the EEG signals to recognize the subject's intentions. By decoding these signals, external devices such as robotic arms and wheelchairs can be controlled, effectively helping patients with limb movement disorders to perform some daily tasks based on their own will.

[0003] Traditional EEG signal decoding is based on machine learning, involving manual feature extraction from the signal, followed by the input of these features into a classifier for classification and decoding. Common feature extraction methods include Common Spatial Patterns (CSP) and Filter Bank Common Spatial Patterns (FBCSP). Common classifiers include Linear Discriminant Analysis (LDA), Support Vector Machines (SVM), and Random Forests (RF).

[0004] Traditional machine learning feature extraction requires a large amount of prior knowledge and consumes a lot of time when analyzing data. At the same time, EEG signals are affected by external environmental interference and the state of the subjects, resulting in large differences between data. This leads to weak generalization performance of the trained model and low classification accuracy on the test set, thus posing a significant challenge in practical applications. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the present invention provides a motion imagery decoding method based on CSP multi-scale convolutional neural network. By segmenting the normalized data and performing CSP spatial filtering on the segmented data, the useful information of EEG signals can be preserved in a partitioned manner to the greatest extent, which greatly improves the filtering effect and thus effectively improves the accuracy of motion imagery decoding classification.

[0006] Furthermore, this invention also extracts features from signals of different frequency bands using multi-scale convolution and then performs feature fusion. This prevents information loss due to inappropriate selection of convolution kernel scale, thus preserving more information in the signal, which is beneficial to improving classification accuracy, enhancing model generalization performance, and improving the robustness of the classification model.

[0007] To achieve at least one of the advantages of the present invention, in a first aspect, the present invention provides a motion imagination decoding method based on a CSP multi-scale convolutional neural network, comprising the following steps:

[0008] Standardize EEG brainwave signal data;

[0009] The standardized EEG brainwave signal data was subjected to bandpass filtering to divide it into several filtering frequency bands;

[0010] CSP (Common Spatial Filtering) is applied to the single-band data after bandpass filtering.

[0011] A self-attention mechanism module is used to assign weights to the data in each frequency band;

[0012] The signal with assigned weights is automatically extracted using a multi-scale hybrid neural network, filtered at different levels of fineness, then pooled and convolved, and the features extracted by multiple convolutions at different scales are concatenated with the same dimension to form a feature map.

[0013] The attention mechanism module is used to redistribute feature weights to feature maps of different frequency bands, and a gated recurrent network is used to extract deeper temporal features from the feature maps after the feature weights have been redistributed. Finally, the temporal features of multiple frequency bands are concatenated to form three-dimensional features.

[0014] Fully connected network classification processing ensures that the number of neurons in the final output is equal to the number of classification categories.

[0015] According to an embodiment of the present invention, the bandpass filtering of the normalized EEG signal data to divide it into several filter frequency bands includes:

[0016] The frequency bands are divided into four: (1-4Hz), (4-8Hz), (8-13Hz), and (13-30Hz).

[0017] According to an embodiment of the present invention, the CSP co-spatial filtering process includes:

[0018] The training data is segmented according to the label category. First, the covariance of the i-th class signal is calculated. Where N i This represents the number of samples in the i-th class;

[0019] Then, the spatial projection matrix W is calculated based on the two types of covariance matrices. The calculation method is as follows: ∑1W=λ∑2W, where w is a column of the spatial projection matrix W;

[0020] Finally, the EEG signal is transformed using the spatial filtering matrix W to obtain Z = W. T X, where signal Z is the signal after CSP co-space spatial filtering.

[0021] According to an embodiment of the present invention, the method of assigning weights to the data of each frequency band using the self-attention mechanism module specifically involves:

[0022] The spatially filtered signal is input into the self-attention mechanism module, then global pooling is used to compress the data to the number of frequency bands, followed by two layers of feedforward network for further compression, and then two layers of fully connected network to expand the data back to the number of frequency bands, before finally passing it through the sigmoid function. Further mapping yields the frequency band weighting coefficients, which are then multiplied by the original input signal Z to obtain the data. Then, a residual connection is used to add the original input signal to the redistributed data to obtain... That is, the data after the frequency band weights have been redistributed.

[0023] According to an embodiment of the present invention, the automatic extraction of the spatially filtered signal using a multi-scale hybrid neural network includes:

[0024] Data after reallocation of frequency band weights Where C is the number of channels, N s To determine the number of sampling points, multiple different temporal convolutional kernels are used for temporal feature extraction;

[0025] Then, activate using the ReLU activation function, with the activation formula as follows:

[0026] According to an embodiment of the present invention, the pooling and convolution include:

[0027] After activating the data extracted from temporal features through temporal convolution, a max pooling layer is used for the first pooling. Then, a second convolution is performed on the data after the first pooling, and a second pooling is performed on the data after the second convolution. The pooling kernel in the first and second pooling is (1, p), the stride is (1, p), and the padding is valid.

[0028] According to one embodiment of the present invention, the different temporal convolution kernels are implemented as four, and the parameters of the four convolution kernels are respectively: (filters,1,t1), (filters,1,t2), (filters,1,t3), and (filters,1,t4). These four convolution kernels perform feature extraction in parallel.

[0029] According to one embodiment of the present invention, the number of filters in a single convolution is 14, the stride is (1,1), and the padding is the same.

[0030] In the second convolution, the filters are set to 28, while other parameters remain unchanged.

[0031] According to one embodiment of the present invention, the gated loop network has two control gates that control the memory unit and the output unit respectively, which can save the information of the previously acquired data at one time, and can also forget the information of the previous time at any time.

[0032] According to an embodiment of the present invention, the fully connected network classification process includes:

[0033] First, the three-dimensional features are flattened and converted into one-dimensional feature vectors;

[0034] The one-dimensional feature vector is then input into a fully connected network with two hidden layers, where the number of neurons in the two layers decreases sequentially, and the activation function is the ReLU function.

[0035] A random dropout operation is performed after the activation function to reduce the number of neurons in a single iteration;

[0036] Finally, the number of neurons in the output is set to the number of classification categories;

[0037] The entire fully connected network model uses the cross-entropy loss function, the formula of which is:

[0038]

[0039] The optimizer uses stochastic gradient descent for backpropagation to update the network model parameters, with the learning rate set to 0.0001.

[0040] These and other objects, features and advantages of the present invention will become fully apparent from the following detailed description. Attached Figure Description

[0041] Figure 1 A flowchart illustrating a preferred embodiment of the motion imagery decoding method of this application is shown.

[0042] Figure 2 A detailed flowchart of a preferred embodiment of the motion imagery decoding method of this application is shown.

[0043] Figure 3 A schematic diagram of the attention weight allocation module in this application is shown.

[0044] Figure 4 A schematic diagram of the multi-scale convolution structure in this application is shown.

[0045] Figure 5 The structural diagram of the gated loop network module in this application is shown.

[0046] Figure 6 The chart shows a comparison of the accuracy rates of nine subjects using different methods.

[0047] Figure 7The accuracy data of nine subjects under different methods are shown in the graph. Detailed Implementation

[0048] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0049] Those skilled in the art should understand that, in the disclosure of this specification, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting the present invention.

[0050] It is understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.

[0051] refer to Figures 1 to 7 A preferred embodiment of the present invention, a motion imagery decoding method based on a CSP multi-scale convolutional neural network, will be described in detail below, wherein the motion imagery decoding method based on a CSP multi-scale convolutional neural network includes the following steps:

[0052] EEG brainwave signal data was acquired using an EEG cap.

[0053] To standardize EEG signal data, the EEG signal data is normalized using the following formula:

[0054]

[0055] Where x represents the input signal, μ is the mean, and δ is the standard deviation. Normalization makes the mean of the channel data 0 and keeps the data of each channel at the same scale. In this way, the adverse effects of data offset caused by environmental factors can be reduced.

[0056] The standardized EEG signal data is subjected to bandpass filtering, such as using a Chebyshev filter, to divide it into several filtering frequency bands to facilitate feature extraction.

[0057] CSP co-spatial filtering is applied to the single-band data after bandpass filtering to enable the data of each lead to contain lead-related information, which in turn helps to increase the richness of features automatically extracted by the subsequent multi-scale convolutional network and improve the classification accuracy of the motion imagery decoding method.

[0058] A self-attention mechanism module is used to assign weights to the data in each frequency band;

[0059] Multi-scale hybrid neural networks are used to automatically extract spatially filtered signals and perform filtering at different levels of fineness so that each data point after convolution contains information from different ranges around it, resulting in a larger amount of data information. Then, pooling and convolution are performed, and the features extracted by multiple convolutions at different scales are stitched together with the same dimension. The stitched feature map is the feature automatically extracted by the multi-scale convolutional network.

[0060] The attention mechanism module is used to redistribute feature weights to feature maps of different frequency bands, and a gated recurrent network is used to extract deeper temporal features from the feature maps after the feature weights have been redistributed. Finally, the temporal features of multiple frequency bands are concatenated to form three-dimensional features.

[0061] Fully connected network classification processing ensures that the number of neurons in the final output is equal to the number of classification categories.

[0062] Different frequency bands exhibit different characteristics in motion graphics. However, using data from the entire frequency band for CSP spatial filtering can lead to relative mixing between different frequency bands, resulting in poor filtering performance. Therefore, in one embodiment, a combination of... Figure 2 The process of bandpass filtering the standardized EEG signal data to divide it into several filter frequency bands includes:

[0063] Four filtering frequency bands were segmented: δ (1-4Hz), θ (4-8Hz), α (8-13Hz), and β (13-30Hz). Motor imagery exhibits strong energy variations in these four frequency bands. Subsequently, CSP spatial filtering was performed on each of these four frequency bands to retain useful information in the signal with maximum differentiation, thereby improving the quality of the EEG signal before it enters the network.

[0064] More preferably, the CSP co-spatial filtering process includes:

[0065] The training data is segmented according to the label category. First, the covariance of the i-th class signal is calculated. Where N i This represents the number of samples in the i-th class;

[0066] Then, the spatial projection matrix W is calculated based on the two types of covariance matrices. The calculation method is as follows: ∑1W=λ∑2W, where w is a column of the spatial projection matrix W;

[0067] Finally, the EEG signal is transformed using the spatial filtering matrix W to obtain Z = W. T X, where signal Z is the signal after CSP co-space spatial filtering. In this way, after spatial filtering, the data of each lead contains lead-related information, which is beneficial to increase the richness of features automatically extracted by the subsequent multi-scale convolutional network.

[0068] More preferably, the automatic extraction of the spatially filtered signal using a multi-scale hybrid neural network includes:

[0069] Combination Figure 3 The spatially filtered signal is input into the self-attention mechanism module, then global pooling is used to compress the data to the number of frequency bands, followed by two layers of feedforward network for further compression, and then two layers of fully connected network to expand the data back to the number of frequency bands, before finally passing it through the sigmoid function. Further mapping yields the frequency band weighting coefficients, which are then multiplied by the original input signal Z to obtain the data. This is the data after frequency band redistribution. Then, a residual connection is used to add the original input signal and the redistributed data to obtain... That is, the data after the frequency band weights have been redistributed;

[0070] The filtered signal (where C is the number of channels, N) s Temporal feature extraction is performed using multiple different temporal convolution kernels (to account for the number of sampling points).

[0071] Then, activate using the ReLU activation function, with the activation formula as follows: Furthermore, by using convolutions of different scales, the same time signal sequence can be filtered with different levels of fineness, and each convolved data will contain information from different surrounding ranges. In this way, the amount of data information will be greater, making the motion imagery decoding method provided in this application more accurate.

[0072] More preferably, the pooling and convolution steps include:

[0073] After activating the data extracted from temporal features through temporal convolution, a max pooling layer is used for the first pooling. Then, a second convolution is performed on the data after the first pooling, and a second pooling is performed on the data after the second convolution. The pooling kernel in the first and second pooling is (1, p), the stride is (1, p), and the padding is valid. This allows us to find the maximum value within the pooling kernel matrix as the output.

[0074] More preferably, combined with Figure 4 The different temporal convolutional kernels are implemented as four, with parameters as follows: (filters, 1, t1), (filters, 1, t2), (filters, 1, t3), and (filters, 1, t4). In the first convolution, filters are set to 14, stride is (1, 1), and padding is set to "same," meaning it remains the same as before convolution. This is equivalent to using four convolutional kernels of different scales to extract features in parallel for each convolutional layer. The four kernels are arranged sequentially from smallest to largest, preventing information loss due to inappropriate kernel scale selection and preserving more information in the signal, thus improving classification accuracy. In the second convolution, filters are set to 28, with other parameters remaining unchanged. This doubles the dimensionality of the original convolution, resulting in better fitting ability. In other words, single-pass pooling and convolution can be performed, or multiple pooling and convolution can be performed. However, double pooling and convolution generally yields better fitting ability than single pooling and convolution.

[0075] More preferably, combined with Figure 5 The gated recurrent network has two control gates, which control the memory unit and the output unit respectively. It can save the information of the previously acquired data at one time, and can also forget the information of the previous time at any time. This can reduce the risk of gradient vanishing and gradient explosion in the data transmission process of the neural network, which is conducive to the convergence of the model. At the same time, this ability to remember previous sequences has strong performance in temporal feature extraction, and it is easier to extract the deep features of temporal data.

[0076] More preferably, the fully connected network classification process includes:

[0077] First, the three-dimensional features are flattened and converted into one-dimensional feature vectors;

[0078] Then, the one-dimensional feature vector is input into a fully connected network with two hidden layers. To ensure the stability of the model, the number of neurons in the two layers decreases sequentially, and the activation function is the ReLU function.

[0079] In addition, to reduce the risk of model overfitting, a random dropout operation, such as Dropout, is performed after the activation function to randomly drop neurons, thereby reducing the number of neurons in a single iteration, thus reducing the number of network parameters, and further reducing the risk of overfitting to improve the generalization performance of the model. The dropout rate can be set to 0.5.

[0080] Finally, the number of neurons in the output is set to the number of classification categories;

[0081] The entire fully connected network model uses the cross-entropy loss function, the formula of which is:

[0082]

[0083] The optimizer uses stochastic gradient descent for backpropagation to update the network model parameters, with the learning rate set to 0.0001.

[0084] Overall, the motion imagery decoding method based on CSP multi-scale convolutional neural networks in this application not only significantly improves the accuracy, but also improves the convenience of classification model development by using neural networks for automatic feature extraction of signals. In addition, the use of CSP multi-scale convolution also improves the robustness of the classification model.

[0085] Combination Figure 6 and Figure 7 The results demonstrate the performance of three different methods on the BCI Competition 4b left and right hand motor imagery dataset, in tests (a), (b), and (c):

[0086] (a) Feature extraction is performed using the Common Spatial Pattern (CSP), and classification is performed using the Random Forest (RF) classifier method;

[0087] (b) Feature extraction is performed using Common Spatial Patterns (CSP), and classification is performed using a Support Vector Machine (SVM) classifier.

[0088] (c) The method proposed in this application uses a multi-scale convolutional neural network (MSCNN) to automatically extract features and a fully connected network for classification.

[0089] Depend on Figure 5 and Figure 6 As can be seen, the MSCNN method proposed in this application achieves significantly higher accuracy than the other two traditional machine learning methods on the dataset, with an average accuracy improvement of approximately 10%.

[0090] This application also provides a computer program product, wherein the computer program product includes computer program code, and when the computer program code is run on a computer, the computer is able to perform the methods described in the various possible implementations above.

[0091] This application also provides a chip including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, enabling a device with the chip installed to perform the methods described in the various possible implementations above.

[0092] It is understood that the above scenarios are merely examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, as those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0093] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0094] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0095] The units in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0096] In this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions are generally described in detail only when they appear for the first time. When they appear again, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions that are not described in detail later can be referred to their previous relevant detailed descriptions.

[0097] In this application, the descriptions of the various embodiments have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0098] The technical features of the present application can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, iPad, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0100] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, storage disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0101] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the invention. The advantages of the present invention have been fully and effectively realized. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments; any variations or modifications can be made to the implementation of the present invention without departing from these principles.

Claims

1. A motion imagination decoding method based on CSP multi-scale convolutional neural networks, characterized in that, The steps are as follows: Standardize EEG brainwave signal data; The standardized EEG brainwave signal data was subjected to bandpass filtering to divide it into several filtering frequency bands; CSP (Common Spatial Filtering) is applied to the single-band data after bandpass filtering. A self-attention mechanism module is used to assign weights to the data in each frequency band; The signal with assigned weights is automatically extracted using a multi-scale hybrid neural network, filtered at different levels of fineness, then pooled and convolved, and the features extracted by multiple convolutions at different scales are concatenated with the same dimension to form a feature map. An attention mechanism module is used to reassign feature weights to feature maps of different frequency bands. A gated recurrent network is then used to extract deeper temporal features from the weighted feature maps. Finally, the temporal features from multiple frequency bands are concatenated to form a three-dimensional feature set. Specifically, the self-attention mechanism module assigns weights to the data in each frequency band as follows: the spatially filtered signal is input into the self-attention mechanism module, then global pooling compresses the data to the number of frequency bands. Two layers of feedforward network further compress the data, followed by two layers of fully connected network to expand the data back to the number of frequency bands, and finally, a sigmoid function is applied. Further mapping yields the frequency band weighting coefficients, which are then multiplied by the original input signal Z to obtain the data. Then, a residual connection is used to add the original input signal to the redistributed data to obtain... That is, the data after the frequency band weights have been redistributed; Fully connected network classification processing ensures that the number of neurons in the final output is equal to the number of classification categories; Automatic extraction of spatially filtered signals using multi-scale hybrid neural networks includes: Data after reallocation of frequency band weights ,in For the number of channels, To determine the number of sampling points, multiple different temporal convolutional kernels are used for temporal feature extraction; Then, activate using the ReLU activation function, with the activation formula as follows: ; The different temporal convolution kernels are implemented as four, and the parameters of the four convolution kernels are as follows: , , , These four convolutional kernels perform feature extraction in parallel.

2. The motion imagery decoding method based on CSP multi-scale convolutional neural networks as described in claim 1, characterized in that, The standardized EEG signal data is subjected to bandpass filtering to divide it into several filtering frequency bands, including: The frequency bands are divided into four: (1-4Hz), (4-8Hz), (8-13Hz), and (13-30Hz).

3. The motion imagery decoding method based on CSP multi-scale convolutional neural networks as described in claim 2, characterized in that, The CSP co-space filtering process includes: The training data is split according to the label category. First, the first... Covariance of signal class ,in Indicates the first The number of class samples; Then, the spatial projection matrix is ​​calculated based on the two types of covariance matrices. The calculation method is as follows , ,in Spatial projection matrix A certain column; Finally, the EEG signal is filtered through a spatial filtering matrix. Transformation , among which the signal This is the signal after CSP co-space spatial filtering.

4. The motion imagery decoding method based on CSP multi-scale hybrid neural networks as described in claim 3, characterized in that, The pooling and convolution processes include: After activating the data extracted from temporal features through temporal convolution, a max pooling layer is used for the first pooling. Then, a second convolution is performed on the data after the first pooling, and a second pooling is performed on the data after the second convolution. The pooling kernel in the first and second pooling is (1, p), the stride is (1, p), and the padding is valid.

5. The motion imagery decoding method based on CSP multi-scale hybrid neural networks as described in claim 4, characterized in that, In one convolution, there are 14 filters, a stride of (1,1), and the same padding. In the second convolution, the filters are set to 28, while other parameters remain unchanged.

6. The motion imagery decoding method based on CSP multi-scale hybrid neural network as described in claim 4, characterized in that, The gated loop network has two control gates, which control the memory unit and the output unit respectively. It can save the information of the last acquired data at one time, and can also forget the information of the last time at any time.

7. The motion imagery decoding method based on CSP multi-scale convolutional neural networks as described in claim 6, characterized in that, The fully connected network classification process includes: First, the three-dimensional features are flattened and converted into one-dimensional feature vectors; The one-dimensional feature vector is then input into a fully connected network with two hidden layers, where the number of neurons in the two layers decreases sequentially, and the activation function is the ReLU function. A random dropout operation is performed after the activation function to reduce the number of neurons in a single iteration; Finally, the number of neurons in the output is set to the number of classification categories; The entire fully connected network model uses the cross-entropy loss function, the formula of which is: ; The optimizer uses stochastic gradient descent for backpropagation to update the network model parameters, with the learning rate set to 0.0001.

Images