Motor imagery electroencephalogram signal decoding method and system based on Sinc filter

Abstract

The invention discloses a motor imagery electroencephalogram signal decoding method and system based on a Sinc filter, and belongs to the field of brain-computer interfaces, and the method comprises the following steps: carrying out convolution calculation on a motor imagery electroencephalogram signal and the Sinc filter to obtain a frequency characteristic signal of the motor imagery electroencephalogram signal; performing channel connection on the characteristic signals of the motor imagery electroencephalogram signals of different frequency bands to generate high-dimensional electroencephalogram characteristic signals; obtaining a parallel multi-branch space-time convolutional network, performing time domain feature extraction, spatial feature extraction and average pooling dimension reduction processing on the high-dimensional electroencephalogram feature signal to obtain a channel weight, and performing weighted calculation on the target feature map based on the channel weight to obtain a space-time feature map; and obtaining class labels of the motor imagery electroencephalogram signals after mapping processing and activation processing, and obtaining a decoding result. According to the method, the Sinc filter and the space-time convolutional neural network are combined, and the accuracy and robustness of motor imagery electroencephalogram signal classification are improved.

Description

Technical Field

[0001] The present application belongs to the field of brain-computer interface, and specifically relates to a method and system for decoding motor imagery EEG signals based on Sinc filters. Background Art

[0002] The brain-computer interface is a communication channel connecting the nervous system with the external environment. It allows users to directly control external devices through brain activity, replacing the traditional peripheral nervous system control mechanism. Motor imagery EEG signals are transmitted in the brain-computer interface. Over the years, EEG-based brain-computer interfaces have developed several paradigms, including steady-state visual evoked potentials, event-related potentials, emotions, and motor imagery. The brain-computer interface based on motor imagery has received great attention because it can decode the user's motor intention from motor imagery EEG signals.

[0003] However, the prior art has the following deficiencies: First, when extracting features from motor imagery EEG signals, since motor imagery EEG signals in different frequency domains have different physical meanings and waveform shapes, it is difficult for traditional filters to effectively extract features in the high-frequency range of motor imagery EEG signals during convolution operations, and motor imagery EEG signal distortion is prone to occur when extracting in the high-frequency range; Second, before extracting features from motor imagery EEG signals, it is usually necessary to collect a large number of motor imagery EEG signal samples from different subjects, and the degree of smoothness of motor imagery EEG signals from different subjects is different, so it is difficult to adaptively adjust the parameters of feature extraction according to the specificity of the subjects, and the classification results of motor imagery EEG signals obtained are inaccurate. Therefore, the prior art cannot effectively extract motor imagery EEG signals in each frequency domain range, and cannot be applied to feature extraction of different motor imagery EEG signals, resulting in low accuracy and robustness of the obtained decoding results. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method and system for decoding motor imagery EEG signals based on a Sinc filter, which can solve the problem of low accuracy and robustness of motor imagery EEG signal classification results.

[0005] In order to solve the above technical problems, this application is implemented as follows: In a first aspect, an embodiment of the present application provides a method for decoding motor imagery EEG signals based on a Sinc filter, the method comprising: Convolution calculation is performed on the motor imagery EEG signal and the Sinc filter to obtain the characteristic signal of the motor imagery EEG signal; Based on point convolution, the characteristic signals of motor imagery EEG signals of different frequency bands are connected through channels to generate high-dimensional EEG characteristic signals; Based on the temporal convolution layer, spatial convolution layer and average pooling layer, a parallel multi-branch spatiotemporal convolutional network is built; Based on the multi-branch spatiotemporal convolutional network, respectively, the high-dimensional EEG feature signal is subjected to time domain feature extraction, spatial feature extraction, and average pooling dimensionality reduction processing in sequence to obtain an output feature map corresponding to each branch; Based on the preset spatial feature dimension and temporal feature dimension, the plurality of output feature maps are subjected to average pooling dimensionality reduction processing in the time dimension, and are concatenated in the channel dimension to obtain a target feature map; Obtaining a target convolution kernel based on the relationship between the number of channels of the target feature map and the convolution kernel, and obtaining a channel weight based on the target convolution kernel; Performing weighted calculation on the target feature map based on the channel weight to obtain a spatiotemporal feature map; The spatiotemporal feature map is sequentially subjected to mapping processing and activation processing to obtain a class label of the motor imagery EEG signal and obtain a decoding result.

[0006] As an optional implementation of the first aspect of the present application, the process of convolving a motor imagery EEG signal with a Sinc filter to obtain a characteristic signal of the motor imagery EEG signal includes: obtaining a frequency domain representation of the filter group based on a cutoff frequency, and inversely transforming the frequency domain representation based on an inverse Fourier transform to obtain a time domain representation of the filter group; windowing the time domain representation to obtain a Sinc filter, and convolving the motor imagery EEG signal with the Sinc filter to obtain the characteristic signal of the motor imagery EEG signal.

[0007] As an optional implementation of the first aspect of the present application, the mathematical expression of the Sinc filter is: in, Represents the time domain expression of the Sinc filter, represents the time domain expression of the Sinc filter before windowing, represents the length of the motor imagery EEG signal, represents the length of the window, is the low cutoff frequency, is the high cutoff frequency.

[0008] As an optional implementation of the first aspect of the present application, a parallel multi-branch spatiotemporal convolutional network is connected in parallel through multiple branches, and each branch consists of, from left to right: a temporal convolution layer, a spatial convolution layer, and an average pooling layer. The number of convolution kernels in the temporal convolution layer of each branch is the same as the number of convolution kernels in the spatial convolution layer. The sizes of the convolution kernels of the temporal convolution layers in different branches are different, and the sizes of the convolution kernels of the spatial convolution layers are the same.

[0009] As an optional implementation manner of the first aspect of the present application, the process of obtaining the target feature map includes: The convolution kernel size in the time convolution layer of multiple branches is set based on the length of the filter on the time axis to obtain the set time convolution kernel, and the time convolution layer of multiple branches performs time domain feature extraction according to the set time convolution kernel to obtain multiple time domain feature maps; the convolution kernel size in the space convolution layer of multiple branches is set based on the number of electrode channels of the motor imagery EEG signal to obtain the set space convolution kernel, and the space convolution layer of multiple branches performs space feature extraction according to the set space convolution kernel to obtain multiple space feature maps, and the space feature maps and the time domain feature maps are fused to generate multiple output feature maps; based on the preset spatial feature dimension and time feature dimension, the multiple output feature maps are averaged pooled in the time dimension for dimensionality reduction and spliced ​​in the channel dimension to obtain the target feature map.

[0010] As an optional implementation of the first aspect of the present application, the size of the set temporal convolution kernel and the size of the set spatial convolution kernel are respectively (1, K) and (N, 1), wherein K represents the length of the filter on the time axis, and N represents the number of electrode channels of the motor imagery EEG signal.

[0011] As an optional implementation of the first aspect of the present application, the process of obtaining a class label of a motor imagery EEG signal after sequentially mapping and activating the spatiotemporal feature map to obtain a decoding result includes: constructing an initial SMANet network model based on a Sinc convolution layer, a point convolution layer, a multi-branch spatiotemporal convolutional network, a channel attention layer, a fully connected layer, and a classification layer, establishing a total loss function based on a cross entropy loss function and a center loss function, and training the initial SMANet network model based on the total loss function to obtain a trained SMANet network model; after regularizing the weights of the fully connected layer based on a norm constraint, the spatiotemporal feature map is input into the fully connected layer of the trained SMANet network model for mapping and activation processing to generate a class label of a motor imagery EEG signal to obtain a decoding result.

[0012] In a second aspect, an embodiment of the present application provides a motor imagery EEG signal decoding system based on a Sinc filter, comprising: A Sinc convolution module, wherein the Sinc convolution module is used to obtain a characteristic signal of the motor imagery EEG signal after performing convolution calculation on the motor imagery EEG signal and the Sinc filter; A point convolution module, the point convolution module is used to combine the characteristic signals of the motor imagery EEG signals of different frequency bands according to a preset point convolution to generate a high-dimensional EEG characteristic signal; A multi-branch spatiotemporal convolution module, which is used to perform time domain feature extraction, spatial feature extraction, and average pooling dimensionality reduction processing on the high-dimensional EEG feature signal in sequence to obtain output feature maps of each branch, and to splice the output feature maps in the channel dimension to obtain a target feature map; A channel attention mechanism module, wherein the channel attention mechanism module is used to perform weighted calculation on the target feature map to obtain a spatiotemporal feature map; A classification module is used to obtain a category probability after mapping and activating the spatiotemporal feature map, and obtain a class label of the motor imagery EEG signal based on the category probability to obtain a decoding result.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method of the first aspect.

[0014] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0015] Compared with the prior art, the present invention proposes a method for decoding motor imagery EEG signals based on Sinc filters. After windowing the Sinc filters, feature extraction and convolution operations are performed on the motor imagery EEG signals to obtain high-dimensional EEG feature signals, thereby reducing high-frequency distortion. An initial SMANet network model is constructed based on Sinc convolution layers, point convolution layers, multi-branch spatiotemporal convolution networks, channel attention layers, fully connected layers, and classification layers. A parallel multi-branch spatiotemporal convolution network is used to perform time domain feature extraction, spatial feature extraction, average pooling dimensionality reduction, and splicing on the high-dimensional EEG feature signals to obtain a target feature map, thereby being able to finely capture the spatiotemporal features of the target feature map at different scales. The channel attention mechanism is used to adaptively The channel attention weight of the target feature map is adjusted to obtain the channel weight, and the target feature map is weighted according to the channel weight to obtain the spatiotemporal feature map, which can focus on the extraction of key channel information in the target feature map and reduce redundant spatiotemporal features; the cross entropy loss function and the center loss function are used as the total loss function of the network model to train the initial SMANet network model, and the cross entropy loss function is used to supervise the accuracy of the classification task, and optimize the model's ability to distinguish motor imagery categories by minimizing the probability distribution difference between the predicted category and the true label; the center loss function is used to constrain the distribution density of similar samples in the feature space, and by reducing the feature distance within the class and increasing the feature difference between the classes, the model's robustness to individual differences and noise interference is enhanced. The present invention can effectively extract motor imagery EEG signals in each frequency domain range, and can be applied to the feature extraction of different motor imagery EEG signals, and the decoding results obtained are highly accurate and robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flowchart of a method for decoding motor imagery EEG signals based on a Sinc filter provided in the first embodiment of the present application; Figure 2 It is a BCI experiment timing scheme diagram in a method for decoding motor imagery EEG signals based on a Sinc filter provided in the first embodiment of the present application; Figure 3 It is a network structure diagram of a channel attention mechanism module in a method for decoding motor imagery EEG signals based on a Sinc filter provided in the first embodiment of the present application; Figure 4 This is a structural diagram of the SMANet network model in a method for decoding motor imagery EEG signals based on a Sinc filter provided in the first embodiment of the present application.

[0017] Figure 5 This application is a structural schematic diagram of a motor imagery EEG signal decoding system based on a Sinc filter provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0020] In combination with the accompanying drawings, a Sinc filter-based motor imagery EEG signal decoding method, a Sinc filter-based motor imagery EEG signal decoding system, an electronic device and a readable storage medium provided in an embodiment of the present application are described in detail through specific embodiments and their application scenarios.

[0021] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0022] Example 1 See also Figure 1 , which is a flowchart of a method for decoding motor imagery EEG signals based on a Sinc filter proposed in the first embodiment of the present application. The proposed method includes S01 to S08.

[0023] Step S01: performing convolution calculation on the motor imagery EEG signal and the Sinc filter to obtain a characteristic signal of the motor imagery EEG signal.

[0024] In step S01 of the present invention, a Sinc filter is constructed. First, a frequency domain representation of the filter group is obtained based on the cutoff frequency, and the frequency domain representation is inversely transformed based on the inverse Fourier transform to obtain a time domain representation of the filter group. Secondly, the time domain representation is windowed to obtain a Sinc filter, and a convolution calculation is performed on the motor imagery EEG signal and the Sinc filter to obtain a characteristic signal of the motor imagery EEG signal.

[0025] Specifically, the process of obtaining the time domain representation of the filter bank is: according to the learnable low cutoff frequency and high cutoff frequency, the frequency domain representation of the filter bank is obtained, and its mathematical expression is: In formula (1), is the frequency domain representation of the filter bank, is a rectangular function, is the low cutoff frequency, is the high cutoff frequency, is the preset frequency.

[0026] The process of inversely transforming the frequency domain representation based on the inverse Fourier transform to obtain the time domain representation of the filter bank is shown in formula (2): In formula (2), represents the time domain representation of the filter bank. In order to meet the training process and The restrictions must meet the following conditions: , = ,in, is the frequency band of the bandpass filter, is the time domain representation of the filter bank.

[0027] The process of windowing the time domain representation to obtain the Sinc filter includes: the time domain representation of the filter bank based on the Hamming window After windowing, a Sinc filter is obtained. This filter is a bandpass filter, and its mathematical expression is as shown in formula (3): In formula (3), represents the time domain expression of the Sinc filter, represents the length of the motor imagery EEG signal, represents the length of the window, is the low cutoff frequency, is the high cutoff frequency.

[0028] The process of convolving the motor imagery EEG signal with the Sinc filter to obtain the characteristic signal of the motor imagery EEG signal is shown in formula (4): In formula (4), It is the characteristic signal of motor imagery EEG signal. Is the length The method schematic diagram of step S01 of the present invention is shown in Figure 1 .

[0029] The reason why the present invention uses a Sinc filter to extract features from motor imagery EEG signals is that when an unwindowed Sinc filter is used, the filter has an infinite impulse response that cannot be implemented in a real system and must be truncated. Truncation will cause the Gibbs phenomenon, i.e., spectrum leakage and oscillation, thereby causing high-frequency signal distortion. By windowing, the mutations at both ends of the Sinc function can be suppressed, thereby reducing the high-frequency distortion caused by truncation.

[0030] Step S02: performing channel connection on characteristic signals of motor imagery EEG signals of different frequency bands based on point convolution to generate high-dimensional EEG characteristic signals; In step S02 of the present invention, the number of point convolutions is 36, and the kernel sizes are all set to (1,1). The present invention uses point convolution operations to integrate feature information in different frequency domains into a higher-dimensional representation, which can retain the time series information of the original features and effectively fuse the features of multiple frequency bands, thereby improving the ability of overall feature expression, improving the accuracy of motor imagery EEG signal classification, and improving the accuracy and robustness of decoding results.

[0031] Step S03: Building a parallel multi-branch spatiotemporal convolutional network based on the temporal convolutional layer, the spatial convolutional layer and the average pooling layer; In step S03 of the present invention, the parallel multi-branch spatiotemporal convolutional network is connected in parallel through multiple branches, and the multiple branches are respectively from top to bottom: a time convolution layer, a space convolution layer and an average pooling layer. The time convolution layer is a convolution layer specifically used to capture feature changes in the time dimension, and is used to capture feature maps of high-dimensional EEG feature signals at different time scales. The spatial convolution layer is responsible for capturing and transmitting spatial correlation information between different electrode channels, and is used to transmit information between different electrode channels; the average pooling layer is a layer that downsamples the input feature map, and performs dimensionality reduction processing by calculating the average value of the features in the local area, thereby reducing the size of the feature map while retaining the important features of the feature map. In the present invention, the number of convolution kernels in the time convolution layer of each branch is the same as the number of convolution kernels in the space convolution layer, the size of the convolution kernel of the time convolution layer in each branch is different, and the size of the convolution kernel of the space convolution layer is the same.

[0032] In addition, the present invention connects multi-branch spatiotemporal convolutional networks in parallel, which can finely capture the spatiotemporal characteristics of the target feature map at different scales when the collected motor imagery EEG signal samples are large and the motor imagery EEG signal is complex.

[0033] Step S04: Based on the multi-branch spatiotemporal convolutional network, respectively perform time domain feature extraction, spatial feature extraction, and average pooling dimensionality reduction processing on the high-dimensional EEG feature signal to obtain an output feature map corresponding to each branch; In step S04 of the present invention, a plurality of output feature maps are generated. First, the convolution kernel size in the time convolution layer of the plurality of branches is set based on the length of the filter on the time axis, and the set time convolution kernel is obtained. The time convolution layer of the plurality of branches performs time domain feature extraction according to the set time convolution kernel to obtain a plurality of time domain feature maps. Secondly, the convolution kernel size in the space convolution layer of the plurality of branches is set based on the number of electrode channels of the motor imagery EEG signal, and the set space convolution kernel is obtained. The space convolution layer of the plurality of branches performs space feature extraction according to the set space convolution kernel to obtain a plurality of space feature maps, and the space feature maps and the time domain feature maps are fused to generate a plurality of output feature maps. Specifically, the present invention sets the number of branches in the multi-branch spatiotemporal convolutional network to three, and the lengths K of the filters on the time axis of the three branch convolution kernels are 7, 9, and 11 respectively. Each branch outputs 5 feature maps, including feature signals of different bandpass frequencies. The set time convolution kernel is expressed as (1, K), and its step length is (1, 1). K represents the length of the filter on the time axis. Among the three branches, each branch outputs 5 feature maps, and the spatial convolution layer acts as a spatial filter to transmit information between different electrode channels; the set spatial convolution kernel is expressed as (N, 1), and its step length is (1, 1). N represents the number of electrode channels of the motor imagery EEG signal.

[0034] Step S05: performing average pooling dimensionality reduction processing on the plurality of output feature maps in the time dimension based on the preset spatial feature dimension and time feature dimension, and performing splicing processing on the channel dimension to obtain a target feature map; Specifically, in step S05 of the present invention, during the average pooling dimensionality reduction process, the pooling kernel size is (1, 85), the pooling step size is 1, and the SafeLog activation function is used for nonlinear transformation. The feature map size of each branch after the average pooling operation is (1, 61). During the splicing process, the mathematical expression of splicing is shown in formula (5): In formula (5), Represent the output feature maps of the three branches respectively. Represents the target feature map, which is distributed in four-dimensional space Among them, Indicates the batch size, represents the number of feature channels, represents the spatial feature dimension, Represents the time feature dimension.

[0035] The present invention splices the output feature maps in the channel dimension to obtain the target feature map. Although the traditional activation function is used for feature extraction and nonlinear mapping, it has the disadvantages of gradient vanishing and eigenvalue underflow. To this end, the present invention introduces the SafeLog activation function to perform nonlinear transformation on the target feature map, which effectively avoids the feature underflow problem when it is close to zero and improves the ability to extract weak features in the target feature map.

[0036] Step S06: obtaining a target convolution kernel based on the relationship between the number of channels of the target feature map and the convolution kernel, and obtaining a channel weight based on the target convolution kernel; In step S06 of the present invention, the mathematical expression for obtaining the target convolution kernel based on the ratio of the number of channels of the target feature map to the convolution kernel is shown in formula (6): In formula (6), represents the convolution kernel size of the target feature map, Represents the adjustment factor between the number of adjustment channels and the target feature map convolution kernel, represents the bias term, represents the number of feature channels, Indicates that odd numbers are used for calculation.

[0037] The process of obtaining channel weights based on the target convolution kernel includes: combining the target feature map with the target convolution kernel Perform element-by-element multiplication to obtain the convolution result, and normalize the convolution result to the interval [0,1] based on the Sigmoid activation function to obtain the channel weight. The mathematical expression of the convolution result is shown in formula (7): In formula (7), represents the convolution result, represents the convolution operation, represents the target feature map, Represents the target convolution kernel.

[0038] The mathematical expression of channel weight is shown in formula (8): In formula (8), represents the channel weight, Represents the Sigmoid activation function.

[0039] Step S07: performing weighted calculation on the target feature map based on the channel weight to obtain a spatiotemporal feature map; In step S07 of the present invention, the target feature map is weighted based on the channel weight to obtain a spatiotemporal feature map. The mathematical expression for obtaining the spatiotemporal feature map is shown in formula (9): In formula (9), represents the spatiotemporal feature map, represents the transpose symbol, represents element-by-element multiplication calculation, represents the channel weight. At the same time, in order to prevent overfitting, the present invention uses the maximum norm constraint to regularize the channel weight, that is, The class label corresponding to the maximum probability value is determined as the class label of the motor imagery EEG signal, and the class label is the decoding result. The network structure diagram of the channel attention mechanism module in step S07 of the present invention is shown in Figure 3 .

[0040] Step S08: The spatiotemporal feature map is sequentially mapped and activated to obtain a class label of the motor imagery EEG signal and a decoding result.

[0041] In step S08 of the present invention, a process of obtaining a decoding result is provided. First, an initial SMANet network model is constructed based on a Sinc convolution layer, a point convolution layer, a multi-branch spatiotemporal convolution network, a channel attention layer, a fully connected layer, and a classification layer. A total loss function is established based on a cross entropy loss function and a center loss function. The initial SMANet network model is trained based on the total loss function to obtain a trained SMANet network model. Secondly, after regularizing the weights of the fully connected layer based on the norm constraint, the spatiotemporal feature map is input into the fully connected layer of the trained SMANet network model for mapping and activation processing, and a class label of a motor imagery EEG signal is generated to obtain a decoding result. The network structure diagram of the channel attention mechanism module in step S08 of the present invention is shown in FIG. Figure 4 .

[0042] The central loss function set by the present invention Mathematical expression and cross entropy loss function The mathematical expressions of are shown in formula (10) and formula (11): In formula (10), represents the center loss function, Indicates The feature vector of each sample, Indicates The feature center of the category to which the sample belongs, Indicates the batch size of the minimum batch.

[0043] In formula (11), represents the cross entropy loss function, and denote the true class label and the predicted class label respectively, is the number of categories of motor imagery EEG signals.

[0044] The mathematical expression of the total loss function is the superposition of the center loss function and the cross entropy loss function. The mathematical expression of the total loss function is shown in formula (12): In formula (12), represents the total loss function, Represents the weighting coefficient.

[0045] The present invention uses the cross entropy loss function and the center loss function as the total loss function of the network model to train the initial SMANet network model. The cross entropy loss function is used to supervise the accuracy of the classification task, and optimizes the model's ability to distinguish the motor imagery EEG signal category by minimizing the probability distribution difference between the predicted category and the true label, thereby improving the classification performance of the motor imagery EEG signal; the center loss function is used to constrain the distribution density of similar samples in the feature space, and optimizes the motor imagery EEG signal feature representation by reducing the feature distance within the class and increasing the feature difference between the classes. By combining the cross entropy loss and the center loss as the total loss function of the network model, the accuracy and robustness of the classification of the motor imagery EEG signal are improved.

[0046] In order to further verify the feasibility of the method proposed in the present invention, the present invention carried out experimental simulation verification, and used the motor imagery datasets BCIC-IV2a, BCIC-IV2b and OpenBMI to illustrate the implementation of a motor imagery EEG signal decoding method based on Sinc filter provided by the present invention.

[0047] It should be noted that the data set selected for this experiment is described as follows: 1. BCIC-IV2a: This dataset contains motor imagery EEG data collected by 9 subjects performing motor imagery BCI experiments. This BCI experiment uses 22 electrodes placed in the international 10-20 system to record motor imagery EEG data for each subject for 2 sessions at a sampling rate of 250Hz. Four types of motor imagery tasks were performed in each session: left hand, right hand, foot, and tongue, and 72 experiments were performed for each task. The data from the first session was used for model training, and the data from the second session was used for model classification.

[0048] 2. BCIC-IV2b: This dataset contains motor imagery EEG data collected from 9 subjects performing motor imagery BCI experiments. This BCI experiment uses 3 electrodes placed in the international 10-20 system to record motor imagery EEG data for 5 sessions of each subject at a sampling rate of 250Hz. 120 experiments were performed in each of the first 2 sessions; 160 experiments were performed in each of the last 3 sessions. The number of experiments for the left hand and the right hand in each session is the same. The data from the first 3 sessions are used for model training, and the data from the last 2 sessions are used for model classification.

[0049] 3. OpenBMI: This dataset contains motor imagery EEG signal data collected by 54 subjects performing motor imagery BCI experiments. This BCI experiment uses 62 electrodes placed in the international 10-20 system to record motor imagery EEG signal data for each subject for 2 sessions at a sampling rate of 1000Hz. Two types of motor imagery tasks, left-hand and right-hand, were performed in each session, and 100 experiments were performed for each task. The data recorded in the first session was used for model training, and the data recorded in the second session was used for model classification.

[0050] The BCI experiment timing schemes for these three datasets are shown in Figure 2 . Figure 2 They represent the cue tone, fixation cross tone, motor imagery, and rest, respectively. When training the initial SMANet network model, subject-specific classification experiments were performed. For each subject, 80% of their training data were randomly selected as the training set, and the remaining 20% ​​were used as the validation set.

[0051] In order to reduce model overfitting and optimize its training process, the present invention adopts a two-stage training scheme. In the first stage, the network model is trained using the training set, and the classification accuracy and classification loss are detected using the verification set. The training is stopped when the accuracy is not improved after 200 consecutive training verifications; in the second stage, the network is initialized using the model parameters obtained in the first stage of training, and all training data (i.e., training set + verification set) are used as the training set to retrain the network model until the average loss of the verification set is lower than the verification loss of the first stage of training. Each experiment lasts about 8 seconds, including four stages: experimental preparation, visual cue, motor imagery, and short rest. This embodiment uses the 4 seconds of data from the start of visual cue to the end of motor imagery for model training and classification. The SMANet network model parameter settings selected in this experiment are shown in Table 1.

[0052] Table 1 SMANet network model parameter setting table In order to prevent the model from overfitting and ensure the termination of training, the present invention adopts an early stopping mechanism for the training process, and the maximum number of training times in the first and second stages is limited to 1500 and 600 respectively. The Adam optimizer is used to minimize the loss function, and the Adam parameters are set by default, that is, the learning rate is 0.001, and the two beta parameters are 0.9 and 0.999. The batch size is set to 16. PyTorch is used as the software environment for model building and training, and the hardware environment includes Intel Core i5-12400F CPU and NVIDIA GeForce RTX 3080 GPU.

[0053] In order to evaluate the performance of the SMANet network model, the method provided by the present invention is compared with five international classic traditional methods and deep learning methods. The five baseline methods are described as follows: 1. FBCSP method: This method first filters the original motor imagery EEG signal through a filter bank, and the signal is decomposed into sub-band signals in different frequency domains. Then, the FBCSP method is applied to each sub-band to extract spatial features. Finally, the features extracted from each sub-band are fused into a comprehensive feature vector. Compared with other traditional methods, this method can more comprehensively utilize the EEG features in different frequency domains, which helps to improve the classification performance of motor imagery EEG signals.

[0054] 2. ShallowConvNet method: This network is a shallow convolutional neural network structure, which contains a small number of convolutional layers, pooling layers and fully connected layers. Its convolutional layer consists of temporal convolution and spatial convolution, which respectively complete the functions of temporal filtering and spatial filtering. The simple design of this network makes it an effective alternative for tasks that do not require greater network complexity, and is suitable for situations where computing resources are limited or for rapid deployment.

[0055] 3. EEGNet method: This network is a lightweight and compact network designed to reduce the number of parameters and improve computational efficiency. It is implemented through depthwise separable convolution, which splits the standard convolution into depthwise convolution and pointwise convolution, significantly reducing the number of parameters and computational complexity. The network structure includes convolutional layers, pooling layers, and fully connected layers. It has a simple and efficient design and is very suitable for processing motor imagery EEG signal data.

[0056] 4. FBCNet method: This network uses multi-view data representation, then performs spatial filtering, and then calculates the temporal variance to extract discriminative features in the frequency domain, spatial domain, and temporal domain. This method enables the network to be effectively trained even under conditions of limited training data. This design enables the network to simultaneously learn feature information from motor imagery EEG signals in different frequency bands, improving the representation ability of the model.

[0057] 5. FBMSNet method: The network first uses filter bank technology to generate multi-view spectral representation of motor imagery EEG signal data, then applies hybrid deep convolution to extract multi-scale temporal features, and then performs spatial filtering to mitigate volume conduction effects. The subsequent processing steps are the same as FBCNet. The average accuracy and standard deviation of the proposed method and the five baseline methods on the three data sets are shown in Table 2.

[0058] Table 2 Average accuracy and standard deviation of the proposed method and five baseline methods on three datasets Table 2 shows the average accuracy and standard deviation of the trained SMANet network model and five baseline methods on three datasets. The average accuracy of SMANet on datasets BCIC-IV2a, BCIC-IV2b, and OpenBMI is 80.21%, 84.02%, and 72.70%, respectively, which is higher than all baseline methods. The average standard deviation of SMANet on these three datasets is smaller than that of most baseline methods. Compared with the traditional motor imagery EEG signal decoding method FBCSP, SMANet improves the classification accuracy by 12.51%, 15% and 12.42% on these three datasets respectively; compared with two classic deep learning methods ShallowConvNet / Motor imagery EEG signal Net, SMANet improves the classification accuracy by 5.71% / 7.11%, 4.86% / 2.50% and 10.98% / 8.69% respectively; compared with two state-of-the-art deep learning methods FBCNet / FBMSNet, SMANet improves the classification accuracy by 4.04% / 2.94%, 2.15% / 1.92% and 5.59% / 2.65% respectively. These results show that the trained SMANet network model not only has superior classification performance, but also has good classification robustness.

[0059] Example 2 See also Figure 5 , which is a schematic diagram of the structure of a motor imagery EEG signal decoding system based on a Sinc filter proposed in the second embodiment of the present application, and the system comprises: A Sinc convolution module 100, which is used to obtain a characteristic signal of the motor imagery EEG signal after performing convolution calculation on the motor imagery EEG signal and the Sinc filter; A point convolution module 200, which is used to combine the characteristic signals of the motor imagery EEG signals of different frequency bands according to a preset point convolution to generate a high-dimensional EEG characteristic signal; A multi-branch spatiotemporal convolution module 300, which is used to perform time domain feature extraction, spatial feature extraction, and average pooling dimensionality reduction processing on the high-dimensional EEG feature signal in sequence to obtain output feature maps of each branch, and to splice the output feature maps in the channel dimension to obtain a target feature map; A channel attention mechanism module 400, wherein the channel attention mechanism module is used to perform weighted calculation on the target feature map to obtain a spatiotemporal feature map; The classification module 500 is used to obtain the category probability after mapping and activating the spatiotemporal feature map, and obtain the class label of the motor imagery EEG signal based on the category probability to obtain a decoding result.

[0060] The present invention proposes a motor imagery EEG signal decoding system based on Sinc filter, which can utilize the Sinc convolution module 100 to extract signal features of each frequency domain range of the motor imagery EEG signal, solve the distortion problem occurring in the high-frequency range extraction process, and at the same time combine the multi-branch spatiotemporal convolution module 300 and the channel attention mechanism module 400 to perform deep coupling modeling of the time domain features and spatial features of the motor imagery EEG signal, thereby improving the accuracy and robustness in the motor imagery EEG signal classification process.

[0061] A motor imagery EEG signal decoding system based on a Sinc filter in an embodiment of the present application can be a system, or a component, an integrated circuit, or a chip in a terminal. The system can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0062] A motor imagery EEG signal decoding system based on a Sinc filter in the embodiment of the present application may be a system having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0063] The motor imagery EEG signal decoding system based on Sinc filter provided in the embodiment of the present application can realize Figures 1 to 4 In the method embodiment, each process of a method for decoding motor imagery EEG signals based on Sinc filter will not be described here to avoid repetition.

[0064] Optionally, an embodiment of the present application also provides a motor imagery EEG signal decoding system based on a Sinc filter, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, each process of the above-mentioned method embodiment of motor imagery EEG signal decoding based on a Sinc filter is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0065] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned embodiment of a method for decoding motor imagery EEG signals based on a Sinc filter is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0066] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0067] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0068] It should be noted that, in this article, the term "comprises", "includes" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or system including the element. In addition, it should be pointed out that the scope of the method and system in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0069] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0070] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for decoding motor imagery EEG signals based on Sinc filter, characterized in that: include: Convolution calculation is performed on the motor imagery EEG signal and the Sinc filter to obtain the characteristic signal of the motor imagery EEG signal; Based on point convolution, the characteristic signals of motor imagery EEG signals of different frequency bands are connected through channels to generate high-dimensional EEG characteristic signals; Based on the temporal convolution layer, spatial convolution layer and average pooling layer, a parallel multi-branch spatiotemporal convolutional network is built; Based on the multi-branch spatiotemporal convolutional network, respectively, the high-dimensional EEG feature signal is subjected to time domain feature extraction, spatial feature extraction, and average pooling dimensionality reduction processing in sequence to obtain an output feature map corresponding to each branch; Based on the preset spatial feature dimension and temporal feature dimension, the plurality of output feature maps are subjected to average pooling dimensionality reduction processing in the time dimension, and are concatenated in the channel dimension to obtain a target feature map; Obtaining a target convolution kernel based on the relationship between the number of channels of the target feature map and the convolution kernel, and obtaining a channel weight based on the target convolution kernel; Performing weighted calculation on the target feature map based on the channel weight to obtain a spatiotemporal feature map; The spatiotemporal feature map is sequentially subjected to mapping processing and activation processing to obtain a class label of the motor imagery EEG signal and obtain a decoding result.

2. The method for decoding motor imagery EEG signals based on Sinc filter according to claim 1, characterized in that: The process of performing convolution calculation on the motor imagery EEG signal and the Sinc filter to obtain the characteristic signal of the motor imagery EEG signal includes: Acquire a frequency domain representation of the filter bank based on the cutoff frequency, and inversely transform the frequency domain representation based on an inverse Fourier transform to acquire a time domain representation of the filter bank; The time domain representation is subjected to windowing processing to obtain a Sinc filter, and the motor imagery EEG signal and the Sinc filter are subjected to convolution calculation to obtain a characteristic signal of the motor imagery EEG signal.

3. The method for decoding motor imagery EEG signals based on Sinc filter according to claim 1, characterized in that: The mathematical expression of the Sinc filter is: in, represents the time domain expression of the Sinc filter, represents the time domain expression of the Sinc filter before windowing, represents the length of the motor imagery EEG signal, represents the length of the window, is the low cutoff frequency, is the high cutoff frequency.

4. The method for decoding motor imagery EEG signals based on Sinc filter according to claim 1, characterized in that: The parallel multi-branch spatiotemporal convolutional network is connected in parallel through multiple branches, and each branch consists of, from left to right, a temporal convolution layer, a spatial convolution layer, and an average pooling layer. The number of convolution kernels in the temporal convolution layer of each branch is the same as the number of convolution kernels in the spatial convolution layer. The sizes of the convolution kernels of the temporal convolution layers in different branches are different, and the sizes of the convolution kernels of the spatial convolution layers are the same.

5. The method for decoding motor imagery EEG signals based on Sinc filter according to claim 1, characterized in that: The process of obtaining the target feature map includes: The convolution kernel sizes in the time convolution layers of the multiple branches are set based on the length of the filter on the time axis to obtain the set time convolution kernels. The time convolution layers of the multiple branches perform time domain feature extraction according to the set time convolution kernels to obtain multiple time domain feature maps. The convolution kernel size in the spatial convolution layer of multiple branches is set based on the number of electrode channels of the motor imagery EEG signal, and the set spatial convolution kernel is obtained. After the spatial convolution layer of multiple branches performs spatial feature extraction according to the set spatial convolution kernel, multiple spatial feature maps are obtained, and the spatial feature maps and the time domain feature maps are fused to generate multiple output feature maps; Based on the preset spatial feature dimension and temporal feature dimension, multiple output feature maps are average-pooled and dimension-reduced in the time dimension and concatenated in the channel dimension to obtain the target feature map.

6. The method for decoding motor imagery EEG signals based on Sinc filter according to claim 5, characterized in that: The size of the set temporal convolution kernel and the size of the set spatial convolution kernel are (1, K) and (N, 1), respectively, where K represents the length of the filter on the time axis, and N represents the number of electrode channels of the motor imagery EEG signal.

7. The method for decoding motor imagery EEG signals based on Sinc filter according to claim 1, characterized in that: The process of sequentially performing mapping processing and activation processing on the spatiotemporal feature map to obtain a class label of the motor imagery EEG signal and obtain a decoding result includes: An initial SMANet network model is constructed based on a Sinc convolution layer, a point convolution layer, a multi-branch spatiotemporal convolution network, a channel attention layer, a fully connected layer, and a classification layer; a total loss function is established based on a cross entropy loss function and a center loss function; and the initial SMANet network model is trained based on the total loss function to obtain a trained SMANet network model; After the weights of the fully connected layer are regularized based on the norm constraint, the spatiotemporal feature map is input into the fully connected layer of the trained SMANet network model for mapping and activation processing, and the class label of the motor imagery EEG signal is generated to obtain the decoding result.

8. A motor imagery EEG signal decoding system based on Sinc filter, characterized in that: include: A Sinc convolution module, wherein the Sinc convolution module is used to obtain a characteristic signal of the motor imagery EEG signal after performing convolution calculation on the motor imagery EEG signal and the Sinc filter; A point convolution module, the point convolution module is used to combine the characteristic signals of the motor imagery EEG signals of different frequency bands according to a preset point convolution to generate a high-dimensional EEG characteristic signal; A multi-branch spatiotemporal convolution module, which is used to perform time domain feature extraction, spatial feature extraction, and average pooling dimensionality reduction processing on the high-dimensional EEG feature signal in sequence to obtain output feature maps of each branch, and to splice the output feature maps in the channel dimension to obtain a target feature map; A channel attention mechanism module, wherein the channel attention mechanism module is used to perform weighted calculation on the target feature map to obtain a spatiotemporal feature map; A classification module is used to obtain a category probability after mapping and activating the spatiotemporal feature map, and obtain a class label of the motor imagery EEG signal based on the category probability to obtain a decoding result.

9. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of a method for decoding motor imagery EEG signals based on a Sinc filter as described in claims 1-7 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps of the method for decoding motor imagery EEG signals based on Sinc filters as described in claims 1-7 are implemented.

Images