Frequency prior feature guided lightweight motor imagery classification method and device

Abstract

The invention relates to the technical field of neural decoding, in particular to a frequency prior feature guided lightweight motor imagery classification method and device. The method comprises the following steps: extracting time domain features of a high-dimensional EEG signal; performing weighted integration on EEG signals collected by different electrodes through a spatial convolution layer; down-sampling is carried out through average pooling, and high-frequency information of the EEG signals after weighted integration is compressed; inputting the compressed EEG signal into a residual network, processing high-frequency information of the compressed EEG signal, and further optimizing extraction of signal features; through the design of causal convolution and expansion convolution, the time sequence characteristics of the EEG signals are effectively captured, and it is ensured that the network structure has good calculation efficiency and interpretability. According to the method, an efficient and explainable solution is provided in EEG signal analysis, and the performance of the model can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of neural decoding technology, and in particular to a lightweight motor imagery classification method and device guided by frequency prior features. Background Art

[0002] In the field of brain-computer interface (BCI), human-computer interaction through decoding EEG signals has become an important direction of research and application. Among them, motor imagery (MI) refers to the activation of activities in the brain area related to certain motor behaviors by imagining them without actual limb movement. During the motor imagery process, electroencephalogram (EEG) signals are recorded as a bioelectric signal that directly reflects brain electrical activity, which is used to distinguish and identify the individual's motor intentions. This technology has been widely used in stroke rehabilitation, wheelchair control and other fields, and its application prospects are constantly expanding with the development of technology.

[0003] However, EEG signals are susceptible to a variety of interferences and noises during the acquisition process, such as eye movements, muscle artifacts, and environmental noise. These noises and interferences will not only significantly reduce the quality of EEG signals, but may also lead to a significant decrease in classification accuracy in motor imagery tasks. In addition, EEG signals have strong nonlinear characteristics, and their time-frequency characteristics vary greatly among different individuals and under different experimental conditions. Traditional linear methods often find it difficult to effectively capture and utilize these complex patterns and features, which in turn affects the recognition effect of the model. Therefore, how to effectively remove noise and improve the classification accuracy of EEG signals has become an important technical problem in the application of BCI technology.

[0004] To solve the above problems, with the continuous development of deep learning technology, researchers have tried to use methods based on deep neural networks (DNN) to automatically extract and classify EEG signals. Compared with traditional signal processing and pattern recognition methods, deep learning models can automatically extract high-level features from raw EEG signals, overcoming the limitations of manually designed features, thereby achieving higher classification accuracy in motor imagery tasks. In particular, deep learning models such as convolutional neural networks (CNNs) perform well in extracting spatiotemporal features, can better capture the time-frequency features in EEG signals, and maintain high stability under a variety of experimental conditions.

[0005] The nonlinear characteristics and individual differences of EEG signals require that deep learning models can effectively deal with signal uncertainties and have a certain degree of adaptive ability. In this context, combining prior knowledge in the field of brain science to design the structure and training strategy of deep learning models has become an effective way to improve model performance. For example, prior knowledge such as signal analysis based on brain functional areas and spatiotemporal patterns of neural signals can help deep learning models more accurately capture features related to motor imagery when processing EEG signals and improve classification accuracy. At the same time, combined with the concept of lightweight model design, the computational complexity of the model can be effectively reduced, making it adaptable to computing resource constraints in real-time application scenarios.

[0006] At present, there are only a few papers and related patents on motor imagery classification methods. It is also very rare to use deep learning technology to complete neural decoding to achieve motor imagery classification in this field. According to investigation and understanding, the currently disclosed existing technologies are as follows:

[0007] Lawhern et al. proposed EEGNet in their paper [Lawhern VJ, Solon AJ, Waytowich NR, et al.EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces[J]. Journal of neural engineering, 2018, 15(5): 056013]. EEGNet is a deep convolutional neural network (CNN) architecture designed for electroencephalogram (EEG) signal classification. It can efficiently extract features and adapt to the spatiotemporal characteristics of EEG data when processing EEG signals. The main advantages of EEGNet are its lightweight structure and high computational efficiency, which makes it particularly suitable for real-time brain-computer interface (BCI) applications. The architectural design of EEGNet is inspired by convolutional neural networks (CNNs), but is optimized for the characteristics of EEG signals. The lightweight nature of EEGNet makes it widely used in BCI applications, especially for real-time brain-computer interface systems that require high computing resources. However, the classification effect of EEGNet in practice is difficult to meet the specified requirements.

[0008] Ma et al. proposed a Transformer-based classification method in the paper [Ma X, Chen W, Pei Z, et al. Attention-based convolutional neural network with multi-modal temporal information fusion for motor imagery EEG decoding[J]. Computers in Biology and Medicine, 2024, 175:108504]. EEGTransNet is an electroencephalogram (EEG) signal classification model based on the transformer architecture, which is designed to process and analyze the temporal and spatial information in EEG signals. Compared with traditional convolutional neural networks (CNNs), EEGTransNet uses the self-attention mechanism of the transformer to more efficiently capture long-term dependencies and temporal patterns in EEG signals. EEGTransNet has performed well in multiple EEG classification tasks, especially when processing complex EEG signals, it can significantly improve classification accuracy. Thanks to the powerful modeling ability of the transformer architecture for long-range dependencies and spatiotemporal features, EEGTransNet has shown great potential in the field of brain-computer interfaces (BCI). However, its running time and computational complexity often make it difficult to meet the real-time requirements in practical scenarios.

[0009] Existing EEG signal samples are usually limited. Although neural networks have strong nonlinear fitting capabilities, they are prone to overfitting in this case. In addition, most neural network designers are not experts in the field of brain science. Therefore, while existing models emphasize powerful feature extraction capabilities, they often ignore the integration of prior knowledge in brain science, which can effectively improve the performance and interpretability of the model. Furthermore, computing resources in actual brain-computer interface (BCI) applications are usually limited, so it is very necessary to design lightweight and efficient models that are more suitable for deployment in actual scenarios.

[0010] EEG signals are a typical time series data with complex temporal dependencies and high non-stationarity, which makes it a challenging task to extract meaningful features from them. Traditional neural network architectures, especially convolutional neural networks (CNNs) and recurrent neural networks (RNNs), often face problems such as overfitting, high computational effort, and difficulty in capturing long-term dependencies when processing EEG signals. Summary of the invention

[0011] In order to solve the technical problems of overfitting, large amount of calculation and difficulty in capturing long-term dependence when processing EEG signals in the prior art, the embodiment of the present invention provides a lightweight motor imagery classification method and device guided by frequency prior features. The technical solution is as follows:

[0012] On the one hand, a lightweight motor imagery classification method guided by frequency prior features is provided, characterized in that the method comprises:

[0013] S1, obtaining a high-dimensional EEG signal, performing preliminary processing on the high-dimensional EEG signal, and extracting the time domain features of the high-dimensional EEG signal;

[0014] S2, input the EEG signal after extracting the time domain features into the spatial convolution layer; the EEG signals collected by different electrodes are weighted and integrated through the spatial convolution layer, and merged into a single channel; down-sampling is performed through average pooling, and the high-frequency information of the weighted integrated EEG signal is compressed to below 30Hz to obtain a compressed EEG signal;

[0015] S3, inputting the compressed EEG signal into the residual network, processing the high-frequency information of the compressed EEG signal, further optimizing the extraction of signal features, and obtaining the spatiotemporal features of the EEG signal;

[0016] S4. Sequence modeling is performed through a temporal convolutional network designed based on causal convolution. The previous sequence information is represented by the last time step to obtain a sparse expression of the features and a lightweight expression of the model. The classifier is used to complete lightweight motor imagery classification guided by frequency prior features.

[0017] Optionally, S1, obtaining a high-dimensional EEG signal, performing preliminary processing on the high-dimensional EEG signal, and extracting time domain features of the high-dimensional EEG signal, including:

[0018] Get a high-dimensional EEG input signal, where the shape of the input signal is N×1×C×T, where N is the number of samples, C is the number of electrodes, and T is the number of time samples;

[0019] The signal is preliminarily processed through a 1D convolution operation with weight constraints to extract preliminary features;

[0020] The preliminary features enter the batch normalization layer to standardize the preliminary features;

[0021] Using the exponential linear unit as the activation function, the data is nonlinearly modeled and mapped into a nonlinear space to obtain the time domain feature representation.

[0022] Optionally, a 1D convolution operation with weight constraints, including:

[0023] The size of the convolution kernel K1 is determined by the design parameters of the temporal convolution block, which are calculated based on the sampling rate Fs of the device and the predetermined parameter T:

[0024]

[0025] Among them, T is set to 4 or 8; when the number of electrodes is large, T is set to 8; conversely, when the number of electrodes is small, T is set to 4.

[0026] Optionally, downsampling is performed by average pooling to compress the high-frequency information of the weighted integrated EEG signal to below 30 Hz to obtain a compressed EEG signal, including:

[0027] The size of the downsampling operation is determined by the following formula:

[0028] ;

[0029] in, Represents the convolution kernel size used for average pooling; Represents data points within one second.

[0030] Optionally, the equivalent receptive field of the residual network, that is, the equivalent convolution kernel size K3 represented by the multi-layer network, is calculated as:

[0031] .

[0032] Optionally, in S3, the number of residual blocks is adjusted according to the actual number and spatial distribution of electrodes.

[0033] Optionally, sequence modeling is performed through a temporal convolutional network designed based on causal convolution, and the previous sequence information is represented by the last time step to obtain a sparse expression of features and a lightweight expression of the model; and lightweight motor imagery classification guided by frequency prior features is completed through a classifier, including:

[0034] Get time series signal , fill the time series signal to obtain a new sequence signal ;

[0035] The new sequence signal is input into the convolution layer of the temporal convolutional network for convolution operation. The output of the next layer is only related to the previous data.

[0036] Through multi-layer convolution operations, the last time point / step will obtain all the previous information to form a new feature expression. At this time, the entire sequence is not needed, thus obtaining a lightweight representation of the model;

[0037] Motor imagery classification is performed through classifiers to obtain new representational expressions and complete lightweight motor imagery classification guided by frequency prior features.

[0038] On the other hand, a lightweight motor imagery classification device guided by a frequency prior feature is provided, and the device is applied to a lightweight motor imagery classification method guided by a frequency prior feature, and the device comprises:

[0039] A time domain feature extraction module is used to obtain high-dimensional EEG signals, perform preliminary processing on the high-dimensional EEG signals, and extract time domain features of the high-dimensional EEG signals;

[0040] The signal compression module is used to input the EEG signal after extracting the time domain features into the spatial convolution layer; the EEG signals collected by different electrodes are weighted and integrated through the spatial convolution layer and fused into a single channel; down-sampling is performed through average pooling to compress the high-frequency information of the weighted integrated EEG signal to below 30 Hz to obtain a compressed EEG signal;

[0041] The optimization extraction module is used to input the compressed EEG signal into the residual network, process the high-frequency information of the compressed EEG signal, further optimize the extraction of signal features, and obtain the spatiotemporal features of the EEG signal;

[0042] The sequence modeling module is used to perform sequence modeling through a temporal convolutional network designed based on causal convolution. The previous sequence information is represented by the last time step to obtain a sparse expression of features and a lightweight expression of the model. The classifier is used to complete lightweight motor imagery classification guided by frequency prior features.

[0043] On the other hand, a lightweight motor imagery classification device guided by frequency prior features is provided, and the lightweight motor imagery classification device guided by frequency prior features comprises: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the lightweight motor imagery classification methods guided by the above-mentioned frequency prior features is implemented.

[0044] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned frequency prior feature guided lightweight motor imagery classification methods.

[0045] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0046] In the embodiment of the present invention, a method combining a temporal convolutional network and a residual network is proposed. Through the design of causal convolution and dilated convolution, the temporal characteristics of EEG signals are effectively captured, and the network structure is ensured to have good computational efficiency and interpretability. This method provides an efficient and interpretable solution in EEG signal analysis, which can significantly improve the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A schematic flow chart of a lightweight motor imagery classification method guided by frequency prior features provided by an embodiment of the present invention;

[0049] Figure 2 A REDIES structure diagram provided for an embodiment of the present invention;

[0050] Figure 3 A block diagram of a lightweight motor imagery classification device guided by frequency prior features provided in an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0053] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0054] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0055] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0056] The embodiment of the present invention provides a lightweight motor imagery classification method guided by a frequency prior feature, which can be implemented by a lightweight motor imagery classification device guided by a frequency prior feature, and the lightweight motor imagery classification device guided by a frequency prior feature can be a terminal or a server. Figure 1 The flowchart of the lightweight motor imagery classification method guided by frequency prior features is shown in Figure 1 As shown, the lightweight motor imagery classification method guided by frequency prior features proposed in the present invention is as follows: Figure 2 The figure shows a REDIES structure diagram provided by an embodiment of the present invention; the processing flow of the method may include the following steps:

[0057] S1, obtaining a high-dimensional EEG signal, performing preliminary processing on the high-dimensional EEG signal, and extracting the time domain features of the high-dimensional EEG signal;

[0058] In a feasible implementation, S1, obtaining a high-dimensional EEG signal, performing preliminary processing on the high-dimensional EEG signal, and extracting the time domain features of the high-dimensional EEG signal, including:

[0059] Get a high-dimensional EEG input signal, where the shape of the input signal is N×1×C×T, where N is the number of samples, C is the number of electrodes, and T is the number of time samples;

[0060] In order to process these high-dimensional EEG signals, the signals are first processed by a 1D convolution operation with weight constraints (Conv1dWithConstraint) to extract preliminary features;

[0061] The preliminary features enter the batch normalization layer to standardize the preliminary features;

[0062] Using the exponential linear unit as the activation function, the data is nonlinearly modeled and mapped into a nonlinear space to obtain the time domain feature representation.

[0063] In one feasible implementation, a 1D convolution operation with weight constraints includes:

[0064] The size of the convolution kernel K1 is determined by the design parameters of the temporal convolution block, which are calculated based on the sampling rate Fs of the device and the predetermined parameter T:

[0065]

[0066] Among them, T is set to 4 or 8; when the number of electrodes is large, T is set to 8 to extract more information; conversely, when the number of electrodes is small, T is set to 4 to better adapt to the feature extraction requirements of the signal.

[0067] In a feasible implementation, in this way, we can choose the appropriate convolution kernel size according to the actual situation, avoid over-smoothing the signal, and retain enough time information. After extracting preliminary features through the convolution layer, the signal enters the batch normalization layer. The role of batch normalization is to standardize the output of the convolution layer, reduce internal covariate shift, thereby accelerating the training process and improving the stability of the model.

[0068] In one feasible implementation, an exponential linear unit (ELU) is used as an activation function. For the positive part of the time data, the positive features are retained. For the negative part, the activation function will output an exponential decay to provide gradient information for the negative area, thereby performing nonlinear modeling on the data. ELU has strong nonlinear modeling capabilities and can effectively alleviate the gradient vanishing problem in neural networks. This combination ensures that the model can fully exploit the nonlinear relationship in the EEG signal during the learning process.

[0069] S2. Input the EEG signal after extracting the time domain features into the spatial convolution layer; perform weighted integration of the EEG signals collected by different electrodes through the spatial convolution layer and fuse them into a single channel; perform down-sampling through average pooling, compress the high-frequency information of the weighted integrated EEG signal to below 30 Hz, and obtain a compressed EEG signal.

[0070] In a feasible implementation, after extracting the time domain features, the EEG signal enters the spatial convolution layer. The spatial convolution layer uses a convolution kernel of size (C, 1) to weightedly integrate the EEG signals collected by different electrodes and fuse them into a single channel. This operation can learn the spatial correlation between different electrodes and provide strong support for subsequent signal processing.

[0071] In a feasible implementation, downsampling is performed by average pooling to compress the high-frequency information of the weighted integrated EEG signal to below 30 Hz to obtain a compressed EEG signal, including:

[0072] The size of the downsampling operation is determined by the following formula:

[0073] ;

[0074] in, Represents the convolution kernel size used for average pooling; Represents data points within one second.

[0075] In a feasible implementation, assuming that the sampling rate of the current device is , one second of data will be The points indicate that points as a group, The values ​​of the points are averaged as new points, and the original long signal sequence will be compressed into a new signal sequence, and the high-frequency information of the signal is compressed to about 30Hz. This operation not only reduces the redundant information in the signal, but also reduces the computational burden of the subsequent model.

[0076] By using this method, we are able to effectively limit the frequency range of the EEG signal to around 30 Hz, remove unnecessary high-frequency noise, and ensure that the temporal dependencies of the signal are fully preserved.

[0077] S3, inputting the compressed EEG signal into the residual network, processing the high-frequency information of the compressed EEG signal, further optimizing the extraction of signal features, and obtaining the spatiotemporal features of the EEG signal;

[0078] In a feasible implementation, in S3, the number of residual blocks is adjusted according to the actual number and spatial distribution of electrodes.

[0079] In a feasible implementation, the signal compressed to 30Hz will enter a small residual network. In order to further optimize the extraction of signal features, we introduced a residual network into the model. The signal first passes through several convolution filtering operations. The size of the small convolution kernel in the front order will extract higher frequency bands in the compressed signal. Through multiple small convolution kernels, the receptive field of the network will continue to expand until it expands to a specified length, and sufficient low-frequency information can be extracted. Through the connection between the layers of the residual network, the relatively high frequency band of the compressed signal will be retained and fused with the low-frequency information. The residual network transmits information between each layer through skip connections, avoiding the gradient vanishing problem and being able to retain more high-frequency information. Considering the number of electrodes and spatial distribution, the number of residual blocks needs to be adjusted according to the specific situation. Adding residual blocks can expand the receptive field of the model, thereby capturing more spatiotemporal features. Compared with the traditional multi-branch network structure, the residual network is more efficient in processing high-frequency information and reduces the amount of calculation.

[0080] In a feasible implementation, the equivalent receptive field of the residual network, that is, the equivalent convolution kernel size K3 represented by the multi-layer network, is calculated as follows:

[0081] .

[0082] In a feasible implementation, it should be noted that although increasing the depth of the network can improve the receptive field, it may also cause the model to pay more attention to low-frequency information, thereby affecting the accuracy of the model. Therefore, it is necessary to balance depth and accuracy when designing the network to ensure that the model can retain high-frequency information without paying too much attention to low-frequency components.

[0083] S4. Sequence modeling is performed through a temporal convolutional network designed based on causal convolution to complete lightweight motor imagery classification guided by frequency prior features.

[0084] In a feasible implementation, sequence modeling is performed by using a temporal convolutional network designed based on causal convolution, and the previous sequence information is represented by the last time step to obtain a sparse expression of features and a lightweight expression of the model; and lightweight motor imagery classification guided by frequency prior features is completed by a classifier, including:

[0085] Get time series signal , fill the time series signal to obtain a new sequence signal ;

[0086] The new sequence signal is input into the convolution layer of the temporal convolutional network for convolution operation. The output of the next layer is only related to the previous data.

[0087] Through multi-layer convolution operations, the last time point / step will obtain all the previous information to form a new feature expression. At this time, the entire sequence is not needed, thus obtaining a lightweight representation of the model;

[0088] Motor imagery classification is performed through classifiers to obtain new representational expressions and complete lightweight motor imagery classification guided by frequency prior features.

[0089] In a feasible implementation, the new sequence signal is input into the convolution layer of the temporal convolutional network for convolution operation, and the output of the next layer is only related to the previous data, including:

[0090] The input sequence is represented as , first fill it and get a new sequence , then convolution is performed again, the output of the next layer Will only related, Will only Similarly, the output of the next layer will only be related to the previous data. After multiple layers of causal convolution operations, a new feature expression will be obtained, which means that the output of each time step only depends on the current and previous input signals, thus ensuring temporal causality.

[0091] In a feasible implementation, through this design, the model can effectively eliminate the influence of future signals on current signals and ensure that time dependencies are strictly maintained. In addition, the dilated convolution in TCN expands the receptive field by introducing gaps between filter elements, allowing the network to capture dependencies over longer time spans without increasing computational complexity. Compared with traditional recurrent neural networks (RNNs, such as LSTM and GRU), TCN has the advantages of parallel computing, faster training speed, and excellent performance in a variety of sequence modeling tasks. Its simple architecture and low computational complexity make it have strong application potential in EEG signal processing and other time series data analysis.

[0092] In the embodiment of the present invention, there is no need to introduce complex Transformer, channel or spatial attention mechanism, or multi-branch structure, and still achieves leading performance in the motor imagery task. This achievement proves that superior results can still be obtained without increasing the complexity of the network, highlighting the efficiency of the method. On the BCI-2A dataset and BCI-2B dataset, this method achieved an accuracy of 86.23% and 86.75%, and at the same time, compared with the mainstream Transformer architecture, the computational complexity is only 50%.

[0093] By focusing on selecting appropriate frequency bands, our model can significantly improve computational efficiency and reduce the risk of overfitting without sacrificing performance. This approach demonstrates efficient EEG signal processing and feature extraction capabilities, which is suitable for resource-limited scenarios in real-world applications.

[0094] By combining prior knowledge from brain science and calculating the appropriate convolution kernel size and compression signal ratio, this paper not only optimizes the structure of the model, but also enhances its interpretability. In the motor imagery task, reasonable frequency band selection and simplified network design help the model capture key features related to brain activity more clearly. This interpretability is particularly important for applications in fields such as brain-computer interfaces (BCI), which can enhance users' trust and the application potential of the model in actual scenarios.

[0095] Figure 3 is a block diagram of a frequency prior feature guided lightweight motor imagery classification device 300 according to an exemplary embodiment, and the device 300 is used for a frequency prior feature guided lightweight motor imagery classification method. Figure 3 The device includes a time domain feature extraction module 310, a signal compression module 320, an optimization extraction module 330 and a sequence modeling module 340. Among them:

[0096] The time domain feature extraction module 310 is used to obtain high-dimensional EEG signals, perform preliminary processing on the high-dimensional EEG signals, and extract time domain features of the high-dimensional EEG signals;

[0097] The signal compression module 320 is used to input the EEG signal after extracting the time domain features into the spatial convolution layer; the EEG signals collected by different electrodes are weighted and integrated through the spatial convolution layer, and merged into a single channel; down-sampling is performed through average pooling, and the high-frequency information of the weighted integrated EEG signal is compressed to below 30 Hz to obtain a compressed EEG signal;

[0098] The optimization extraction module 330 is used to input the compressed EEG signal into the residual network, process the high-frequency information of the compressed EEG signal, further optimize the extraction of signal features, and obtain the spatiotemporal features of the EEG signal;

[0099] The sequence modeling module 340 is used to perform sequence modeling through a temporal convolutional network designed based on causal convolution, represent the previous sequence information through the last time step, obtain a sparse expression of features and a lightweight expression of the model; and complete lightweight motor imagery classification guided by frequency prior features through a classifier.

[0100] Optionally, a high-dimensional EEG signal is obtained, and the high-dimensional EEG signal is preliminarily processed to extract time domain features of the high-dimensional EEG signal, including:

[0101] Get a high-dimensional EEG input signal, where the shape of the input signal is N×1×C×T, where N is the number of samples, C is the number of electrodes, and T is the number of time samples;

[0102] The signal is preliminarily processed through a 1D convolution operation with weight constraints to extract preliminary features;

[0103] The preliminary features enter the batch normalization layer to standardize the preliminary features;

[0104] Using the exponential linear unit as the activation function, the data is nonlinearly modeled and mapped into a nonlinear space to obtain the time domain feature representation.

[0105] Optionally, a 1D convolution operation with weight constraints, including:

[0106] The size of the convolution kernel K1 is determined by the design parameters of the temporal convolution block, which are calculated based on the sampling rate Fs of the device and the predetermined parameter T:

[0107]

[0108] Among them, T is set to 4 or 8; when the number of electrodes is large, T is set to 8; conversely, when the number of electrodes is small, T is set to 4.

[0109] Optionally, downsampling is performed by average pooling to compress the high-frequency information of the weighted integrated EEG signal to below 30 Hz to obtain a compressed EEG signal, including:

[0110] The size of the downsampling operation is determined by the following formula:

[0111] ;

[0112] in, Represents the convolution kernel size used for average pooling; Represents data points within one second.

[0113] Optionally, the equivalent receptive field calculation formula of the residual network is:

[0114] .

[0115] Optionally, the number of residual blocks is adjusted according to the actual number and spatial distribution of electrodes.

[0116] Optionally, the sequence modeling module 340 is used to obtain the time series signal , fill the time series signal to obtain a new sequence signal ;

[0117] The new sequence signal is input into the convolution layer of the temporal convolutional network for convolution operation. The output of the next layer is only related to the previous data.

[0118] Through multi-layer convolution operations, the last time point / step will obtain all the previous information to form a new feature expression. At this time, the entire sequence is not needed, thus obtaining a lightweight representation of the model;

[0119] Motor imagery classification is performed through classifiers to obtain new representational expressions and complete lightweight motor imagery classification guided by frequency prior features.

[0120] Figure 4 is a structural schematic diagram of a lightweight motor imagery classification device guided by frequency prior features provided by an embodiment of the present invention, such as Figure 4 As shown, the lightweight motor imagery classification device guided by frequency prior features may include the above Figure 3 The frequency prior feature guided lightweight motor imagery classification device shown. Optionally, the frequency prior feature guided lightweight motor imagery classification device 410 may include a first processor 2001.

[0121] Optionally, the frequency prior feature guided lightweight motor imagery classification device 410 may further include a memory 2002 and a transceiver 2003 .

[0122] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0123] Combine the following Figure 4 The components of the lightweight motor imagery classification device 410 guided by frequency prior features are specifically introduced as follows:

[0124] The first processor 2001 is the control center of the frequency prior feature guided lightweight motor imagery classification device 410, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention, such as one or more microprocessors (digital signal processor, DSP), or one or more field programmable gate arrays (field programmable gate array, FPGA).

[0125] Optionally, the first processor 2001 can perform various functions of the frequency prior feature guided lightweight motor imagery classification device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0126] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 are shown in FIG.

[0127] In a specific implementation, as an embodiment, the frequency prior feature guided lightweight motor imagery classification device 410 may also include multiple processors, such as Figure 4 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0128] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0129] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be connected to the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0130] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0131] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0132] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and may be connected to the interface circuit of the lightweight motor imagery classification device 410 guided by the frequency prior feature ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0133] It should be noted that Figure 4 The structure of the frequency prior feature guided lightweight motor imagery classification device 410 shown in the figure does not constitute a limitation on the router, and the actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0134] In addition, the technical effects of the lightweight motor imagery classification device 410 guided by frequency prior features can refer to the technical effects of the lightweight motor imagery classification method guided by frequency prior features described in the above method embodiment, and will not be repeated here.

[0135] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0136] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0137] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable sensors. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0138] It should be understood that the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship, which can be understood by referring to the context before and after.

[0139] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0140] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0143] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

[0144] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A lightweight motor imagery classification method guided by frequency prior features, characterized in that: The method comprises: S1. Acquire a high-dimensional EEG signal, perform preliminary processing on the high-dimensional EEG signal, and extract time domain features of the high-dimensional EEG signal; S2, inputting the EEG signal after extracting the time domain features into the spatial convolution layer; weighting and integrating the EEG signals collected by different electrodes through the spatial convolution layer, and fusing them into a single channel; down-sampling by average pooling, compressing the high-frequency information of the weighted integrated EEG signal to below 30 Hz, and obtaining a compressed EEG signal; S3, inputting the compressed EEG signal into a residual network, processing the high-frequency information of the compressed EEG signal, further optimizing the extraction of signal features, and obtaining the spatiotemporal features of the EEG signal; S4. Sequence modeling is performed through a temporal convolutional network designed based on causal convolution. The previous sequence information is represented by the last time step to obtain a sparse expression of the features and a lightweight expression of the model. The classifier is used to complete lightweight motor imagery classification guided by frequency prior features.

2. The lightweight motor imagery classification method guided by frequency prior features according to claim 1 is characterized in that: S1, obtaining a high-dimensional EEG signal, performing preliminary processing on the high-dimensional EEG signal, and extracting time domain features of the high-dimensional EEG signal, including: Get a high-dimensional EEG input signal, where the shape of the input signal is N×1×C×T, where N is the number of samples, C is the number of electrodes, and T is the number of time samples; The signal is preliminarily processed through a 1D convolution operation with weight constraints to extract preliminary features; The preliminary features enter the batch normalization layer to perform standardization on the preliminary features; Using the exponential linear unit as the activation function, the data is nonlinearly modeled and mapped into a nonlinear space to obtain the time domain feature representation.

3. The lightweight motor imagery classification method guided by frequency prior features according to claim 2 is characterized in that: The 1D convolution operation with weight constraints includes: The size of the convolution kernel K1 is determined by the design parameters of the temporal convolution block, which are calculated based on the sampling rate Fs of the device and the predetermined parameter T: ； Among them, T is set to 4 or 8; when the number of electrodes is large, T is set to 8; conversely, when the number of electrodes is small, T is set to 4.

4. The lightweight motor imagery classification method guided by frequency prior features according to claim 3 is characterized in that: The downsampling is performed by average pooling to compress the high-frequency information of the weighted integrated EEG signal to below 30 Hz to obtain a compressed EEG signal, including: The size of the downsampling operation is determined by the following formula: ； in, Represents the convolution kernel size used for average pooling; Represents data points within one second.

5. The lightweight motor imagery classification method guided by frequency prior features according to claim 4 is characterized in that: The equivalent receptive field of the residual network, i.e., the equivalent convolution kernel size K3 represented by the multi-layer network, is calculated as follows: 。 6. The lightweight motor imagery classification method guided by frequency prior features according to claim 5 is characterized in that: In S3, the number of residual blocks is adjusted according to the actual number and spatial distribution of electrodes.

7. The lightweight motor imagery classification method guided by frequency prior features according to claim 6 is characterized in that: The method performs sequence modeling through a temporal convolution network designed based on causal convolution, represents the previous sequence information through the last time step, obtains a sparse expression of features and a lightweight expression of the model; and completes lightweight motor imagery classification guided by frequency prior features through a classifier, including: Get time series signal , fill the time series signal to obtain a new sequence signal ; The new sequence signal is input into the convolution layer of the temporal convolutional network for convolution operation. The output of the next layer is only related to the previous data. Through multi-layer convolution operations, the last time point / step will obtain all the previous information to form a new feature expression. At this time, the entire sequence is not needed, thus obtaining a lightweight representation of the model; Motor imagery classification is performed through classifiers to obtain new representational expressions and complete lightweight motor imagery classification guided by frequency prior features.

8. A lightweight motor imagery classification device guided by a frequency prior feature, the lightweight motor imagery classification device guided by a frequency prior feature is used to implement the lightweight motor imagery classification method guided by a frequency prior feature as described in any one of claims 1 to 7, characterized in that: The device comprises: A time domain feature extraction module, used to obtain a high-dimensional EEG signal, perform preliminary processing on the high-dimensional EEG signal, and extract the time domain features of the high-dimensional EEG signal; A signal compression module is used to input the EEG signal after extracting the time domain features into a spatial convolution layer; weightedly integrate the EEG signals collected by different electrodes through the spatial convolution layer and fuse them into a single channel; downsample by average pooling to compress the high-frequency information of the weighted integrated EEG signal to below 30 Hz to obtain a compressed EEG signal; An optimization extraction module, used for inputting the compressed EEG signal into a residual network, processing the high-frequency information of the compressed EEG signal, further optimizing the extraction of signal features, and obtaining the spatiotemporal features of the EEG signal; The sequence modeling module is used to perform sequence modeling through a temporal convolutional network designed based on causal convolution. The previous sequence information is represented by the last time step to obtain a sparse expression of features and a lightweight expression of the model. The classifier is used to complete lightweight motor imagery classification guided by frequency prior features.

9. A lightweight motor imagery classification device guided by a frequency prior feature, the lightweight motor imagery classification device guided by a frequency prior feature comprising: A processor; a memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, any one of the lightweight motor imagery classification methods guided by frequency prior features as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the lightweight motor imagery classification methods guided by frequency prior features as described in any one of claims 1-7.

Images