A motor imagery electroencephalogram signal classification method and device, a terminal and a storage medium

Abstract

The application discloses a motor imagery electroencephalogram signal classification method and device, a terminal and a storage medium, and relates to the field of biomedical signal processing. The method obtains an electroencephalogram signal based on user motor imagery, pre-processes the electroencephalogram signal, and determines electroencephalogram features. The electroencephalogram features are subjected to multi-scale space-time feature extraction and spatial and channel decoupling reconstruction to determine target space-time enhanced features. The target space-time enhanced features are classified by a classification output layer to determine a classification result. The application decouples and reconstructs the features in space and channels, systematically eliminating the redundant correlation of cross-electrode electroencephalogram signals, thereby effectively solving the problem that the prior art jointly processes spatial and channel information, leading to information redundancy and increasing the computational burden.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of biomedical signal processing, and particularly relates to a motor imagery electroencephalography classification method and device, a terminal and a storage medium. BACKGROUND

[0002] Motor imagery electroencephalography classification (MI-EEG) is a core technology in the field of brain-computer interface (BCI), which aims to identify and distinguish the electroencephalography patterns corresponding to different motor imagery tasks (such as imagining left hand and right hand movement) through algorithms, so as to realize precise control of external devices. In recent years, a series of motor imagery electroencephalography classification methods based on deep learning have emerged, but they jointly process spatial and channel information without considering the high correlation between adjacent electrode signals in multi-channel electroencephalography, which produces a large amount of redundant information, increases the computational burden and has limited contribution to the discriminability of classification.

[0003] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a motor imagery electroencephalography classification method, device, terminal and storage medium, which aims to solve the problem of information redundancy and increased computational burden caused by joint processing of spatial and channel information in the prior art.

[0005] The technical solution adopted by the present application to solve the problem is as follows:

[0006] In a first aspect, the present application provides a motor imagery electroencephalography classification method, wherein the method comprises:

[0007] obtaining electroencephalography signals based on user motor imagery, pre-processing the electroencephalography signals, and determining electroencephalography features;

[0008] performing multi-scale spatio-temporal feature extraction and spatial and channel decoupling reconstruction on the electroencephalography features to determine target spatio-temporal enhanced features;

[0009] classifying the target spatio-temporal enhanced features through a classification output layer to determine a classification result.

[0010] In one embodiment, performing multi-scale spatio-temporal feature extraction and spatial and channel decoupling reconstruction on the electroencephalography features to determine target spatio-temporal enhanced features comprises:

[0011] enhance the time domain features of the electroencephalogram features through a time domain feature enhancement module to determine first spatio-temporal enhanced features;

[0012] enhance the spatial domain features of the first spatio-temporal enhanced features through a spatial domain feature enhancement module to determine second spatio-temporal enhanced features;

[0013] perform feature refinement on the second spatio-temporal enhanced features through a spatio-temporal feature refinement module to determine the target spatio-temporal enhanced features.

[0014] In an implementation method, the time domain feature enhancement module includes an efficient channel attention module, a multi-scale time domain feature extraction module, and a spatial and channel reconstruction convolution module. The time domain features of the electroencephalogram features are enhanced through the time domain feature enhancement module to determine the first spatio-temporal enhanced features, including:

[0015] The channel features of the electroencephalogram features are enhanced through the efficient channel attention module to determine initial enhanced features;

[0016] The initial enhanced features are extracted through the multi-scale time domain feature extraction module to determine first spatio-temporal features. The multi-scale time domain feature extraction module includes a plurality of parallel time domain convolution branches and a time domain feature fusion module. The widths of the convolution kernels in each time domain convolution branch are different;

[0017] The first spatio-temporal features are reconstructed through the spatial and channel reconstruction convolution module to determine first initial spatio-temporal enhanced features;

[0018] The initial enhanced features and the first initial spatio-temporal enhanced features are fused to determine the first spatio-temporal enhanced features.

[0019] In an implementation method, the spatial domain feature enhancement module includes a multi-scale spatial domain feature extraction module and the spatial and channel reconstruction convolution module. The spatial domain features of the first spatio-temporal enhanced features are enhanced through the spatial domain feature enhancement module to determine the second spatio-temporal enhanced features, including:

[0020] The first spatio-temporal enhanced features are extracted through the multi-scale spatial domain feature extraction module to determine second spatio-temporal features. The multi-scale spatial domain feature extraction module includes a plurality of parallel spatial domain convolution branches and a spatial domain feature fusion module. The heights of the convolution kernels in each spatial domain convolution branch are different;

[0021] The second spatio-temporal features are reconstructed through the spatial and channel reconstruction convolution module to determine second initial spatio-temporal enhanced features;

[0022] The first spatio-temporal enhanced features and the second initial spatio-temporal enhanced features are fused to determine the second spatio-temporal enhanced features.

[0023] In an implementation method, the target spatio-temporal enhanced feature is determined by performing feature refinement on the second spatio-temporal enhanced feature through a spatio-temporal feature refinement module, comprising:

[0024] The initial spatio-temporal refined feature is determined by performing channel and spatial decoupling reconstruction on the second spatio-temporal enhanced feature through the spatial and channel reconstruction convolution module.

[0025] The target spatio-temporal enhanced feature is determined by performing time domain compression and regularization on the initial spatio-temporal refined feature.

[0026] In an implementation method, the spatial and channel reconstruction convolution module includes a spatial reconstruction module and a channel reconstruction module, and the output feature is determined by performing channel and spatial decoupling reconstruction on the input feature through the spatial and channel reconstruction convolution module, comprising:

[0027] The input feature is one of the first spatio-temporal feature, the second spatio-temporal feature, and the second spatio-temporal enhanced feature.

[0028] The information-rich feature and the information-redundant feature are determined by segmenting the input feature based on a gating mechanism through the spatial reconstruction module.

[0029] The first information-rich feature is determined by performing grouped convolution on the information-rich feature, and the second information-rich feature is determined by performing point-by-point convolution on the first information-rich feature.

[0030] The first information-redundant feature is determined by performing point-by-point convolution on the information-redundant feature, and the second information-redundant feature is determined by performing grouped convolution on the first information-redundant feature.

[0031] The first cross-reconstruction feature is determined by multiplying the first information-rich feature and the second information-redundant feature element by element, and the second cross-reconstruction feature is determined by multiplying the second information-rich feature and the first information-redundant feature element by element.

[0032] The spatial reconstruction feature is determined by concatenating the first cross-reconstruction feature and the second cross-reconstruction feature.

[0033] The output feature is determined by processing the spatial reconstruction feature through the channel reconstruction module.

[0034] In an implementation method, the output feature is determined by processing the spatial reconstruction feature through the channel reconstruction module, comprising:

[0035] The upper spatial reconstruction feature and the lower spatial reconstruction feature are determined by segmenting the spatial dimension of the spatial reconstruction feature based on a preset ratio through the channel reconstruction module.

[0036] performing group convolution on the upper space reconstruction features to obtain first upper space reconstruction features; performing point-by-point convolution on the upper space reconstruction features to obtain second upper space reconstruction features; fusing the first upper space reconstruction features and the second upper space reconstruction features to determine upper channel reconstruction features;

[0037] performing point-by-point convolution on the lower space reconstruction features to obtain first lower space reconstruction features; and performing splicing on the lower space reconstruction features and the first lower space reconstruction features to determine lower channel reconstruction features.

[0038] performing adaptive fusion on the upper channel reconstruction features and the lower channel reconstruction features to determine the output features.

[0039] In a second aspect, an embodiment of the present application further provides a motor imagery electroencephalogram signal classification device, wherein the motor imagery electroencephalogram signal classification device comprises:

[0040] a signal preprocessing module configured to acquire an electroencephalogram signal based on user motor imagery, to preprocess the electroencephalogram signal, and to determine electroencephalogram features;

[0041] a feature enhancement module configured to perform multi-scale space-time feature extraction and spatial and channel decoupling reconstruction on the electroencephalogram features, and to determine target space-time enhanced features;

[0042] a feature classification module configured to classify the target space-time enhanced features through a classification output layer, and to determine a classification result.

[0043] In a third aspect, an embodiment of the present application further provides a terminal, which comprises a memory and one or more processors; the memory stores one or more programs; the programs contain instructions for executing the motor imagery electroencephalogram signal classification method according to any one of the above embodiments; and the processor is configured to execute the programs.

[0044] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a plurality of instructions, wherein the instructions are adapted to be loaded and executed by a processor to implement the motor imagery electroencephalogram signal classification method according to any one of the above embodiments.

[0045] The beneficial effects of the present application: the embodiment of the present application obtains the electroencephalogram signal based on the user's motor imagery, pre-processes the electroencephalogram signal, and determines the electroencephalogram features; the electroencephalogram features are subjected to multi-scale space-time feature extraction and spatial and channel decoupling reconstruction to determine the target space-time enhanced features; the target space-time enhanced features are classified by a classification output layer to determine the classification results. Since the present application decouples and reconstructs the features in space and channel, the redundant correlation of the cross-electrode electroencephalogram signal is systematically eliminated, so the problem that the existing technology jointly processes the spatial and channel information, resulting in information redundancy and increasing the computational burden, can be effectively solved. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0047] Figure 1 is a flowchart of the motor imagery electroencephalogram signal classification method provided by the embodiment of the present application.

[0048] Figure 2 is a flowchart of the specific implementation of the motor imagery electroencephalogram signal classification method provided by the embodiment of the present application.

[0049] Figure 3 is a schematic diagram of the time domain feature enhancement module of the motor imagery electroencephalogram signal classification method provided by the embodiment of the present application.

[0050] Figure 4 is a schematic diagram of the spatial domain feature enhancement module of the motor imagery electroencephalogram signal classification method provided by the embodiment of the present application.

[0051] Figure 5 is a schematic diagram of the space-time feature refining module of the motor imagery electroencephalogram signal classification method provided by the embodiment of the present application.

[0052] Figure 6 is a schematic diagram of the spatial reconstruction module of the motor imagery electroencephalogram signal classification method provided by the embodiment of the present application.

[0053] Figure 7 is a schematic diagram of the channel reconstruction module of the motor imagery electroencephalogram signal classification method provided by the embodiment of the present application.

[0054] Figure 8 is a convolution branch schematic diagram of the motor imagery electroencephalogram signal classification method provided by the embodiment of the present application.

[0055] Figure 9is a schematic diagram of an attention fusion mechanism of a motor imagery electroencephalogram signal classification method provided by an embodiment of the application.

[0056] Figure 10 is an internal module schematic diagram of a motor imagery electroencephalogram signal classification device provided by an embodiment of the application.

[0057] Figure 11 is a principle block diagram of a terminal provided by an embodiment of the application. DETAILED DESCRIPTION

[0058] A motor imagery electroencephalogram signal classification method, device, terminal and storage medium are disclosed. To make the purpose, technical solution and effects of the application more clear and explicit, the application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and not to limit the application.

[0059] Those skilled in the art can understand that the singular forms "a", "an" and "the" used herein include plural forms unless specifically stated otherwise. It should be further understood that the use of the term "include" in the specification of the application means that the stated features, integers, steps, operations, elements and / or components exist, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0060] Those skilled in the art can understand that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as that generally understood by those skilled in the art to which the application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have meanings consistent with those in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0061] Motor imagery electroencephalography (MI-EEG) classification is a core technology in the field of brain-computer interfaces (BCI). Its goal is to identify and differentiate EEG signal patterns corresponding to different motor imagery tasks (such as imagining left and right hand movements) through algorithms, thereby enabling precise control of external devices. In recent years, a series of deep learning-based MI-EEG classification methods have emerged. However, these methods jointly process spatial and channel information without considering the high correlation between adjacent electrode signals in multi-channel EEG signals. This results in a large amount of redundant information, increasing the computational burden while contributing only a limited margin to the discriminative power of the classification.

[0062] To address the aforementioned shortcomings of existing technologies, this invention provides a method for classifying motor imagery EEG signals. The method acquires EEG signals based on a user's motor imagery, preprocesses the EEG signals to determine EEG features, performs multi-scale spatiotemporal feature extraction and spatial and channel decoupling reconstruction on the EEG features to determine target spatiotemporal enhancement features, and classifies the target spatiotemporal enhancement features through a classification output layer to determine the classification result. Because this invention performs spatial and channel decoupling reconstruction of features, it systematically eliminates redundant correlations in cross-electrode EEG signals, thus effectively solving the problem of information redundancy and increased computational burden caused by the joint processing of spatial and channel information in existing technologies.

[0063] Exemplary method:

[0064] like Figure 1 As shown, the method includes:

[0065] Step S100: Acquire EEG signals based on the user's motor imagery, preprocess the EEG signals, and determine the EEG characteristics.

[0066] Motor imagery (MI) refers to the mental process by which an individual imagines or simulates a specific limb movement in their mind without actually performing the movement. This imaginative activity produces characteristic neurophysiological activity patterns in the cerebral cortex, particularly the primary motor cortex and sensorimotor cortex. The main manifestation is event-related desynchronization / synchronization, where the power of the μ rhythm (8-13 Hz) and β rhythm (13-30 Hz) in the sensorimotor area of ​​the brain significantly decreases during motor imagery, and rebounds after the imagery ends. Electroencephalography (EEG), as a non-invasive, portable, and low-cost neural signal acquisition technique, offers advantages over techniques such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) in terms of portability, real-time performance, and cost.

[0067] When the electroencephalogram signal of the user in motor imagery is acquired, the electroencephalogram signal is input into a preprocessing layer of a deep learning network based on spatial and channel reconstruction, and the electroencephalogram signal in the dimension (wherein, is the number of channels, is the number of time sampling points) is subjected to band-pass filtering (8-30 Hz) and z-score standardization to obtain the preprocessed electroencephalogram features. In the preprocessing layer, a two-dimensional convolution feature extraction layer (using two-dimensional convolution, 3x3 kernel) can be used to preliminarily extract the spatio-temporal features to obtain the electroencephalogram features.

[0068] Step S200, multi-scale spatio-temporal feature extraction and spatial and channel decoupling reconstruction are performed on the electroencephalogram features to determine the target spatio-temporal enhanced features.

[0069] In view of the problem that motor imagery cannot effectively capture the frequency-specific neural oscillation pattern at different time scales, multi-scale spatio-temporal feature extraction is performed on the electroencephalogram features to capture multi-time scale frequency-specific neural oscillation and hierarchical spatial relationship in the time dimension and the spatial dimension, respectively; in view of the problem that the adjacent electrode signals in the electroencephalogram signal recording are highly correlated, have a large amount of redundant information, and result in a large computational burden and limited contribution to the discrimination of classification, the traditional convolution is decomposed into independent spatial and channel processing paths, and the cross-electrode redundant correlation is systematically eliminated while the task-related neurophysiological pattern is retained.

[0070] In an implementation manner, the electroencephalogram features can also be subjected to wavelet transform or Fourier transform, and the frequency domain features are explicitly extracted before multi-scale spatio-temporal feature extraction and spatial and channel decoupling reconstruction, so as to more fully utilize the frequency spectrum information of the electroencephalogram signal.

[0071] In an implementation manner, the multi-scale spatio-temporal feature extraction and spatial and channel decoupling reconstruction are performed on the electroencephalogram features to determine the target spatio-temporal enhanced features, including:

[0072] Step S201, the time domain features of the electroencephalogram features are enhanced by a time domain feature enhancement module to determine first spatio-temporal enhanced features;

[0073] Step S202, the space domain features of the first spatio-temporal enhanced features are enhanced by a space domain feature enhancement module to determine second spatio-temporal enhanced features;

[0074] Step S203, the second spatio-temporal enhanced features are subjected to feature refinement by a spatio-temporal feature refinement module to determine the target spatio-temporal enhanced features.

[0075] As Figure 2As shown, the input-based deep learning network, which utilizes spatial and channel reconstruction, employs a three-stage multi-scale spatiotemporal feature extraction architecture. This architecture comprises three modules: a temporal feature enhancement module, a spatial feature enhancement module, and a spatiotemporal feature refinement module. These modules work collaboratively to capture frequency-specific neural oscillations and hierarchical spatial relationships across multiple time scales. By performing progressive feature extraction from coarse to fine across these three stages, and by employing multi-scale parallel processing to capture motor imagery patterns at different spatiotemporal scales, an optimal balance between accuracy and computational efficiency is achieved.

[0076] In one implementation, the temporal feature enhancement module includes an efficient channel attention module, a multi-scale temporal feature extraction module, and a spatial and channel reconstruction convolution module. The temporal feature enhancement module enhances the temporal features of the EEG characteristics to determine a first spatiotemporal enhancement feature, including:

[0077] Step S2011: Enhance the channel features of the EEG features through the high-efficiency channel attention module to determine the initial enhancement features;

[0078] Step S2012: Extract features from the initial enhanced features using the multi-scale temporal feature extraction module to determine the first spatiotemporal feature. The multi-scale temporal feature extraction module includes several parallel temporal convolution branches and a temporal feature fusion module. The width of the convolution kernel in each temporal convolution branch is different.

[0079] Step S2013: Perform channel and spatial decoupling reconstruction on the first spatiotemporal feature through the spatial and channel reconstruction convolution module to determine the first initial spatiotemporal enhancement feature;

[0080] Step S2014: Perform feature fusion on the initial enhancement feature and the first initial spatiotemporal enhancement feature to determine the first spatiotemporal enhancement feature.

[0081] like Figure 3 As shown, the temporal feature enhancement module includes an efficient channel attention module, a multi-scale temporal feature extraction module, and a spatial and channel reconstruction convolution module. The efficient channel attention module (ECA) performs global average pooling on the EEG features, compressing the spatiotemporal information of each channel into a scalar descriptor to obtain the global pooled features. A one-dimensional convolution with an adaptive kernel size is then used to perform local cross-channel interaction on the global pooled features, as shown below: ,in, , For hyperparameters, , , This means taking the nearest odd number. The number of channels is taken; a channel weight in the interval (0, 1) is generated through Sigmoid activation; and the initial electroencephalogram feature and the channel weight are multiplied element by element to weight the electroencephalogram feature. The channel weight is adaptively adjusted through the efficient channel attention mechanism, without dimension reduction of the full connection layer, so that channel information loss is avoided; the kernel size is adaptively adjusted: a large kernel is used to capture extensive dependence when the number of channels is large, and a small kernel is used to reduce the overhead when the number of channels is small; the motion imagination related channels (C3 / C4 motor cortex) are automatically enhanced, and irrelevant channel noise is suppressed; the computational overhead is extremely small, and the system is suitable for a brain-computer interface system with limited resources.

[0082] The multi-scale time domain feature extraction module extracts time domain features of different time scales, and obtains first space-time features. The multi-scale time domain feature extraction module includes a plurality of parallel time domain convolution branches and a time domain feature fusion module; each time domain convolution branch includes a two-dimensional convolution and a batch normalization operation; the convolution kernel width of the two-dimensional convolution in each time domain convolution branch is different.

[0083] The multi-scale time domain feature extraction module extracts time domain features of different time scales, and obtains first space-time features. The multi-scale time domain feature extraction module includes a plurality of parallel time domain convolution branches and a time domain feature fusion module; each time domain convolution branch includes a two-dimensional convolution and a batch normalization operation; the convolution kernel width of the two-dimensional convolution in each time domain convolution branch is different. Figure 8 The multi-scale time domain feature extraction module extracts time domain features of different time scales, and obtains first space-time features. The multi-scale time domain feature extraction module includes a plurality of parallel time domain convolution branches and a time domain feature fusion module; each time domain convolution branch includes a two-dimensional convolution and a batch normalization operation; the convolution kernel width of the two-dimensional convolution in each time domain convolution branch is different. Figure 9 Specifically, three parallel two-dimensional convolution branches are set as the time domain convolution branches in the embodiment, and the three parallel time domain convolution branches include: branch 1: a convolution kernel (1x64) covering a 256ms time window (250Hz sampling), corresponding to a mu rhythm (8-13Hz); branch 2: a convolution kernel (1x32) covering a 128ms time window, corresponding to a beta rhythm (13-30Hz); and branch 3: a convolution kernel (1x16) covering a 64ms time window, corresponding to a gamma rhythm (>30Hz). The multi-scale time domain feature extraction module processes the initial enhanced feature, which can expand the time domain receptive field coverage and capture frequency-specific patterns of multiple time scales; the mu / β / γ band features are indirectly extracted without explicit frequency domain transformation; compared with a single scale, the accuracy can be improved by 3-5 percentage points. The multi-scale time domain feature extraction module can be provided with more than three parallel time domain convolution branches to cover a wider range of time scales and frequencies, and is suitable for tasks containing more frequency band information.

[0084] For the first spatio-temporal feature enhanced in the time domain, the channel and spatial decoupling reconstruction is performed through the spatial and channel reconstruction convolution module to eliminate the spatial and channel information redundancy, and the first initial spatio-temporal enhanced feature is obtained. The initial enhanced feature and the first initial spatio-temporal enhanced feature are fused through the residual connection to obtain the first spatio-temporal enhanced feature. The residual connection can also be: wherein, is the feature obtained after the residual connection, is the output feature of the spatial and channel reconstruction convolution module, is the input feature of the spatial and channel reconstruction convolution module, is the weight, The preset fixed parameter can also be a learnable parameter, and the optimal residual ratio of each stage is automatically learned through the network to improve the model adaptive ability. Specifically, when the residual connection is set, the weight of the initial enhanced feature is 0.5 to retain the original time domain information. Through the residual connection, the gradient disappearance and performance degradation problem of the deep network can be alleviated; the network is allowed to learn incremental optimization rather than complete transformation; the subtle discriminant pattern in the original electroencephalogram feature is retained; the training process is stabilized, and the convergence is accelerated.

[0085] In an implementation manner, the spatial feature enhancement module includes a multi-scale spatial feature extraction module and the spatial and channel reconstruction convolution module, and the spatial feature of the first spatio-temporal enhanced feature is enhanced through the spatial feature enhancement module to determine the second spatio-temporal enhanced feature, including:

[0086] Step S2021, the first spatio-temporal enhanced feature is extracted through the multi-scale spatial feature extraction module to determine the second spatio-temporal feature, and the multi-scale spatial feature extraction module includes a plurality of parallel spatial convolution branches and a spatial feature fusion module, and the convolution kernels in each spatial convolution branch have different heights;

[0087] Step S2022, the second spatio-temporal feature is reconstructed through the spatial and channel reconstruction convolution module to determine the second initial spatio-temporal enhanced feature;

[0088] Step S2023, the first spatio-temporal enhanced feature and the second initial spatio-temporal enhanced feature are fused to determine the second spatio-temporal enhanced feature.

[0089] As Figure 4As shown, the spatial feature enhancement module includes a multi-scale spatial feature extraction module and a spatial and channel reconstruction convolution module. The multi-scale spatial feature extraction module is used to extract multi-resolution spatial (inter-channel) features, modeling hierarchical electrode interaction. The multi-scale spatial feature extraction module includes a plurality of parallel spatial convolution branches and a spatial feature fusion module. Each spatial convolution branch includes a two-dimensional convolution, a batch normalization operation, an exponential linear unit, an average pooling, and a regularization. The two-dimensional convolution in each spatial convolution branch has different kernel heights.

[0090] The multi-scale spatial feature extraction module is used to extract features from the first spatio-temporal enhanced feature to determine the second spatio-temporal feature. Specifically, each spatial convolution branch processes the first spatio-temporal enhanced feature to determine a corresponding spatial branch feature. The spatial feature fusion module adaptively weights and fuses each spatial branch feature to obtain the second spatio-temporal feature. The spatial feature fusion module is based on an attention fusion mechanism. In this embodiment, the multi-scale spatial feature extraction module includes three parallel spatial convolution branches, which are branch 1: kernel height or channel number C, modeling global brain connectivity patterns; branch 2: kernel height or channel number / 2 (C / 2), focusing on inter-hemisphere interaction (left and right motor cortex); and branch 3: kernel height or channel number / 4 (C / 4), extracting sensory motor area local electrode cluster features. By setting three parallel spatial convolution branches with different kernel heights, global, hemispheric, and local spatial patterns can be learned simultaneously. This is consistent with the contralateral control mechanism of motor imagery and enhances the sensitivity to key electrodes (such as C3 / C4).

[0091] After the spatial feature enhancement of the first spatio-temporal enhanced feature, the spatial and channel reconstruction convolution module is used to decouple and reconstruct the channels and spaces of the second spatio-temporal feature to suppress redundant spatial responses and obtain a second initial spatio-temporal enhanced feature. The residual connection is used to fuse the first spatio-temporal enhanced feature and the second initial spatio-temporal enhanced feature to obtain the second spatio-temporal enhanced feature, preventing excessive information loss during spatial compression. Specifically, the weight of the first spatio-temporal enhanced feature can be set to 0.1 during the residual connection. Through the residual connection, the gradient vanishing and performance degradation problems of the deep network can be alleviated, allowing the network to learn incremental optimization rather than complete transformation.

[0092] In an implementation, the temporal feature fusion module and the spatial feature fusion module can use multiple independent attention fusion branches, each branch learning different feature importance patterns to improve the diversity and robustness of feature fusion through integration.

[0093] In an implementation, the temporal feature fusion module and the spatial feature fusion module can use multiple independent attention fusion branches, each branch learning different feature importance patterns to improve the diversity and robustness of feature fusion through integration.

[0094] Step S2031: Perform channel and spatial decoupling reconstruction on the second spatiotemporal enhanced feature through the spatial and channel reconstruction convolution module to determine the initial spatiotemporal refined feature;

[0095] Step S2032: Perform temporal compression and regularization on the initial spatiotemporal refined features to determine the target spatiotemporal enhancement features.

[0096] like Figure 5 As shown, the spatiotemporal feature refinement module is used to integrate and refine the spatiotemporal features from the first two stages. The spatiotemporal feature refinement module includes a single standard 2D convolution (1×16), a spatial and channel reconstruction convolution module, batch normalization, exponential linear units, average pooling, and regularization. The spatial dimension of the second spatiotemporal enhancement feature has already been compressed to 1, resulting in depthwise convolution redundancy. Therefore, standard convolution is first used to process the second spatiotemporal enhancement feature to obtain the processed second spatiotemporal enhancement feature. Simultaneously, based on this, the spatial and channel convolution module is used to extract fine-grained temporal variations and eliminate residual redundancy, obtaining the initial spatiotemporal refined feature. Temporal compression and regularization are performed through average pooling and regularization to obtain the target spatiotemporal enhancement feature. Processing the second spatiotemporal enhancement feature through the spatiotemporal feature refinement module to obtain the target spatiotemporal enhancement feature maximizes the discriminative power of the refined feature; stabilizes the classifier input, reduces the computational burden of the fully connected layer, and improves the model's generalization ability.

[0097] In one implementation, the temporal feature enhancement module, the spatial feature enhancement module, and the spatiotemporal refinement module may include multiple spatial and channel reconstruction convolutional modules. These multiple spatial and channel reconstruction convolutional modules are connected in series to form a recursive feature refinement, which gradually improves the feature discrimination capability.

[0098] In one implementation, the electrode space topology can be modeled as a graph structure. Graph convolution can be added to the spatial and channel reconstruction convolution modules in the temporal feature enhancement module, spatial feature enhancement module, and spatiotemporal refinement module to explicitly utilize the spatial adjacency relationships between electrodes.

[0099] In one implementation, the spatial and channel reconstruction convolutional module includes a spatial reconstruction module and a channel reconstruction module. The spatial and channel reconstruction convolutional module performs channel and spatial decoupling reconstruction on the input features to determine the output features, including:

[0100] The input feature is one of the first spatiotemporal feature, the second spatiotemporal feature, and the second spatiotemporal enhancement feature;

[0101] The spatial reconstruction module segments the input features based on a gating mechanism to determine information-rich features and information-redundant features;

[0102] The information-rich features are grouped and convolved to determine the first information-rich features; the first information-rich features are then convolved point-by-point to determine the second information-rich features.

[0103] Perform pointwise convolution on the information redundancy features to determine the first information redundancy feature; perform group convolution on the first information redundancy feature to determine the second information redundancy feature;

[0104] The first information-rich feature and the second information-redundant feature are multiplied element by element to determine the first cross-reconstruction feature; the second information-rich feature and the first information-redundant feature are multiplied element by element to determine the second cross-reconstruction feature.

[0105] By splicing the first cross-reconstruction feature and the second cross-reconstruction feature, spatial reconstruction features are determined;

[0106] The spatial reconstruction features are processed by the channel reconstruction module to determine the output features.

[0107] The spatial and channel reconstruction convolutional module includes a spatial reconstruction module and a channel reconstruction module. The spatial reconstruction module eliminates spatial redundancy, and the channel reconstruction module eliminates reconstruction redundancy. The spatial reconstruction module and the channel reconstruction module can be connected in series or in parallel. This embodiment uses a series configuration.

[0108] like Figure 6 As shown, the spatial reconstruction module mainly consists of two parts: a gating mechanism and cross-reconstruction. For the input features, a learnable scaling factor is first generated based on the input features through group normalization. Then, based on a gating mechanism, the scaling factor is compared with a preset threshold. The input feature channels are segmented according to the comparison result to obtain information-rich features and information-redundant features. For example, using the scaling factor and a preset threshold... Comparison yields information-rich features and information redundancy features For the gating mechanism in the spatial reconstruction module, activation functions can be used instead of threshold-based hard gating, learnable temperature parameters can be used to adjust the gating distribution, or channel attention (such as the Squeeze-and-Excitation Module) can be used to generate weights to replace the scaling factor.

[0109] Dual-path cross-reconstruction is performed on information-rich features and information-redundant features: the information-rich features are grouped and convolved to obtain the first information-rich feature. ; Perform pointwise convolution on the first information-rich feature to obtain the second information-rich feature. Perform pointwise convolution on the information redundancy features to obtain the first information redundancy feature. grouping convolution is performed on the first information redundancy feature to obtain a second information redundancy feature element-wise multiplication is performed on the first information richness feature and the second information redundancy feature to obtain a first cross-reconstruction feature; element-wise multiplication is performed on the second information richness feature and the first information redundancy feature to obtain a second cross-reconstruction feature; and the first cross-reconstruction feature and the second cross-reconstruction feature are spliced to obtain a spatial reconstruction feature. The expression of the double-path cross-reconstruction is as follows:

[0110] the first cross-reconstruction feature: ;

[0111] the second cross-reconstruction feature: ;

[0112] wherein, is a grouping convolution, is a 1x1 point-wise convolution, , and are superscripts, used to distinguish the first cross-reconstruction feature and the second cross-reconstruction feature.

[0113] The input feature is processed by the spatial reconstruction module, and about 50% sparsity is generated through the gating mechanism, which significantly reduces the calculation amount; effectively suppresses the interference of irrelevant brain areas (such as electrooculogram artifacts and electromyographic noise); and retains the key information of the motor imagination related brain area (such as C3 / C4 motor cortex).

[0114] After spatial reconstruction, the spatial reconstruction feature is processed by the channel reconstruction module to obtain an output feature.

[0115] The input feature of the spatial and channel reconstruction convolution module can be one of the first spatio-temporal feature, the second spatio-temporal feature and the second spatio-temporal enhanced feature. Corresponding to the input feature, the input feature of the spatial and channel reconstruction convolution module is one of the first initial spatio-temporal enhanced feature, the second initial spatio-temporal enhanced feature and the initial spatio-temporal refined feature. That is, when the input feature is the first spatio-temporal feature, the corresponding output feature is the first initial spatio-temporal enhanced feature; when the input feature is the second spatio-temporal feature, the corresponding output feature is the second initial spatio-temporal enhanced feature; and when the input feature is the second spatio-temporal enhanced feature, the corresponding output feature is the initial spatio-temporal refined feature.

[0116] Compared with the standard convolution, the spatial and channel reconstruction convolution module can reduce about 50% of the parameter amount under the same output channel number; by cascading the spatial reconstruction module and the channel reconstruction module, the redundancy of the spatial and channel dimensions is eliminated at the same time, the task-related neurophysiological pattern is retained, and the discrimination ability is improved.

[0117] ​In an implementation manner, the spatial reconstruction feature is processed by the channel reconstruction module, and the output feature is determined, including:

[0118] The spatial reconstruction feature is divided in the spatial dimension based on a preset ratio by the channel reconstruction module, to determine an upper spatial reconstruction feature and a lower spatial reconstruction feature;

[0119] The upper spatial reconstruction feature is grouped convolution to obtain a first upper spatial reconstruction feature; the upper spatial reconstruction feature is point-by-point convolution to obtain a second upper spatial reconstruction feature; the first upper spatial reconstruction feature and the second upper spatial reconstruction feature are fused to determine an upper channel reconstruction feature;

[0120] The lower spatial reconstruction feature is point-by-point convolution to obtain a first lower spatial reconstruction feature; the lower spatial reconstruction feature and the first lower spatial reconstruction feature are spliced to determine a lower channel reconstruction feature;

[0121] The upper channel reconstruction feature and the lower channel reconstruction feature are adaptively fused to determine the output feature.

[0122] As shown in the formula (1), the channel reconstruction module divides the feature in the channel dimension into two parts, upper and lower, to obtain an upper spatial reconstruction feature and a lower spatial reconstruction feature. Figure 7 The preset ratio is set as 1 / 2 in this embodiment. The upper spatial reconstruction feature and the lower spatial reconstruction feature are processed by a double-branch processing. In the upper branch, the upper spatial reconstruction feature is compressed by grouped convolution to , representing the channel, and then restored to by point-by-point convolution, to capture high-level semantic information; the lower branch is only processed by point-by-point convolution, to retain fine-grained local differences. Specifically, the upper spatial reconstruction feature is grouped convolution to obtain a first upper spatial reconstruction feature; the upper spatial reconstruction feature is point-by-point convolution to obtain a second upper spatial reconstruction feature; the first upper spatial reconstruction feature and the second upper spatial reconstruction feature are fused to determine an upper channel reconstruction feature; the lower spatial reconstruction feature is point-by-point convolution to obtain a first lower spatial reconstruction feature; the lower spatial reconstruction feature and the first lower spatial reconstruction feature are spliced to determine a lower channel reconstruction feature. The upper channel reconstruction feature and the lower channel reconstruction feature are adaptively fused to obtain the output feature, represented as:

[0123] ,

[0124] ,

[0125] wherein, are weights, representing global average pooling,​ denotes a full connection layer, is the upper channel reconstruction feature, is the lower channel reconstruction feature, is the output feature.

[0126] The channel reconstruction module can greatly reduce channel redundancy through compression-expansion design, determine the relative importance of semantic features and local details through adaptive weights, and accurately extract key features because the discriminative information in the electroencephalogram signal is sparse and localized.

[0127] The channel reconstruction module can also be modified as follows, such as using standard convolution instead of the above separable convolution to enhance cross-channel modeling, using factorized convolution to decompose into multiple small convolution kernels, or introducing residual connections.

[0128] Step S300, classifying the target spatio-temporal enhanced feature through the classification output layer to determine the classification result.

[0129] The input deep learning network based on spatial and channel reconstruction further includes a classification output layer, which includes a global average pooling, a regularization (Dropout), a full connection layer, and a Softmax (activation function). The target spatio-temporal enhanced feature is classified through the classification output layer, and the classification result is obtained, so that the specific action corresponding to the user during motor imagery is determined according to the classification result. For the full connection layer in the classification output layer, a capsule network can be used instead to aggregate features of different channels and time periods through dynamic routing and retain more hierarchical structure information.

[0130] For the input deep learning network based on spatial and channel reconstruction disclosed above, each network layer or each module in the input deep learning network based on spatial and channel reconstruction can be adjusted according to different needs. Adjustment scheme 1: only use the time domain feature enhancement module, without the spatial domain feature enhancement module and the spatio-temporal feature refining module; adjustment scheme 2: use the efficient channel attention module and the spatial and channel reconstruction convolution module for feature extraction, without multi-scale and multi-stage design; adjustment scheme 3: use the time domain feature enhancement module, the spatial domain feature enhancement module, and the attention fusion mechanism for feature extraction, without the spatio-temporal feature refining module; adjustment scheme 4: the spatial and channel reconstruction convolution module separately uses a spatial reconstruction module or a channel reconstruction module; adjustment scheme 5: replace the spatial and channel reconstruction convolution module with the self-attention mechanism of the Transformer to calculate the global dependency relationship through query-key-value.

[0131] Specific implementation details:

[0132] The following two datasets are used to train and test the deep learning network based on spatial and channel reconstruction in the present application:

[0133] BCI Competition IV-2a (an electroencephalogram dataset): , (4 seconds @ 250 Hz);

[0134] BCI Competition IV-2b (an electroencephalogram dataset): , (4 seconds @ 250 Hz).

[0135] The electroencephalogram data in the two datasets were preprocessed, for example, channel-level zero-mean standardization (z-score normalization) was performed:

[0136] ,

[0137] wherein is the feature after zero-mean standardization, is the feature before zero-mean standardization, is the mean, is the standard deviation.

[0138] Table 1. Configuration table of key parameters of network architecture

[0139]

[0140] Training process (data augmentation): each trial was divided into 8 time segments; random extraction and recombination from the same sample; 2 augmented samples were generated from each original sample; time structure and class-specific features were preserved.

[0141] Network training strategy: the optimizer was configured as follows: Adam (an optimization algorithm) was used as the optimizer, the initial learning rate was 0.01, the weight decay was 0.001 (L2 regularization), the batch size was 64; the number of training rounds was 500 epochs (time segments); the loss function was cross-entropy loss; the regularization technique was regularization before the fully connected layer with a parameter of 0.5, and batch normalization was performed after each convolutional layer, with L2 weight decay: to prevent overfitting.

[0142] Training protocol: BCI Competition IV-2a: training set (T) training, test set (E) evaluation; BCI Competition IV-2b: the first 2 sessions training, the last 3 sessions evaluation; cross-session and subject training and testing to ensure time independence.

[0143] The parameter configuration of the deep learning network based on spatial and channel reconstruction is shown in Table 1, wherein, denotes the number of feature maps, C is the number of channels (C = 22 in the BCI Competition IV-2a dataset, C = 3 in the BCI Competition IV-2a dataset), , , , denotes the number of time points, respectively, N_classes is the number of classes (4 in the BCI Competition IV-2a dataset, 2 in the BCI Competition IV-2a dataset).

[0144] The parameter size of the deep learning network based on spatial and channel reconstruction and the parameter size of each comparative method are shown in Table 2. The comparative methods include: EEGNet method (EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces, a compact convolutional network for EEG-based brain-computer interfaces), EEGNeX method (EEGNeX: A Robust Convolutional Neural Network for EEG-Based Brain-Computer Interfaces, a robust convolutional neural network for EEG-based brain-computer interfaces), ATCNet method (ATCNet: Attention Temporal Convolutional Network for EEG-Based Motor Imagery Classification, attention temporal convolutional network for EEG-based motor imagery classification), and Conformer method (Conformer: Convolution-augmented Transformer for EEG-based Motor Imagery Classification, convolution-augmented Transformer for EEG-based motor imagery classification).

[0145] Table 2. Model size comparison table

[0146]

[0147] The performance of the deep learning network based on spatial and channel reconstruction is compared with the performance of other methods as shown in Table 3, which also includes the performance comparison with the EEGTCNet (EEG-TCNet: An Accurate Temporal Convolutional Network for Embedded Motor-Imagery Brain–Machine Interfaces, an accurate temporal convolutional network for embedded motor-imagery brain-machine interfaces) method:

[0148] Table 3 Performance comparison table

[0149]

[0150] Among them, IV-2a is the BCI Competition IV-2a dataset, and IV-2b is the BCI Competition IV-2b dataset. According to Table 3, the deep learning network based on spatial and channel reconstruction proposed in this embodiment achieves an average accuracy of 80.17% on the BCI Competition IV-2a dataset (9 subjects), which is 4.98 percentage points higher than EEGNet and 1.50 percentage points higher than ATCNet. The Kappa coefficient (Cohen's coefficient) is 0.74, indicating excellent classification consistency, and the accuracy of five subjects (S1, S3, S7, S8, S9) is over 85%, with the highest being 95.83% (S3). On the BCI Competition IV-2b dataset (9 subjects), the average accuracy is 87.98%, which is 2.58 percentage points higher than EEGNet and 2.11 percentage points higher than ATCNet. The standard deviation is 8.61%, which is the lowest among all methods and has the strongest cross-subject consistency. The accuracy of four subjects (S4, S5, S7, S8) is over 93%, with the highest being 96.88% (S4).

[0151] The confusion matrix analysis results on the BCI Competition IV-2a dataset are as follows: the accuracy of the left hand is 82%, the accuracy of the right hand is 77%, the accuracy of the feet is 79%, and the accuracy of the tongue is 83%; the highest confusion cases include: considering the feet as the tongue (10%), considering the left hand as the right hand (8%); the tongue motor imagery accuracy is the highest (83%).

[0152] In terms of cross-subject robustness: On the BCI Competition IV-2a dataset, the standard deviation of the deep learning network based on spatial and channel reconstruction is 12.36%, which is equivalent to EEGNet (12.19%) and superior to EEGTCNet (13.67%); on the BCI Competition IV-2b dataset, the standard deviation of the deep learning network based on spatial and channel reconstruction is 8.61%, which is the lowest among all comparison methods, significantly better than Conformer (13.98%) and EEGTCNet (10.07%).

[0153] In terms of technical implementation: The three-stage multi-scale processing improves the adaptability to different individual EEG signals, the adaptive fusion mechanism of the spatial and channel reconstruction convolution module enhances the feature generalization ability, and the attention fusion mechanism dynamically adjusts the contribution degree of different branches to adapt to individual differences.

[0154] Cross-dataset verification: The deep learning network based on spatial and channel reconstruction maintains excellent performance under different electrode configurations (22 channels and 3 channels) and achieves optimal results under different classification tasks (4 classification and 2 classification), proving the robustness and generalization ability of the method to different experimental paradigms.

[0155] Computational efficiency: The deep learning network based on spatial and channel reconstruction has 24,804 parameters (2.48 million), a model size of 0.12 MB, a reduction of 96.9% in parameter quantity compared to Conformer (Conformer has 790,000 parameters), and an accuracy increase of 10.19 percentage points; compared to ATCNet, the parameter quantity is reduced by 78.9% (ATCNet has 117,000 parameters), and the accuracy is increased by 1.50 percentage points; compared to EEGNeX, the parameter quantity is reduced by 70.8% (EEGNeX has 85,000 parameters), and the accuracy is increased by 4.67 percentage points.

[0156] In terms of lightweight design: The spatial and channel reconstruction convolution module reduces about 50% of the convolution parameters through spatial-channel decoupling; the three-stage multi-scale convolution architecture improves parameter utilization efficiency through parameter sharing; the efficient channel attention module does not require a fully connected layer, with minimal parameter overhead. In terms of real-time deployment capability, the deep learning network based on spatial and channel reconstruction has a total of 24,804 parameters and a model size of only 0.12 MB, and the lightweight architecture can be deployed on embedded devices with low computational complexity, and has the potential to meet the response speed requirements of real-time brain-computer interface systems; it can provide technical support for the practical deployment of portable and low-cost brain-computer interface products.

[0157] End-to-end learning ability: the deep learning network based on spatial and channel reconstruction jointly optimizes from the original electroencephalogram signal to the classification output through back propagation, avoids the information loss of the traditional multi-stage method (preprocessing, feature extraction, classification), does not need artificial design of features, simplifies the system development process, realizes global optimization instead of local optimization.

[0158] The ablation experiment verification is performed on the BCI Competition IV-2a data set, and the verification result is shown in Table 4. According to the content of Table 4, the accuracy is improved by 4.51% by using the complete configuration compared with the baseline. The forward contribution of each module: the accuracy is improved by 0.89% by adding the efficient channel attention module; the accuracy is improved by 0.19% by adding the spatial and channel reconstruction convolution module; the accuracy is improved by 1.16% by adding the attention fusion mechanism; the accuracy is improved by 2.12% by using the multi-branch. When each module is coordinated (that is, the complete model, the efficient channel attention module, the spatial and channel reconstruction convolution module, the attention fusion mechanism and the multi-branch are used at the same time, the multi-branch is a plurality of time domain convolution branches or a plurality of space domain convolution branches), the accuracy is improved by 4.51%, which is greater than the sum of the independent contributions of each module (4.36%), proving that there is a positive synergistic effect between the modules.

[0159] Table 4 Ablation experiment verification table

[0160]

[0161] Based on the above embodiment, the application further provides a motor imagery electroencephalogram signal classification device, as shown in the accompanying drawings, the device comprises: Figure 10

[0162] A signal preprocessing module 01 is configured to acquire an electroencephalogram signal based on user motor imagery, preprocess the electroencephalogram signal, and determine electroencephalogram features.

[0163] A feature enhancement module 02 is configured to perform multi-scale spatio-temporal feature extraction and spatial and channel decoupling reconstruction on the electroencephalogram features, and determine target spatio-temporal enhanced features.

[0164] A feature classification module 03 is configured to classify the target spatio-temporal enhanced features through a classification output layer, and determine a classification result.

[0165] Based on the above embodiment, the application further provides a terminal, and the principle block diagram can be as shown in the accompanying drawings. Figure 11 ​The terminal includes a processor, a memory, a network interface, and a display screen connected through a system bus. The processor of the terminal is configured to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement the motor imagery electroencephalogram signal classification method. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0166] Those skilled in the art can understand that Figure 11 The principle block diagram shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal to which the scheme of the present application is applied. The specific terminal can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0167] In an implementation manner, the memory of the terminal stores more than one program, and is configured to execute the more than one program by more than one processor to include instructions for performing the motor imagery electroencephalogram signal classification method.

[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments of the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0169] To sum up, the application discloses a motor imagery electroencephalogram signal classification method and device, a terminal and a storage medium, the method obtains an electroencephalogram signal based on user motor imagery, pre-processes the electroencephalogram signal, and determines electroencephalogram features; multi-scale space-time feature extraction and spatial and channel decoupling reconstruction are performed on the electroencephalogram features to determine target space-time enhanced features; the target space-time enhanced features are classified by a classification output layer to determine a classification result. Since the application performs spatial and channel decoupling reconstruction on the features, the redundancy correlation of cross-electrode electroencephalogram signals is systematically eliminated, and therefore the problem that the prior art jointly processes spatial and channel information, resulting in information redundancy and increasing the calculation burden, can be effectively solved.

[0170] It should be understood that the application is not limited to the above examples, and can be improved or changed according to the above description for those of ordinary skill in the art, and all these improvements and changes shall belong to the protection scope of the appended claims of the application.

Claims

1. A motor imagery electroencephalogram signal classification method, characterized in that, The method comprises: acquiring electroencephalogram signals based on user motor imagery, preprocessing the electroencephalogram signals, and determining electroencephalogram features; performing multi-scale spatio-temporal feature extraction and spatial and channel decoupling reconstruction on the electroencephalogram features to determine target spatio-temporal enhanced features; classifying the target spatio-temporal enhanced features through a classification output layer to determine a classification result; performing multi-scale spatio-temporal feature extraction and spatial and channel decoupling reconstruction on the electroencephalogram features to determine target spatio-temporal enhanced features, comprising: enhancing the time domain features of the electroencephalogram features through a time domain feature enhancement module to determine first spatio-temporal enhanced features; enhancing the spatial domain features of the first spatio-temporal enhanced features through a spatial domain feature enhancement module to determine second spatio-temporal enhanced features; performing feature refinement on the second spatio-temporal enhanced features through a spatio-temporal feature refinement module to determine the target spatio-temporal enhanced features; the spatial domain feature enhancement module comprises a multi-scale spatial domain feature extraction module and the spatial and channel reconstruction convolution module, and enhancing the spatial domain features of the first spatio-temporal enhanced features through the spatial domain feature enhancement module to determine second spatio-temporal enhanced features, comprising: performing feature extraction on the first spatio-temporal enhanced features through the multi-scale spatial domain feature extraction module to determine second spatio-temporal features, the multi-scale spatial domain feature extraction module comprising a plurality of parallel spatial domain convolution branches and a spatial domain feature fusion module, the convolution kernels in each of the spatial domain convolution branches having different heights; performing channel and spatial decoupling reconstruction on the second spatio-temporal features through the spatial and channel reconstruction convolution module to determine second initial spatio-temporal enhanced features; fusing the first spatio-temporal enhanced features and the second initial spatio-temporal enhanced features to determine the second spatio-temporal enhanced features.

2. The motor imagery electroencephalogram signal classification method according to claim 1, characterized in that, the time domain feature enhancement module comprises an efficient channel attention module, a multi-scale time domain feature extraction module, and a spatial and channel reconstruction convolution module, and enhancing the time domain features of the electroencephalogram features through the time domain feature enhancement module to determine first spatio-temporal enhanced features, comprising: strengthening the channel features of the electroencephalogram features through the efficient channel attention module to determine initial enhanced features; performing feature extraction on the initial enhanced features through the multi-scale time domain feature extraction module to determine first spatio-temporal features, the multi-scale time domain feature extraction module comprising a plurality of parallel time domain convolution branches and a time domain feature fusion module, the convolution kernels in each of the time domain convolution branches having different widths; performing channel and spatial decoupling reconstruction on the first spatio-temporal features through the spatial and channel reconstruction convolution module to determine first initial spatio-temporal enhanced features; performing feature fusion on the initial enhanced features and the first initial spatio-temporal enhanced features to determine the first spatio-temporal enhanced features.

3. The motor imagery electroencephalogram signal classification method according to claim 1, characterized in that, performing feature refinement on the second spatio-temporal enhanced features through a spatio-temporal feature refinement module to determine the target spatio-temporal enhanced features, comprising: performing channel and spatial decoupling reconstruction on the second spatio-temporal enhanced features through the spatial and channel reconstruction convolution module to determine initial spatio-temporal refined features; performing time domain compression and regularization on the initial spatio-temporal refined features to determine the target spatio-temporal enhanced features.

4. The motor imagery electroencephalogram signal classification method according to claim 3, characterized in that, The space and channel reconstruction convolution module comprises a space reconstruction module and a channel reconstruction module, the input feature is reconstructed by the space and channel reconstruction convolution module, and the output feature is determined, comprising: The input feature is one of the first spatiotemporal feature, the second spatiotemporal feature and the second spatiotemporal enhancement feature; The input feature is segmented based on a gating mechanism by the space reconstruction module to determine an information-rich feature and an information-redundant feature; The information-rich feature is grouped and convolved to determine a first information-rich feature, and the first information-rich feature is point-by-point convolved to determine a second information-rich feature; The information-redundant feature is point-by-point convolved to determine a first information-redundant feature, and the first information-redundant feature is grouped and convolved to determine a second information-redundant feature; The first information-rich feature and the second information-redundant feature are multiplied element by element to determine a first cross-reconstruction feature, and the second information-rich feature and the first information-redundant feature are multiplied element by element to determine a second cross-reconstruction feature; The first cross-reconstruction feature and the second cross-reconstruction feature are spliced to determine a space reconstruction feature; The space reconstruction feature is processed by the channel reconstruction module to determine the output feature.

5. The motor imagery electroencephalogram signal classification method according to claim 4, characterized in that, The space reconstruction feature is processed by the channel reconstruction module to determine the output feature, comprising: The space reconstruction feature is segmented in the space dimension based on a preset ratio by the channel reconstruction module to determine an upper space reconstruction feature and a lower space reconstruction feature; The upper space reconstruction feature is grouped and convolved to obtain a first upper space reconstruction feature, and the upper space reconstruction feature is point-by-point convolved to obtain a second upper space reconstruction feature; the first upper space reconstruction feature and the second upper space reconstruction feature are fused to determine an upper channel reconstruction feature; The lower space reconstruction feature is point-by-point convolved to obtain a first lower space reconstruction feature, and the lower space reconstruction feature and the first lower space reconstruction feature are spliced to determine a lower channel reconstruction feature; The upper channel reconstruction feature and the lower channel reconstruction feature are adaptively fused to determine the output feature.

6. A motor imagery electroencephalogram signal classification apparatus characterized by comprising: The device comprises: A signal preprocessing module is configured to acquire electroencephalogram signals based on user motor imagery, preprocess the electroencephalogram signals, and determine electroencephalogram features; A feature enhancement module is configured to perform multi-scale spatiotemporal feature extraction and spatial and channel decoupling reconstruction on the electroencephalogram features, and determine target spatiotemporal enhancement features; A feature classification module is configured to classify the target spatiotemporal enhancement features through a classification output layer, and determine a classification result; The electroencephalogram features are subjected to multi-scale spatiotemporal feature extraction and spatial and channel decoupling reconstruction to determine target spatiotemporal enhancement features, comprising: A time domain feature enhancement module is configured to enhance the time domain features of the electroencephalogram features to determine first spatiotemporal enhancement features; A space domain feature enhancement module is configured to enhance the space domain features of the first spatiotemporal enhancement features to determine second spatiotemporal enhancement features; A spatiotemporal feature refining module is configured to refine the features of the second spatiotemporal enhancement features to determine the target spatiotemporal enhancement features; and A feature classification module is configured to classify the target spatiotemporal enhancement features through a classification output layer, and determine a classification result. The spatial feature enhancement module comprises a multi-scale spatial feature extraction module and the spatial and channel reconstruction convolution module, and the spatial feature of the first spatio-temporal enhancement feature is enhanced through the spatial feature enhancement module to determine a second spatio-temporal enhancement feature, comprising: The multi-scale spatial feature extraction module is used to perform feature extraction on the first spatio-temporal enhancement feature to determine a second spatio-temporal feature, and the multi-scale spatial feature extraction module comprises a plurality of parallel spatial convolution branches and a spatial feature fusion module, and the convolution kernels in each spatial convolution branch have different heights; The spatial and channel reconstruction convolution module is used to perform channel and spatial decoupling reconstruction on the second spatio-temporal feature to determine a second initial spatio-temporal enhancement feature; The first spatio-temporal enhancement feature and the second initial spatio-temporal enhancement feature are fused to determine the second spatio-temporal enhancement feature.

7. A terminal, characterized by comprising: The terminal comprises a memory and one or more processors; the memory stores one or more programs; the programs contain instructions for executing the motor imagery electroencephalogram signal classification method according to any one of claims 1-5; and the processor is used to execute the programs.

8. A computer-readable storage medium storing a plurality of instructions thereon, characterized in that, The instructions are loaded and executed by the processor to implement the steps of the motor imagery electroencephalogram signal classification method according to any one of claims 1-5.

Images