Motor imagery electroencephalogram signal classification method and system

Abstract

The invention discloses a motor imagery electroencephalogram signal classification method and system, and belongs to the technical field of computers. The method comprises the following steps: acquiring a data set, and preprocessing the data set to obtain a target data set; constructing a channel and time attention mechanism convolutional network model and a loss function, and training and testing the channel and time attention mechanism convolutional network model according to the loss function and the target data set; and inputting to-be-classified motor imagery electroencephalogram signals into the tested channel and time attention mechanism convolutional network model, and outputting a motor imagery electroencephalogram signal data classification result. By intelligently integrating the channel attention convolution module, the depth feature extraction module and the time attention module, the space-time frequency features are effectively extracted and fused, and the brain-computer interface task classification performance based on motor imagery is remarkably improved.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a method and system for classifying motor imagery EEG signals. Background Art

[0002] Brain-Computer Interface (BCI) has shown great potential in the rehabilitation of patients with stroke, spinal cord injury or ALS, helping patients regain certain motor skills or the ability to interact with the outside world. With the continuous advancement of neuroscience, artificial intelligence and hardware technology, BCI is developing in a more efficient, accurate and popular direction; traditional machine learning methods process electroencephalography (EEG) signals, usually focusing on extracting neurophysiological features. This process usually follows a multi-step framework, and traditional feature extraction methods are task-specific, that is, they require specific prior knowledge for different BCI paradigms and have limited generalization capabilities.

[0003] In recent years, deep learning methods, especially convolutional neural networks (CNNs), have demonstrated excellent performance in many fields including image recognition, natural language processing, and EEG signal classification. However, existing deep learning models usually lack sufficient modeling of multi-level dependencies between signal channels and in time, and tend to ignore the complex spatiotemporal structure implicit in EEG signals, resulting in inaccurate feature extraction of motor imagery EEG signals, which in turn leads to low accuracy and low efficiency in the classification of motor imagery EEG signals. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method and system for classifying motor imagery EEG signals, which can solve the technical problem that the deep learning methods in the prior art tend to ignore the complex spatiotemporal structure implicit in the EEG signals.

[0005] In order to solve the above technical problems, this application is implemented as follows: In a first aspect, an embodiment of the present application provides a method for classifying motor imagery EEG signals, the method comprising: Acquire a data set, and preprocess the data set to obtain a target data set; Constructing a channel and time attention mechanism convolutional network model, wherein the channel and time attention mechanism convolutional network model includes a channel attention convolution module, a deep feature extraction module, a time attention module and a classification module; Constructing a loss function, and training and testing the channel and time attention mechanism convolutional network model according to the loss function and the target data set; The motor imagery EEG signal to be classified is input into the tested channel and time attention mechanism convolutional network model, and the classification result of the motor imagery EEG signal data is output.

[0006] As an optional implementation of the first aspect of the present application, the motor imagery EEG signal to be classified is input into the channel and time attention mechanism convolutional network model after the test, and the motor imagery EEG signal data classification result is output; specifically: Inputting the motor imagery EEG signal to be classified into the channel attention convolution module of the channel and time attention mechanism convolution network model after the test, the channel attention convolution module performs convolution processing on the motor imagery EEG signal to be classified, and outputs a channel weighted convolution feature map group; Inputting the channel weighted convolution feature map group into the deep feature extraction module, the deep feature extraction module performs deep feature extraction on the channel weighted convolution feature map group to obtain a deep convolution feature map group; Inputting the deep convolutional feature map group into the temporal attention module, and the temporal attention module performs temporal attention weighted processing on the deep convolutional feature map group to obtain a temporal weighted convolutional feature map group; The time-weighted convolution feature map group is input into the classification module, and the classification module performs classification processing on the time-weighted feature map group to obtain the category corresponding to the motor imagery EEG signal to be classified, thereby completing the classification of the motor imagery EEG signal to be classified.

[0007] As an optional implementation of the first aspect of the present application, the channel attention convolution module performs convolution processing on the motor imagery EEG signal to be classified, and outputs a channel weighted convolution feature map group; specifically: The motor imagery EEG signal to be classified is input into the channel attention convolution module, the first time convolution layer in the channel attention convolution module performs time convolution processing on the motor imagery EEG signal to be classified to obtain an initial feature map group, and the initial feature map group is standardized according to the first batch of normalization layers; Performing spatial convolution processing on the initial feature map group after the standardization processing according to the spatial convolution layer to obtain a spatial feature map group; and then performing standardization processing and nonlinear activation processing on the spatial feature map group in sequence according to the second batch normalization layer and the first nonlinear activation layer; Performing average pooling processing on the spatial feature map group after the nonlinear activation processing according to the average pooling layer, and then performing random deactivation processing on the spatial feature map group after the average pooling processing according to the random deactivation layer; Performing channel attention weighted processing on the spatial feature map group after random dropout processing according to the channel attention layer to obtain a channel weighted feature map group; The channel weighted feature map group is subjected to temporal convolution processing according to the second temporal convolution layer, and then the spatial feature map group is subjected to standardization processing and nonlinear activation processing in sequence according to the third batch normalization layer and the second nonlinear activation layer to obtain the channel weighted convolution feature map group.

[0008] As an optional implementation of the first aspect of the present application, the channel weighted convolution feature map group is input into the deep feature extraction module, and the deep feature extraction module performs deep feature extraction on the channel weighted convolution feature map group to obtain a deep convolution feature map group; specifically: Use time windows M A feature window is used to simultaneously perform windowing processing along the time dimension on each channel weighted convolution feature map in the channel weighted convolution feature map group, and each channel weighted convolution feature map is divided into M Window sub-feature maps of equal width; According to the variance layer, the variance of each sub-feature graph of each window in each feature window is calculated to obtain M The variance of each sub-feature graph of the window in the feature window; Performing a natural logarithm operation on the variance of each of the window sub-feature graphs to obtain a variance feature of each of the window sub-feature graphs; According to each of the channel weighted convolution feature maps corresponding to M The variance features are used to construct each depth convolution feature map corresponding to each channel weighted convolution feature map; According to each of the deep convolutional feature maps, the deep convolutional feature map group is constructed.

[0009] As an optional implementation of the first aspect of the present application, the temporal attention module performs temporal attention weighted processing on the deep convolutional feature map group to obtain a temporal weighted convolutional feature map group; specifically: Divide the variance feature in each feature window equally into h Variance feature groups; For each of the feature windows h Each of the variance feature groups is subjected to multi-head attention processing through a one-dimensional convolutional layer to obtain each attention feature corresponding to each of the variance features; right M Each of the attention features in the feature windows is spliced ​​along the time dimension to obtain each of the time-weighted convolution feature maps corresponding to each of the deep convolution feature maps; According to each of the time-weighted convolutional feature maps, the time-weighted convolutional feature map group is constructed.

[0010] In a second aspect, an embodiment of the present application provides a motor imagery EEG signal classification system, the system comprising: Acquisition module: acquires a data set, preprocesses the data set, and obtains a target data set; Building modules: building a channel and time attention mechanism convolutional network model, wherein the channel and time attention mechanism convolutional network model includes a channel attention convolutional module, a deep feature extraction module, a time attention module and a classification module; Training module: constructing a loss function, and training and testing the channel and time attention mechanism convolutional network model according to the loss function and the data set; Classification module: input the motor imagery EEG signal to be classified into the tested channel and time attention mechanism convolutional network model, and output the classification result of the motor imagery EEG signal data.

[0011] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0012] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0013] Compared with the prior art, this application has the following innovative features and beneficial effects: (1) This application combines convolutional neural networks with channel attention mechanisms, which can not only extract local temporal features of EEG signals, but also capture the global temporal correlation of EEG signals, which helps to extract discriminative features; (2) This application combines the variance layer with the temporal attention mechanism, which can not only reduce the temporal dimension of the feature map, but also extract more statistically significant features, greatly improving the feature representation of EEG signals based on motor imagery; (3) This application effectively extracts and fuses spatiotemporal and frequency features by intelligently integrating the channel attention convolution module, the deep feature extraction module and the temporal attention module, significantly improving the classification performance of brain-computer interface tasks based on motor imagery. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flow chart of a method for classifying motor imagery EEG signals provided by some embodiments of the present application; Figure 2 It is a structural diagram of a channel and time attention mechanism convolutional network model in a motor imagery EEG signal classification method provided by some embodiments of the present application; Figure 3 It is a structural diagram of a channel attention convolution module in a method for classifying motor imagery EEG signals provided by some embodiments of the present application; Figure 4 It is a channel attention layer structure diagram in a motor imagery EEG signal classification method provided by some embodiments of the present application; Figure 5 This is a structural diagram of a temporal attention module in a method for classifying motor imagery EEG signals provided in some embodiments of the present application. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0016] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0017] In the following, in conjunction with the accompanying drawings, a motor imagery EEG signal classification method and system provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0018] Example A method for classifying motor imagery electroencephalogram signals comprises the following steps: S100: Acquire a data set, pre-process the data set, and obtain a target data set; It should be noted that S100 is specifically: S110: acquiring a data set, the data set including four types of motor imagery EEG signals; S120: preprocessing the data set to obtain a target data set; Furthermore, the dataset in S110 comes from a four-category motor imagery EEG data set provided by the 4th International BCI Competition; the dataset includes MI-EEG (motor imagery EEG) data from 9 healthy subjects; each subject conducted two sessions of experiments. Each session contains 288 experiments, including four motor imagery tasks for the left hand, right hand, foot, and tongue, and each task has 72 experiments. The EEG signals are recorded using 22 electrode channels located in the motor cortex area with a sampling rate of 250 Hz; the EEG signals of all these electrodes are used for decoding the motor imagery task, which means that the dataset contains motor imagery EEG data for two sessions; It should be noted that in S120, the data set is preprocessed to obtain the target data set; specifically: S121: segmenting each motor imagery EEG signal in the data set, and intercepting a single motor imagery EEG signal from 0 s to 4 s after the task prompt in each motor imagery EEG signal; S122: performing bandpass filtering and sampling rate processing on each single motor imagery EEG signal according to a fourth-order Butterworth filter; S123: constructing a target data set according to each single motor imagery EEG signal in the data set after the filtering and sampling rate processing; Furthermore, the fourth-order Twos filter in S122 performs bandpass filtering on each single motor imagery EEG signal at an operating frequency of 4 Hz-40 Hz, and processes the sampling rate of each single motor imagery signal to 250 Hz; similarly, the target data set obtained in S123 corresponds to the data set, and includes single motor imagery EEG signal data of two time periods.

[0019] S200: Construct a channel and time attention mechanism convolutional network model, which includes a channel attention convolution module, a deep feature extraction module, a time attention module and a classification module; S300: Construct a loss function, and train and test the channel and time attention mechanism convolutional network model based on the loss function and the target dataset; It should be noted that in S300, the channel and time attention mechanism convolutional network model is trained and tested according to the loss function and the target data set; specifically: S310: construct a loss function and divide the target data set into a training set and a test set; S320: Train the channel and time attention mechanism convolutional network model according to the loss function and training set; S330: Test the trained channel and temporal attention mechanism convolutional network model based on the test set.

[0020] Furthermore, in S310, the target data set is divided into a training set and a test set by using the data of two periods in the target data set as the training set and the test set respectively; in S320, for model training, the PyTorch library in Python 3.7 is used as the software environment for model building and training, and the hardware environment includes an Intel Core i5-12400F CPU and an NVIDIA GeForce RTX 3080 GPU; cross entropy loss is used for network optimization. The Adam optimizer is used with default settings. The initial learning rate is set to 0.001, and if the training loss does not improve for 20 consecutive rounds, the learning rate is reduced by 0.6 times. The random dropout rate is set to 0.5. The batch size for each round is 64. The training process of the model is divided into the following two stages. In the first stage, all weights in the network are initialized using the Glorot initializer, and the bias is set to zero. Early stopping is enabled, and the maximum number of training rounds and maximum patience are set to 500 and 50 respectively, which means that if the validation accuracy does not improve for 50 consecutive rounds, the training of the first stage will be terminated and the current network weight parameters will be saved; in the second stage, the weight parameters of the first stage are inherited and the number of training rounds is set to 200.

[0021] S400: Input the motor imagery EEG signal to be classified into the tested channel and time attention mechanism convolutional network model, and output the classification result of the motor imagery EEG signal data.

[0022] It should be noted that S400 is specifically: S410: inputting the motor imagery EEG signal to be classified into the channel attention convolution module of the tested channel and time attention mechanism convolution network model, the channel attention convolution module performs convolution processing on the motor imagery EEG signal to be classified, and outputs a channel weighted convolution feature map group; S420: Inputting the channel weighted convolution feature map group into the depth feature extraction module, and the depth feature extraction module performs depth feature extraction on the channel weighted convolution feature map group to obtain the depth convolution feature map group; S430: Inputting the deep convolution feature map group into the temporal attention module, and the temporal attention module performs temporal attention weighted processing on the deep convolution feature map group to obtain a temporal weighted convolution feature map group; S440: Input the time-weighted convolution feature map group into the classification module, and the classification module performs classification processing on the time-weighted feature map group to obtain the category corresponding to the motor imagery EEG signal to be classified, thereby completing the classification of the motor imagery EEG signal to be classified.

[0023] It should be noted that the channel attention convolution module in S410 performs convolution processing on the motor imagery EEG signal to be classified, and outputs a channel weighted convolution feature map group; specifically: S411: the motor imagery EEG signal to be classified is input into the channel attention convolution module, the first time convolution layer in the channel attention convolution module performs time convolution processing on the motor imagery EEG signal to be classified to obtain an initial feature map group, and the initial feature map group is normalized according to the first batch of normalization layers; S412: performing spatial convolution processing on the initial feature map group after the normalization processing according to the spatial convolution layer to obtain a spatial feature map group; and then performing normalization processing and nonlinear activation processing on the spatial feature map group in sequence according to the second batch normalization layer and the first nonlinear activation layer; S413: performing average pooling processing on the spatial feature map group after the nonlinear activation processing according to the average pooling layer, and then performing random dropout processing on the spatial feature map group after the average pooling processing according to the random dropout layer; S414: performing channel attention weighting processing on the spatial feature map group after random dropout processing according to the channel attention layer to obtain a channel weighted feature map group; S415: performing temporal convolution processing on the channel weighted feature map group according to the second temporal convolution layer, and then performing normalization processing and nonlinear activation processing on the spatial feature map group in sequence according to the third batch normalization layer and the second nonlinear activation layer to obtain a channel weighted convolution feature map group.

[0024] Furthermore, the first temporal convolutional layer in S411 is used to capture frequency information above 4 Hz. This layer uses The size is The filter is Set to 32, Set to 64. Also, the Padding parameter is set to , ensuring that the dimension of the EEG signal remains unchanged after the convolution operation; the initial feature map group output by this layer includes Then, the initial feature map group output by the first temporal convolutional layer is normalized using the first batch normalization (BN) layer to maintain the stability of the data distribution.

[0025] The spatial convolutional layer in S412 contains The size is , where S is the number of electrode channels used for motor imagery task decoding, and the number of filter groups is set to , which means that each feature channel has D filters; among them, , the spatial feature map group output by this layer includes The first nonlinear (ELU) activation layer is used to standardize and nonlinearly activate the spatial feature map group, respectively.

[0026] In S413, a An average pooling (Avg. Pool) layer and a random dropout layer (random dropout rate is 0.5) are used for average pooling and random dropout. In the average pooling layer, K2 is set to 8. The average pooling is used to reduce the sampling rate of the spatial feature map group after nonlinear activation processing from 250 Hz to about 32 Hz, and reduce the length of the spatial feature map to 1000 / 8=125. The random dropout is to reduce the overfitting of the spatial feature map group.

[0027] In S414, global average pooling (GAP) is performed on the spatial feature map group after random dropout processing, and its dimension is changed from ( N , C , S , T ) is compressed to ( N , C , 1, 1), where N , C , S , T Represents batch size, number of feature channels, spatial dimension size, and temporal dimension size respectively. Global average pooling is used to capture the global information of the entire spatial feature map group; then one-dimensional convolution (C1D) is used to calculate the channel weights of the spatial feature map group after GAP. , as shown below: , in, Represents the spatial feature map after GAP processing, Indicates that the convolution kernel size is One-dimensional convolution of According to the number of feature channels C Adaptively determined as shown below: , in, Representation and The nearest odd number, and Used to change the ratio of channels to convolution kernels. The default setting is 2. The default setting is 1; Using Channel Weights The channels of the spatial feature map are weighted to obtain the channel weighted feature map, as shown in the following formula: , in, represents the channel weighted feature map, represents the spatial feature map, Represents element-wise product.

[0028] The second temporal convolutional layer in S415 uses The size is The filter, where It is empirically set to 16 for decoding 500 ms of MI activity, aiming to further extract temporal features. The third BN layer and the second ELU activation layer are sequentially followed, which are used to normalize and nonlinearly activate the output of the second temporal convolution layer, respectively, to obtain a channel-weighted convolution feature map group.

[0029] It should be noted that in S420, the channel weighted convolution feature map group is input into the depth feature extraction module, and the depth feature extraction module performs depth feature extraction on the channel weighted convolution feature map group to obtain the depth convolution feature map group; specifically: S421: Divide time windows for use M The feature windows simultaneously perform windowing processing along the time dimension on each channel weighted convolution feature map in the channel weighted convolution feature map group, and divide each channel weighted convolution feature map into M Window sub-feature maps of equal width; S422: Calculate the variance of each sub-feature graph in each feature window according to the variance layer to obtain M The variance of the sub-feature graph of each window in the feature window; S423: performing a natural logarithm operation on the variance of each window sub-feature graph to obtain a variance feature of each window sub-feature graph; S424: According to the corresponding weighted convolution feature map of each channel M Variance features are used to construct each depth convolution feature map corresponding to each channel weighted convolution feature map; S425: Construct a deep convolutional feature map group according to each of the deep convolutional feature maps.

[0030] Furthermore, in S421, each channel weighted convolution feature map is divided into M weighted window sub-feature maps of equal width, where , Represents the length of each channel weighted convolution feature map, Indicates that the width of each weighted window sub-feature map is, Set to 20.

[0031] S422 In the feature window The variance of the sub-feature map of a window is calculated as follows: , in, Indicates Channel weighted convolution feature map, Indicates In the feature window The average value of the sub-feature maps of the windows, Indicates The value of a time sampling point.

[0032] It should be noted that the temporal attention module in S430 performs temporal attention weighted processing on the deep convolutional feature map group to obtain a temporal weighted convolutional feature map group; specifically: S431: Divide the variance features in each feature window into equal parts h Variance feature groups; S432: For each feature window corresponding to h Each variance feature group is processed by a one-dimensional convolution layer for multi-head attention to obtain each attention feature corresponding to each variance feature; S433: Yes M Each attention feature in the feature window is concatenated along the time dimension to obtain each time-weighted convolution feature map corresponding to each depth convolution feature map; S434: Construct a time-weighted convolutional feature map group based on each time-weighted convolutional feature map.

[0033] Further, in S432 The time-weighted convolution feature map is expressed as follows: , in, represents the weight of one-dimensional convolution, mod represents the modulo operation, Indicates In the feature window attention feature.

[0034] According to a motor imagery EEG signal classification method of the present embodiment, by constructing a network structure including a channel attention convolution module, a deep feature extraction module, a time attention module and a classification module, the method can more finely capture the features in the motor imagery EEG signal, thereby improving the accuracy of classification; the channel attention mechanism can automatically adjust the importance of different channels, which helps to highlight key signal features. The time attention mechanism can pay attention to the changes in the signal in the time dimension, further enhancing the capture of dynamic features. The deep feature extraction module effectively reduces the feature dimension and reduces the complexity of subsequent processing by dividing the time window and calculating the variance feature. The multi-head attention processing in the time attention module allows parallel computing and improves the processing speed. The method can process different types of motor imagery EEG signals, whether four or two, and can adapt by adjusting the model parameters and structure.

[0035] It should be noted that the motor imagery EEG signal classification method provided in the embodiment of the present application can be executed by a motor imagery EEG signal classification system, or, or, a control module in the motor imagery EEG signal classification system for executing and loading a motor imagery EEG signal classification method. In the embodiment of the present application, a motor imagery EEG signal classification system is used to execute and load a motor imagery EEG signal classification method as an example to illustrate a motor imagery EEG signal classification method provided in the embodiment of the present application.

[0036] A motor imagery EEG signal classification system, comprising: Acquisition module: acquires the data set, preprocesses the data set, and obtains the target data set; Building modules: Building a channel and temporal attention mechanism convolutional network model, which includes a channel attention convolutional module, a deep feature extraction module, a temporal attention module, and a classification module. Training module: construct loss function, train and test channel and time attention mechanism convolutional network models according to loss function and data set; Classification module: The motor imagery EEG signal to be classified is input into the tested channel and temporal attention mechanism convolutional network model, and the classification result of the motor imagery EEG signal data is output.

[0037] A motor imagery EEG signal classification system in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, or a personal digital assistant (personal digital assistant, PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), etc., which is not specifically limited in the embodiment of the present application.

[0038] A motor imagery EEG signal classification system in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0039] The motor imagery EEG signal classification system provided in the embodiment of the present application can achieve Figures 1 to 5The various processes of implementing a method for classifying motor imagery EEG signals in the method embodiment will not be described in detail here to avoid repetition.

[0040] According to a motor imagery EEG signal classification system of this embodiment, the acquisition module acquires a data set, provides rich data resources for subsequent preprocessing and classification, and preprocesses the data set to obtain a target data set, ensuring the consistency and quality of the data, and laying a solid foundation for subsequent feature extraction and classification. The channel and time attention mechanism convolution network model designed in the construction module can make full use of the channel and time dimension information in the EEG signal to achieve efficient feature extraction. The channel attention convolution module can automatically adjust the importance of different channels and highlight key signal features; the deep feature extraction module further extracts the dynamic features of the signal through time window division and variance calculation; the time attention module focuses on the changes in the signal in the time dimension and enhances the expression ability of the features. The training module ensures the accuracy and generalization ability of the model by constructing a loss function and training and testing the model using the target data set. The classification module can input the motor imagery EEG signal to be classified into the tested model to obtain accurate classification results. By combining mechanisms such as channel attention, time attention and deep feature extraction, the system can more comprehensively capture the features in the EEG signal, thereby improving the accuracy of classification. The system provides an effective solution for the classification of motor imagery EEG signals, which helps promote the development and application of brain-computer interface technology. By accurately classifying motor imagery EEG signals, the system provides strong support for the fields of motion control, rehabilitation assistance, etc. in brain-computer interface systems.

[0041] Optionally, an embodiment of the present application also provides an electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, each process of the above-mentioned motor imagery EEG signal classification method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0042] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned motor imagery EEG signal classification method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0043] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0044] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0045] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0046] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A motor imagery EEG signal classification method, characterized in that: The method comprises: Acquire a data set, and preprocess the data set to obtain a target data set; Constructing a channel and time attention mechanism convolutional network model, wherein the channel and time attention mechanism convolutional network model includes a channel attention convolution module, a deep feature extraction module, a time attention module and a classification module; Constructing a loss function, and training and testing the channel and time attention mechanism convolutional network model according to the loss function and the target data set; The motor imagery EEG signal to be classified is input into the tested channel and time attention mechanism convolutional network model, and the classification result of the motor imagery EEG signal data is output.

2. A motor imagery EEG signal classification method according to claim 1, characterized in that: The motor imagery EEG signal to be classified is input into the channel and time attention mechanism convolutional network model after the test, and the motor imagery EEG signal data classification result is output; specifically: Inputting the motor imagery EEG signal to be classified into the channel attention convolution module of the channel and time attention mechanism convolution network model after the test, the channel attention convolution module performs convolution processing on the motor imagery EEG signal to be classified, and outputs a channel weighted convolution feature map group; Inputting the channel weighted convolution feature map group into the deep feature extraction module, the deep feature extraction module performs deep feature extraction on the channel weighted convolution feature map group to obtain a deep convolution feature map group; Inputting the deep convolutional feature map group into the temporal attention module, and the temporal attention module performs temporal attention weighted processing on the deep convolutional feature map group to obtain a temporal weighted convolutional feature map group; The time-weighted convolution feature map group is input into the classification module, and the classification module performs classification processing on the time-weighted feature map group to obtain the category corresponding to the motor imagery EEG signal to be classified, thereby completing the classification of the motor imagery EEG signal to be classified.

3. A motor imagery EEG signal classification method according to claim 2, characterized in that: The channel attention convolution module performs convolution processing on the motor imagery EEG signal to be classified, and outputs a channel weighted convolution feature map group; specifically: The motor imagery EEG signal to be classified is input into the channel attention convolution module, the first time convolution layer in the channel attention convolution module performs time convolution processing on the motor imagery EEG signal to be classified to obtain an initial feature map group, and the initial feature map group is standardized according to the first batch of normalization layers; Performing spatial convolution processing on the initial feature map group after the standardization processing according to the spatial convolution layer to obtain a spatial feature map group; and then performing standardization processing and nonlinear activation processing on the spatial feature map group in sequence according to the second batch normalization layer and the first nonlinear activation layer; Performing average pooling processing on the spatial feature map group after the nonlinear activation processing according to the average pooling layer, and then performing random deactivation processing on the spatial feature map group after the average pooling processing according to the random deactivation layer; Performing channel attention weighted processing on the spatial feature map group after random dropout processing according to the channel attention layer to obtain a channel weighted feature map group; The channel weighted feature map group is subjected to temporal convolution processing according to the second temporal convolution layer, and then the spatial feature map group is subjected to standardization processing and nonlinear activation processing in sequence according to the third batch normalization layer and the second nonlinear activation layer to obtain the channel weighted convolution feature map group.

4. The method for classifying motor imagery EEG signals according to claim 3, characterized in that: The channel weighted convolution feature map group is input into the depth feature extraction module, and the depth feature extraction module performs depth feature extraction on the channel weighted convolution feature map group to obtain the depth convolution feature map group; specifically: Use time windows M A feature window is used to simultaneously perform windowing processing along the time dimension on each channel weighted convolution feature map in the channel weighted convolution feature map group, and each channel weighted convolution feature map is divided into M Window sub-feature maps of equal width; According to the variance layer, the variance of each sub-feature graph of each window in each feature window is calculated to obtain M The variance of each sub-feature graph of the window in the feature window; Performing a natural logarithm operation on the variance of each of the window sub-feature graphs to obtain a variance feature of each of the window sub-feature graphs; According to each of the channel weighted convolution feature maps corresponding to M The variance features are used to construct each depth convolution feature map corresponding to each channel weighted convolution feature map; According to each of the deep convolutional feature maps, the deep convolutional feature map group is constructed.

5. A motor imagery EEG signal classification method according to claim 4, characterized in that: The temporal attention module performs temporal attention weighted processing on the deep convolutional feature map group to obtain a temporal weighted convolutional feature map group; specifically: Divide the variance feature in each feature window equally into h Variance feature groups; For each of the feature windows h Each of the variance feature groups is subjected to multi-head attention processing through a one-dimensional convolutional layer to obtain each attention feature corresponding to each of the variance features; right M Each of the attention features in the feature windows is spliced ​​along the time dimension to obtain each of the time-weighted convolution feature maps corresponding to each of the deep convolution feature maps; According to each of the time-weighted convolutional feature maps, the time-weighted convolutional feature map group is constructed.

6. A motor imagery EEG signal classification system, implementing a motor imagery EEG signal classification method as claimed in any one of claims 1 to 5, characterized in that: The system comprises: Acquisition module: acquires a data set, preprocesses the data set, and obtains a target data set; Building modules: building a channel and time attention mechanism convolutional network model, wherein the channel and time attention mechanism convolutional network model includes a channel attention convolutional module, a deep feature extraction module, a time attention module and a classification module; Training module: constructing a loss function, and training and testing the channel and time attention mechanism convolutional network model according to the loss function and the target data set; Classification module: input the motor imagery EEG signal to be classified into the tested channel and time attention mechanism convolutional network model, and output the classification result of the motor imagery EEG signal data.

7. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of a motor imagery EEG signal classification method as described in any one of claims 1 to 5 are implemented.

8. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps of the motor imagery EEG signal classification method as described in any one of claims 1-5 are implemented.

Images