Method and device for decoding motor imagery electroencephalogram signals of stroke patient

Abstract

The invention relates to the technical field of signal processing, and discloses a stroke patient motor imagery electroencephalogram signal decoding method and device, and the method comprises the steps: obtaining an original electroencephalogram signal; the original electroencephalogram signals are preprocessed; the preprocessed original electroencephalogram signals are input into a pre-established feature extraction model, diversified feature extraction is carried out, a feature set is obtained, and the feature set is used for motor imagery classification; wherein the feature extraction model comprises a channel attention module and a space attention module which are arranged in parallel; wherein the channel attention module is used for adaptively distributing a channel attention weight to each channel in an original electroencephalogram signal; and the channel attention module is used for adaptively allocating a space attention weight to each space in the original electroencephalogram signals. According to the method, weight distribution can be carried out on different channels and spatial positions of a stroke patient in a self-adaptive mode, local channel and spatial feature responses are strengthened, and accurate recognition of stroke rehabilitation motor imagery electroencephalogram signals is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and in particular to a method and device for decoding motor imagery electroencephalogram signals of stroke patients. Background Art

[0002] Brain-Computer Interface (BCI) technology is a technology that communicates or controls by directly connecting the brain with external devices. By reading and decoding neural signals in the brain, these signals are then converted into instructions for controlling machines. BCI based on non-invasive electroencephalogram (EEG) signals is often used in the BCI field due to its high temporal resolution, non-invasiveness, and convenience. The Motor Imagery (MI) paradigm is an important paradigm in the BCI field and can reflect an individual's voluntary movement intention by simulating movements in the brain. The Motor Imagery-based Brain-Computer Interface (MI-BCI) system has broad application prospects in fields such as post-stroke rehabilitation and control of medical auxiliary devices.

[0003] In order to accurately identify the true intention of stroke patients from MI-EEG signals, deep learning-based methods have been applied to the field of MI-EEG decoding. In the field of deep learning, Convolutional Neural Network (CNN) has received extensive attention from researchers because it can learn latent information from raw data. However, due to the limitation of the convolutional kernel size of CNN, the receptive field of the network is limited, resulting in the network being able to only learn local information of EEG signals and being difficult to capture the global spatio-temporal dependence of EEG signals in the motor imagery tasks of stroke patients. And based on the current deep learning MI-EEG decoding methods, usually the same attention is given to each channel or spatial position. Since stroke patients may have impaired functions in some brain regions, and the impaired brain regions may be different for different stroke patients. At this time, if the same attention is still directly given to each channel or spatial position, it is difficult to fully capture the individualized motor imagery characteristics of each stroke patient. Summary of the Invention

[0004] In view of this, the present invention provides a method and device for decoding motor imagery electroencephalogram signals of stroke patients to solve the problem in the prior art that there is no decoding for stroke patients and the individual differences in the motor imagery of stroke patients, resulting in low decoding accuracy of MI-EEG.

[0005] In a first aspect, the present invention provides a method for decoding motor imagery electroencephalogram signals of stroke patients, the method comprising:

[0006] Obtaining raw electroencephalogram signals;

[0007] Preprocess the original EEG signals;

[0008] Input the preprocessed original EEG signals into a pre-established feature extraction model for diverse feature extraction to obtain a feature set, which is used for motor imagery classification; wherein, the feature extraction model includes a channel attention module and a spatial attention module arranged in parallel; wherein, the channel attention module is used to adaptively assign channel attention weights to each channel in the original EEG signals; the spatial attention module is used to adaptively assign spatial attention weights to each space in the original EEG signals.

[0009] The channel attention module of the present invention can focus on the importance of different EEG channels, and the spatial attention module can strengthen the local responses at different spatial positions within the same channel. The channel attention mechanism and the spatial attention mechanism can adaptively assign weights to different channels and spatial positions of stroke patients, strengthen the local channel and spatial feature responses related to EEG signals, and the combination of the two can effectively achieve local feature enhancement with "channel - space" double weighting. In addition, in the present invention, multiple attention mechanisms are combined with a multi-scale convolutional neural network for extracting diverse features. Through enhancing local feature responses and rich feature extraction, accurate recognition of EEG signals for stroke rehabilitation motor imagery can be effectively achieved.

[0010] In an alternative embodiment, the feature extraction model further includes:

[0011] A multi-scale temporal convolution module, which includes temporal convolution layers of different scales arranged in parallel;

[0012] The input end of the multi-scale temporal convolution module is used to input the preprocessed original EEG signals, and the output end is used to connect to the channel attention module and the spatial attention module.

[0013] The deep learning algorithm combining multiple attentions and a multi-scale convolutional neural network adopted in this embodiment can extract effective features from the EEG signals of different stroke patients. Through the cooperation of multi-scale convolution and multi-attention mechanisms, multi-level diverse spatio-temporal feature fusion extraction combining local feature enhancement and global dependence capture is realized, and finally an efficient decoding model adapted to the individual differences of stroke patients is formed to improve the accuracy of motor imagery classification.

[0014] In an alternative embodiment, the feature extraction model further includes:

[0015] A first spatial convolution layer, connected to the output end of the channel attention module;

[0016] The second spatial convolutional layer is connected to the output end of the spatial attention module; both the first spatial convolutional layer and the second spatial convolutional layer are used to extract spatial feature information;

[0017] The first spatial residual convolutional layer is connected to the output end of the first spatial convolutional layer;

[0018] The second spatial residual convolutional layer is connected to the output end of the second spatial convolutional layer;

[0019] The first spatial residual convolutional layer and the second spatial residual convolutional layer are both arranged in parallel with the channel attention module and the spatial attention module.

[0020] In this embodiment, rich spatial features are extracted through non-parameter-sharing spatial convolution, and at the same time, the key feature information is amplified by combining the residual connection convolutional layer, constructing diversified spatial feature extraction, which can adapt to the inter-individual differences of different stroke patients and effectively improve the accuracy of motor imagery classification.

[0021] In an alternative embodiment, the feature extraction model further includes:

[0022] The multi-head attention module is arranged at the output ends of the first spatial convolutional layer and the second spatial convolutional layer, and is used to capture global time and space dependencies.

[0023] In this embodiment, the multi-head attention module can capture the long-range dependencies between different time points and channels, thereby capturing the global spatio-temporal dependencies of the EEG signals in the motor imagery tasks of stroke patients, effectively supplementing the limited receptive field in the convolutional neural network, improving the comprehensiveness and accuracy of the feature extraction of the motor imagery EEG signals of stroke patients, and thus improving the accuracy of inter-individual decoding of stroke patients.

[0024] In an alternative embodiment, the channel attention module includes:

[0025] The global average pooling layer is used to compress the spatial information of each channel in the original EEG signal to obtain global feature information;

[0026] The first fully connected layer is used to perform channel compression on the global feature information to obtain the downsampled global feature information;

[0027] The ReLU activation function layer is connected to the first fully connected layer and is used to introduce non-linearity;

[0028] The second fully connected layer is connected to the ReLU activation function layer and is used to perform channel restoration;

[0029] The first Sigmoid activation function layer is connected to the second fully connected layer and is used to generate the channel attention weights for each channel.

[0030] In this embodiment, the channel attention mechanism can enhance the features of important channels and suppress unimportant features to achieve adaptive allocation of features, effectively enhancing the response of local channels.

[0031] In an alternative embodiment, the spatial attention module includes:

[0032] A global max pooling layer for performing global max pooling on the input feature map in the channel dimension to obtain a first sub-feature map;

[0033] A global average pooling, arranged in parallel with the global max pooling layer, for performing global average pooling on the input feature map in the channel dimension to obtain a second sub-feature map;

[0034] A concatenation layer for concatenating the first sub-feature map and the second sub-feature map to obtain a third sub-feature map;

[0035] A two-dimensional convolutional layer for performing a convolutional operation on the third sub-feature map to reduce the channel dimension;

[0036] A second Sigmoid activation function layer, connected to the output end of the two-dimensional convolutional layer, for generating the spatial attention weights for each channel.

[0037] In this embodiment, the spatial attention mechanism can focus on the key features at different spatial positions within the same channel, effectively enhancing the feature responses at different spatial positions.

[0038] In a second aspect, the present invention provides a device for decoding motor imagery electroencephalogram signals of stroke patients, the device includes:

[0039] An acquisition module for acquiring raw electroencephalogram signals;

[0040] A processing module for preprocessing the raw electroencephalogram signals;

[0041] A decoding module for inputting the preprocessed raw electroencephalogram signals into a pre-established feature extraction model for diverse feature extraction to obtain a feature set, the feature set being used for motor imagery classification; wherein, the feature extraction model includes a channel attention module and a spatial attention module arranged in parallel; wherein, the channel attention module is used to adaptively assign channel attention weights to each channel in the raw electroencephalogram signals; the channel attention module is used to adaptively assign spatial attention weights to each space in the raw electroencephalogram signals.

[0042] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the decoding method for the motor imagery EEG signals of stroke patients according to the first aspect or any corresponding embodiment thereof.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the decoding method for the motor imagery EEG signals of stroke patients according to the first aspect or any corresponding embodiment thereof.

[0044] In a fifth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to execute the decoding method for the motor imagery EEG signals of stroke patients according to the first aspect or any corresponding embodiment thereof.

[0045] It should be noted that since the decoding device for the motor imagery EEG signals of stroke patients, the computer device, the computer-readable storage medium, and the computer program product provided by the present invention correspond to the above-mentioned decoding method for the motor imagery EEG signals of stroke patients. Therefore, for the beneficial effects of the decoding device for the motor imagery EEG signals of stroke patients, the computer device, the computer-readable storage medium, and the computer program product, please refer to the description of the corresponding beneficial effects of the decoding method for the motor imagery EEG signals of stroke patients above, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1 is a flowchart of the decoding method for the motor imagery EEG signals of stroke patients according to an embodiment of the present invention;

[0048] Figure 2 is a structural diagram of the feature extraction model according to an embodiment of the present invention;

[0049] Figure 3 is a block diagram of the decoding device for the motor imagery EEG signals of stroke patients according to an embodiment of the present invention;

[0050] Figure 4 is a schematic hardware structure diagram of the computer device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0052] In recent years, the attention mechanism has been applied to the MI-EEG decoding field to enhance the adaptive ability and robustness of the model because it can selectively focus on relevant EEG information and dynamically adjust the weights of EEG features. The combination of the attention mechanism and CNN can improve the decoding performance to a certain extent. Since brain activities change dynamically over time and brain regions, and EEG signals contain rich temporal and spatial information. Enhancing local channel and spatial feature responses and extracting diverse spatio-temporal features will be more conducive to improving the decoding performance of MI-EEG. However, in some current methods, the extraction of diverse spatio-temporal features has not been effectively combined, resulting in limited decoding performance of electroencephalogram signals for motor imagery in stroke rehabilitation. In addition, since different stroke patients have damaged brain regions differently, it is unreasonable to give the same attention to each channel or spatial position. Therefore, more attention needs to be paid to the channel and spatial position features related to motor imagery.

[0053] In view of this, according to the embodiments of the present invention, an embodiment of a method for decoding electroencephalogram signals for motor imagery of stroke patients is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0054] In this embodiment, a method for decoding electroencephalogram signals for motor imagery of stroke patients is provided, which can be executed by devices such as servers, terminals, and mobile terminals. Figure 1 is a flowchart of the method for decoding electroencephalogram signals for motor imagery of stroke patients according to the embodiments of the present invention, as Figure 1 shown, and the process includes the following steps:

[0055] Step S101, obtain the original electroencephalogram signal.

[0056] The original electroencephalogram signal is the electroencephalogram signal obtained by non-invasive methods at multi-channel scalp electrodes. When using an electroencephalogram signal dataset for model training, it can be obtained from a public dataset.

[0057] Step S102, preprocess the original electroencephalogram signal.

[0058] After obtaining the original EEG signals, first, the original EEG signals are aligned in Euclidean space to reduce the influence of interference signals and inter-individual differences, and then z-score normalization is performed on the original EEG signals. Taking the public data as an example, the original EEG signal data of the i-th trial can be expressed as The corresponding label is y ∈ {1, 2,..., N c}. Where C, T, and N c represent the number of channels, the number of time sampling points, and the total number of categories of the EEG signals, respectively.

[0059] The preprocessing method is specifically described as follows:

[0060] First, Euclidean space alignment is performed, as shown in the following formula:

[0061]

[0062] In the formula, i ∈ {1, 2,..., N} is the i-th trial of the original EEG signal data X, and N is the number of trials.

[0063] Secondly, z-score normalization is performed, as shown in the following formula:

[0064]

[0065] In the formula, represents the EEG data of the j-th channel of the i-th trial after z-score normalization, represents the EEG data of the j-th channel of the i-th trial after Euclidean space alignment, and μ and γ respectively represent the mean and standard deviation.

[0066] Step S103: Input the preprocessed original EEG signals into a pre-established feature extraction model for diverse feature extraction to obtain a feature set, which is used for motor imagery classification; among them, the feature extraction model includes a channel attention module and a spatial attention module arranged in parallel; among them, the channel attention module is used to adaptively assign channel attention weights to each channel in the original EEG signals; the channel attention module is used to adaptively assign spatial attention weights to each space in the original EEG signals.

[0067] Regarding the channel attention module, first, the spatial information of each channel can be compressed through global average pooling operation (F sq ). Then, two fully connected layers (F ex ) are used to generate channel attention weights. The specific implementation process can be expressed as follows:

[0068]

[0069] where z n is the n-th element of the result feature map z after passing through F sq , and is the two-dimensional EEG signal of the n-th channel, σ is the Sigmoid function, the Sigmoid function is used to map the weights, δ is the ReLU activation function, and r are both hyperparameters for controlling feature dimensionality reduction, and ω channel is the channel attention weight.

[0070] In this embodiment, the channel attention mechanism can enhance the features of important channels and suppress unimportant features to achieve adaptive allocation of features, which can effectively enhance the response of local channels.

[0071] Regarding the spatial attention module, first, global max pooling and global average pooling can be performed on the input feature map in the channel dimension to obtain two sub-feature maps. Then, the two sub-feature maps are concatenated, and the channel dimension is reduced through a convolutional operation. Finally, the spatial attention weight ω spatial is mapped using the Sigmoid function. The specific implementation process can be expressed as follows:

[0072] ω spatial = σ(F Conv2D ) = σ(Conv2D k ([F max ; F avg ));

[0073] where σ is the Sigmoid function, Conv2D k represents a two-dimensional convolution with a convolution kernel size of k, F max is the sub-feature map obtained by global max pooling, and F avg is the sub-feature map obtained by global average pooling.

[0074] In this embodiment, the spatial attention mechanism can focus on the key features at different spatial positions within the same channel, which can effectively enhance the feature response at different spatial positions.

[0075] In addition, a multi-scale temporal convolution module, a spatial residual convolution layer, etc. are also set in the feature extraction model for diverse feature extraction. Based on the finally obtained feature set, classification is performed through a fully connected layer, a Softmax function, etc., so as to obtain the prediction result of the motor imagery task.

[0076] In this embodiment, the channel attention module can focus on the importance of different EEG channels, and the spatial attention module can enhance the local responses at different spatial positions within the same channel. The channel attention mechanism and the spatial attention mechanism can adaptively assign weights to different channels and spatial positions of stroke patients, strengthen the local channel and spatial feature responses related to EEG signals, and the combination of the two can effectively achieve local feature enhancement with "channel - space" double weighting. In addition, in this embodiment, multiple attention mechanisms are combined with a multi - scale convolutional neural network for extracting diverse features. Through enhancing local feature responses and rich feature extraction, accurate recognition of EEG signals for stroke rehabilitation motor imagery can be effectively achieved.

[0077] In some alternative embodiments, referring to Figure 2 as shown, the feature extraction model further includes:

[0078] A multi - scale temporal convolution module, which includes temporal convolution layers of different scales arranged in parallel;

[0079] The input end of the multi - scale temporal convolution module is used to input the pre - processed original EEG signal, and the output end is used to connect to the channel attention module and the spatial attention module.

[0080] Specifically, multi - scale convolutional layers can be designed, and the kernel sizes of the convolutional kernels can be 1, 3, 5, 7, etc. Small - scale convolutional kernels can focus on short - term local temporal features, and large - scale convolutional kernels can capture long - term temporal dependencies. The features can be formed into a multi - scale temporal feature representation through concatenation or weighted fusion. By performing temporal feature extraction on the pre - processed EEG signal through multi - scale temporal convolution, motor imagery EEG signals can be better classified. By adaptively extracting rich features of different stroke patients through multi - scale convolutional layers composed of multiple scale convolutional kernels, it is also possible to enhance the adaptability to inter - individual differences while extracting rich local temporal feature information.

[0081] The deep - learning algorithm combining multiple attentions and a multi - scale convolutional neural network adopted in this embodiment can extract effective features from the EEG signals of different stroke patients. Through the cooperation of multi - scale convolution and multi - attention mechanisms, multi - level and diverse spatio - temporal feature fusion extraction combining local feature enhancement and global dependency capture is realized, and finally an efficient decoding model adapted to the individual differences of stroke patients is formed to improve the accuracy of motor imagery classification.

[0082] In some alternative embodiments, the feature extraction model further includes:

[0083] A first spatial convolution layer, connected to the output end of the channel attention module;

[0084] The second spatial convolutional layer is connected to the output end of the spatial attention module; both the first spatial convolutional layer and the second spatial convolutional layer are used to extract spatial feature information;

[0085] The first spatial residual convolutional layer is connected to the output end of the first spatial convolutional layer;

[0086] The second spatial residual convolutional layer is connected to the output end of the second spatial convolutional layer;

[0087] The first spatial residual convolutional layer and the second spatial residual convolutional layer are arranged in parallel with the channel attention module and the spatial attention module.

[0088] After the EEG signals of different branches enhance the local feature information through channel attention and spatial attention, the spatial feature information of the corresponding branches can be extracted through the spatial convolutional layer. For example, 2D spatial convolution can be performed independently on each channel, which can avoid the bias caused by parameter sharing among different channels. Then, the feature information output by the residual connection convolution is added to the output of the spatial convolution, which can retain the underlying key features, enable the model to better fuse low-level and high-level features, and at the same time alleviate the problem of gradient disappearance.

[0089] In this embodiment, rich spatial features are extracted through non-parameter-sharing spatial convolution, and at the same time, the key feature information is amplified by combining the residual connection convolutional layer, constructing diverse spatial feature extraction, which can adapt to the inter-individual differences of different stroke patients and effectively improve the accuracy of motor imagery classification.

[0090] In some alternative embodiments, the feature extraction model further includes:

[0091] The multi-head attention module is arranged at the output ends of the first spatial convolutional layer and the second spatial convolutional layer and is used to capture global time and spatial dependencies.

[0092] The multi-head attention module consists of multiple self-attention heads running in parallel. The input of each attention head consists of a query Q, a key K, and a value V with a dimension of d k . The implementation process of self-attention can be expressed as follows:

[0093]

[0094] In the formula, K T is the transpose of K.

[0095] First, calculate the dot product of Q and all K T , then divide each dot product by Finally, apply the Softmax function to obtain the weights of V. Then, perform h linear transformations on the matrices of Q, K, and V with a dimension of d K , and finally these outputs are concatenated to output global dependence features.

[0096] In this embodiment, the multi-head attention module can capture long-range dependencies between different time points and channels, thereby capturing the global spatio-temporal dependence of EEG signals in the motor imagery task of stroke patients. It can effectively supplement the limited receptive field in the convolutional neural network, improve the comprehensiveness and accuracy of feature extraction of EEG signals in the motor imagery of stroke patients, and thus improve the accuracy of inter-individual decoding of stroke patients.

[0097] In some alternative embodiments, the channel attention module includes:

[0098] A global average pooling layer for compressing the spatial information of each channel in the original EEG signal to obtain global feature information;

[0099] A first fully connected layer for compressing the channels of the global feature information to obtain the global feature information after dimensionality reduction;

[0100] A ReLU activation function layer connected to the first fully connected layer for introducing non-linearity;

[0101] A second fully connected layer connected to the ReLU activation function layer for channel recovery;

[0102] A first Sigmoid activation function layer connected to the second fully connected layer for generating the channel attention weight of each channel.

[0103] First, perform global average pooling on the input feature map to compress the feature map of each channel into a real number. That is, compress the feature map of each channel in the spatial dimension through the pooling operation to obtain a one-dimensional feature vector, which represents the global feature information of each channel in the entire space. Secondly, use two fully connected layers (first reduce the dimension and then increase the dimension) and a ReLU activation function to learn the dependencies between channels, and generate a weight vector through the Sigmoid function. Specifically, the first fully connected layer compresses the number of channels to reduce the computational amount, and then introduces non-linearity through the ReLU activation function; the second fully connected layer restores the number of channels. Finally, limit the output to between (0-1) through the Sigmoid activation function to obtain the channel attention weight of each channel.

[0104] Specific details are shown in the following formula:

[0105]

[0106] After obtaining the channel attention weights for each channel, the model will automatically multiply the learned weights with the original feature map channel by channel to achieve recalibration of the features. The feature map of each channel is multiplied by its corresponding weight. The features of the channels with larger weights will be amplified, while the features of the channels with smaller weights will be suppressed. By weighting the original feature map, the features of important channels are enhanced, and the unimportant features are suppressed, thereby achieving adaptive allocation of features.

[0107] In this embodiment, the channel attention mechanism can enhance the features of important channels and suppress unimportant features to achieve adaptive allocation of features, which can effectively enhance the response of local channels.

[0108] In some alternative embodiments, the spatial attention module includes:

[0109] A global max pooling layer, which is used to perform global max pooling on the input feature map in the channel dimension to obtain a first sub-feature map;

[0110] A global average pooling layer, which is arranged in parallel with the global max pooling layer, and is used to perform global average pooling on the input feature map in the channel dimension to obtain a second sub-feature map;

[0111] A concatenation layer, which is used to concatenate the first sub-feature map and the second sub-feature map to obtain a third sub-feature map;

[0112] A two-dimensional convolutional layer, which is used to perform a convolutional operation on the third sub-feature map to reduce the channel dimension;

[0113] A second Sigmoid activation function layer, which is connected to the output end of the two-dimensional convolutional layer and is used to generate the spatial attention weights for each channel.

[0114] Regarding the spatial attention module, first, global max pooling and global average pooling can be performed on the input feature map in the channel dimension to obtain two sub-feature maps. Then, the two sub-feature maps are concatenated, and the channel dimension is reduced through a convolutional operation. Finally, the Sigmoid function is used to map the spatial attention weight ω spatial . The specific implementation process can be expressed as follows:

[0115] ω spatial = σ(F Conv2D ) = σ(Conv2D k ([F max ; F avg )).

[0116] In this embodiment, the spatial attention mechanism can focus on the key features at different spatial positions within the same channel, which can effectively enhance the feature responses at different spatial positions.

[0117] In the present invention, the channel attention mechanism and the spatial attention mechanism adopted can adaptively assign weights to different channels and spatial positions of stroke patients, strengthen the local channel and spatial feature responses related to EEG signals. At the same time, the multi-head attention module can capture global dependencies and make up for the limited perception field of CNN. The multi-scale convolutional neural network can enhance the model's tolerance to noise and construct diverse feature extractions. By using a deep learning algorithm that combines multiple attentions and a multi-scale convolutional neural network, effective features can be extracted from the EEG signals of different stroke patients.

[0118] The combination of multiple attentions and a multi-scale convolutional neural network comprehensively considers the problem of individual differences in motor imagery of stroke patients. Through the construction of adaptive weight assignment and diverse feature extraction, it can effectively adapt to the problem of individual differences, improve the decoding accuracy of the motor imagery EEG signals of stroke patients, and thus effectively enhance the effect of rehabilitation training.

[0119] In this embodiment, a decoding device for the motor imagery EEG signals of stroke patients is also provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0120] This embodiment provides a decoding device for the motor imagery EEG signals of stroke patients, as Figure 3 shown, the device includes:

[0121] An acquisition module 201, configured to acquire the original EEG signal;

[0122] A processing module 202, configured to preprocess the original EEG signal;

[0123] A decoding module 203, configured to input the preprocessed original EEG signal into a pre-established feature extraction model for diverse feature extraction to obtain a feature set, where the feature set is used for motor imagery classification; wherein, the feature extraction model includes a channel attention module and a spatial attention module arranged in parallel; wherein, the channel attention module is configured to adaptively assign channel attention weights to each channel in the original EEG signal; the channel attention module is configured to adaptively assign spatial attention weights to each space in the original EEG signal.

[0124] The decoding device for the motor imagery EEG signals of stroke patients in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0125] The further function descriptions of the above-mentioned various modules and units are the same as those in the corresponding above-mentioned embodiments, and will not be elaborated here.

[0126] An embodiment of the present invention further provides a computer device having the above-mentioned Figure 3 motor imagery EEG signal decoding device for stroke patients shown.

[0127] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of a computer device provided by an alternative embodiment of the present invention. As shown in Figure 4 , the computer device includes: one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (such as an array of servers, a set of blade servers, or a multi-processor system). Figure 4 One processor 10 is taken as an example in

[0128] The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above-mentioned hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above-mentioned programmable logic device can be a complex programmable logic device, a field programmable gate array, a general array logic, or any combination thereof.

[0129] Among them, the memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above-mentioned embodiment.

[0130] The memory 20 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device and the like. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 may optionally include a memory remotely provided relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0131] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, a hard disk, or a solid-state drive; the memory 20 may further include a combination of the above types of memory.

[0132] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0133] The embodiments of the present invention further provide a computer-readable storage medium. The methods according to the embodiments of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code originally stored in a remote storage medium or a non-transitory machine-readable storage medium and downloaded through a network and to be stored in a local storage medium, so that the methods described herein can be stored in such software processes on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium may also include a combination of the above types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods shown in the above embodiments are implemented.

[0134] A part of the present invention can be applied as a computer program product, for example, computer program instructions, which, when executed by a computer, can call or provide the methods and / or technical solutions according to the present invention through the operations of the computer. Those skilled in the art should understand that the forms of existence of computer program instructions in a computer-readable medium include but are not limited to source files, executable files, installation package files, etc. Correspondingly, the ways for computer program instructions to be executed by a computer include but are not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0135] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for decoding motor imagery EEG signals of stroke patients, characterized in that, The method includes: Obtaining the original electroencephalogram (EEG) signal; Preprocessing the original EEG signal; Inputting the preprocessed original EEG signal into a pre-established feature extraction model for diverse feature extraction to obtain a feature set for motor imagery classification; wherein, the feature extraction model includes a channel attention module and a spatial attention module arranged in parallel; wherein, the channel attention module is used to adaptively assign channel attention weights to each channel in the original EEG signal; the spatial attention module is used to adaptively assign spatial attention weights to each space in the original EEG signal.

2. The method according to claim 1, characterized in that, The feature extraction model further includes: A multi-scale temporal convolutional module, which includes temporal convolutional layers of different scales arranged in parallel; The input end of the multi-scale temporal convolutional module is used to input the preprocessed original EEG signal, and the output end is used to connect to the channel attention module and the spatial attention module.

3. The method according to claim 1, characterized in that The feature extraction model further includes: A first spatial convolutional layer connected to the output end of the channel attention module; A second spatial convolutional layer connected to the output end of the spatial attention module; both the first spatial convolutional layer and the second spatial convolutional layer are used to extract spatial feature information; A first spatial residual convolutional layer connected to the output end of the first spatial convolutional layer; A second spatial residual convolutional layer connected to the output end of the second spatial convolutional layer; The first spatial residual convolutional layer and the second spatial residual convolutional layer are both arranged in parallel with the channel attention module and the spatial attention module.

4. The method according to claim 3, characterized in that, The feature extraction model further includes: A multi-head attention module arranged at the output ends of the first spatial convolutional layer and the second spatial convolutional layer for capturing global temporal and spatial dependencies.

5. The method according to claim 1, wherein The channel attention module includes: A global average pooling layer for compressing the spatial information of each channel in the original EEG signal to obtain global feature information; A first fully-connected layer for channel compression of the global feature information to obtain reduced-dimensional global feature information; A ReLU activation function layer connected to the first fully-connected layer for introducing non-linearity; A second fully-connected layer connected to the ReLU activation function layer for channel restoration; A first Sigmoid activation function layer connected to the second fully-connected layer for generating the channel attention weights for each channel.

6. The method according to claim 1, characterized in that, The spatial attention module includes: A global max pooling layer for performing global max pooling on the input feature map in the channel dimension to obtain a first sub-feature map; A global average pooling layer arranged in parallel with the global max pooling layer for performing global average pooling on the input feature map in the channel dimension to obtain a second sub-feature map; A concatenation layer for concatenating the first sub-feature map and the second sub-feature map to obtain a third sub-feature map; A two-dimensional convolutional layer for performing a convolutional operation on the third sub-feature map to reduce the channel dimension; The second Sigmoid activation function layer, connected to the output end of the two-dimensional convolutional layer, is used to generate the spatial attention weights for each channel.

7. A device for decoding motor imagery EEG signals of stroke patients, characterized in that, The device includes: An acquisition module, configured to acquire the original EEG signal; A processing module, configured to preprocess the original EEG signal; A decoding module, configured to input the preprocessed original EEG signal into a pre-established feature extraction model for diverse feature extraction to obtain a feature set, where the feature set is used for motor imagery classification; wherein, the feature extraction model includes a channel attention module and a spatial attention module arranged in parallel; wherein, the channel attention module is used to adaptively assign channel attention weights to each channel in the original EEG signal; the channel attention module is used to adaptively assign spatial attention weights to each space in the original EEG signal.

8. A computer device, characterized in that, It includes: A memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the decoding method for the motor imagery EEG signal of a stroke patient according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the decoding method for the motor imagery EEG signal of a stroke patient according to any one of claims 1-6.

10. A computer program product, characterized in that, It includes computer instructions, and the computer instructions are used to cause the computer to execute the decoding method for the motor imagery EEG signal of a stroke patient according to any one of claims 1-6.

Images