Unilateral upper limb motor imagery decoding method based on integrated multi-band attention mechanism
By integrating the multi-band attention mechanism and deep learning network, the problem of low decoding accuracy of fine motion imagination of unilateral upper limbs in traditional methods is solved, and more refined movement distinction and higher decoding accuracy are achieved, which improves the model's lucidity Awesomeness and speed of application.
Patent Information
- Application Number
- CN202411144852.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-12-13
AI Technical Summary
The traditional unilateral upper limb fine motor imagination decoding method lacks obvious understanding of the changes in the EEG signal characteristic distribution across subjects and sessions, resulting in limited model generalization performance and high overlap of EEG signals, resulting in low decoding accuracy.
Using an integrated multi-band attention mechanism method, the unilateral upper limb motion imaginary EEG data is obtained through the wireless EEG signal acquisition system, preprocessing and empowering processing, and features are extracted using deep learning network and transferred learning are established to establish a single-sided Upper limb motion imagination decoding model.
It improves the fine distinction ability of unilateral upper limb motion imagination movements, enhances the accuracy of EEG signal decoding, reduces model training time, and improves the robustness of the model and application deployment speed.
Smart Images
Figure CN119128677A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of brain-computer interface technology, and in particular to a unilateral upper limb motor imagery decoding method based on an integrated multi-band attention mechanism. Background Art
[0002] Brain-Computer Interface (BCI) is the abbreviation of brain-computer interface, which is a computer-based system. It records the electroencephalogram (EEG) or other physiological electrical signals on the scalp and decodes them into computer-readable commands to control a single or a series of output devices, providing a way for people who cannot express their ideas or operate devices through language or body movements to communicate with the external environment, thereby improving the quality of life of patients. With the development of BCI technology, its successful application in assisting the rehabilitation of patients with motor dysfunction has attracted widespread attention from researchers. At present, BCI has been widely used in computational neuroscience, physiology, engineering, signal processing, computer science and several interdisciplinary research types.
[0003] Motor imagery (MI) can be considered as a method to access the motor system and rehabilitation at all stages of stroke recovery. Studies have shown that MI can induce plastic changes in the basic neural mechanisms of motor learning in the injured hemisphere. In particular, MI tasks activate sensorimotor areas that are active during actual motor execution and induce a significant increase in cerebral blood flow velocity associated with neural activation in MI tasks. Therefore, the EEG-based MI-BCI system has become a promising tool for motor rehabilitation of stroke patients. Traditional MI includes four movements of the left hand, right hand, feet and tongue, which cannot meet the requirements of upper limb rehabilitation well. Therefore, the use of fine motor imagery (FMI) is imperative. FMI tasks are different movements of unilateral limbs. Compared with traditional MI, the FMI-BCI system is more in line with the needs of upper limb rehabilitation.
[0004] At present, there are still some shortcomings in the research on fine motor imagery decoding of unilateral upper limbs. Due to psychological or physiological reasons, each person's EEG signals will be very different, and individual differences are more obvious under task conditions. Although there are some methods that can make up for this difference between subjects to a certain extent, there is still a lack of obvious understanding of the changes in the distribution of EEG signal features across subjects and sessions, which leads to limited generalization performance of the model and huge challenges in practical applications. At the same time, due to the high overlap of FMI's EEG signals, it is difficult for the decoding model to parse the correct instructions. Summary of the invention
[0005] In view of this, the present invention provides a unilateral upper limb motor imagery decoding method based on an integrated multi-band attention mechanism to solve the above problems.
[0006] The present invention provides a unilateral upper limb motor imagery decoding method based on an integrated multi-band attention mechanism, comprising: performing data acquisition through a wireless EEG signal acquisition system to obtain unilateral upper limb task motor imagery EEG data; preprocessing the unilateral upper limb task motor imagery EEG data to obtain multi-band EEG data; weighting the multi-band EEG data through an attention mechanism module; extracting features from the weighted multi-band EEG data using a deep learning network to obtain EEG data features; splicing and fusing the weighted EEG data features and performing classification processing; performing transfer learning based on the classified EEG data features to establish a unilateral upper limb motor imagery decoding model; and decoding the unilateral upper limb motor imagery EEG data to be processed through the unilateral upper limb motor imagery decoding model.
[0007] In another implementation of the present invention, the four unilateral upper limb movements contained in the unilateral upper limb task motor imagery EEG data include: imagining keeping the upper arm still and flexing and extending the elbow joint; imagining keeping the arm straight and rotating the shoulder joint; imagining keeping the wrist joint still and making a fist; imagining keeping a fist and rotating the wrist joint.
[0008] In another implementation of the present invention, the unilateral upper limb task motor imagery EEG data is preprocessed to obtain multi-band EEG data, including: the preprocessing includes filtering and frequency division, interpolation and decomposition, re-referencing, independent component analysis, segmentation and data enhancement; the unilateral upper limb task motor imagery EEG data is filtered and frequency divided to obtain six frequency bands of 1-4Hz, 4-8Hz, 8-13Hz, 13-31Hz, 31-40Hz, and 1-40Hz; the EEG data of each frequency band is interpolated and decomposition, re-referencing, independent component analysis, segmentation and data enhancement to obtain multi-band EEG data; wherein, the independent component analysis is used to remove artifact signals such as electrooculogram, electromyography and electrocardiogram.
[0009] In another implementation of the present invention, the characteristic matrix of the multi-band EEG data obtained by the weighting process is expressed as:
[0010]
[0011] in, represents the weighted feature matrix of the nth channel; s n Indicates the weight of each frequency band; u n represents EEG data; F scale Represents the empowerment process.
[0012] In another implementation of the present invention, the feature matrix of the EEG data features obtained by the feature extraction is expressed as:
[0013]
[0014] in, Represents the characteristic matrix of the nth frequency band; Indicates the characteristic data of the nth frequency band; F feature Represents the feature extraction process.
[0015] In another implementation of the present invention, the features of the spliced and fused EEG data are expressed as:
[0016]
[0017] in, Represents the feature matrix of the Nth frequency band.
[0018] In another implementation of the present invention, the cross entropy loss function of the unilateral upper limb motor imagery decoding model is expressed as:
[0019]
[0020] Wherein, N represents the number of frequency bands.
[0021] Another aspect of the present invention provides a unilateral upper limb motor imagery decoding system based on an integrated multi-band attention mechanism, comprising: a data acquisition module: performing data acquisition through a wireless EEG signal acquisition system to obtain unilateral upper limb task motor imagery EEG data; a data processing module: preprocessing the unilateral upper limb task motor imagery EEG data to obtain multi-band EEG data; an attention mechanism module: performing weighting processing on the multi-band EEG data through the attention mechanism module; a feature extraction module: performing feature extraction on the weighted multi-band EEG data using a deep learning network to obtain EEG data features; a feature fusion classification module: splicing and fusing the weighted EEG data features, and performing classification processing; a model establishment module: performing transfer learning based on the classified EEG data features to establish a unilateral upper limb motor imagery decoding model; a decoding module: decoding and processing the unilateral upper limb motor imagery EEG data to be processed through the unilateral upper limb motor imagery decoding model.
[0022] Another aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned unilateral upper limb motor imagery decoding method based on an integrated multi-band attention mechanism are implemented.
[0023] Another aspect of the present invention provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-mentioned unilateral upper limb motor imagery decoding method based on an integrated multi-band attention mechanism.
[0024] The unilateral upper limb motor imagery decoding method based on the integrated multi-band attention mechanism of the present invention designs an FMI paradigm for different joints for the unilateral limb. Compared with the traditional MI, it can more finely distinguish the motor imagery actions of different joints, and is more in line with the actual BCI needs; the unilateral upper limb motor imagery decoding based on the integrated multi-band attention mechanism can effectively combine the EEG signal characteristics in the frequency domain, time domain and spatial domain, and can obtain a more accurate decoding effect for the FMI EEG; the transfer learning method is used to effectively establish a cross-subject model, reduce the training time of the new model, and accelerate the convergence speed of the model, so as to achieve a faster application deployment speed in practical applications; it solves the problem that the traditional motor imagery actions cannot distinguish the unilateral limbs and the FMI recognition accuracy is low, while reducing the decoding model training time and improving the robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. By reading the detailed description of the following implementations, the advantages and benefits of the solutions become clear to those skilled in the art. The drawings are only used to illustrate the preferred implementations and are not to be considered as limiting the present invention.
[0026] In the attached picture:
[0027] Figure 1 The present invention is a flowchart of a unilateral upper limb motor imagery decoding method based on an integrated multi-band attention mechanism according to an embodiment of the present invention.
[0028] Figure 2 The figure is a schematic diagram of the experimental paradigm flow of one embodiment of the present invention.
[0029] Figure 3 A schematic diagram of a specific FMI paradigm stimulation according to an embodiment of the present invention.
[0030] Figure 4 A schematic diagram of a decoding model system framework according to an embodiment of the present invention. DETAILED DESCRIPTION
[0031] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and in detail below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.
[0032] Figure 1 A flow chart of a unilateral upper limb motor imagery decoding method based on an integrated multi-band attention mechanism provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, this embodiment mainly includes:
[0033] S101. Collect data through a wireless EEG signal acquisition system to obtain unilateral upper limb task motor imagery EEG data.
[0034] S102, preprocessing the EEG data of unilateral upper limb task motor imagery to obtain multi-band EEG data.
[0035] S103. Perform weighted processing on multi-band EEG data through an attention mechanism module.
[0036] S104. Using a deep learning network to extract features from the weighted multi-band EEG data to obtain EEG data features.
[0037] S105, concatenating and fusing the weighted EEG data features, and performing classification processing.
[0038] S106. Perform transfer learning based on the classified EEG data features to establish a unilateral upper limb motor imagery decoding model.
[0039] S107, decoding the unilateral upper limb motor imagery EEG data to be processed by using a unilateral upper limb motor imagery decoding model.
[0040] The unilateral upper limb motor imagery decoding method based on the integrated multi-band attention mechanism of the present invention designs an FMI paradigm for different joints for the unilateral limb. Compared with the traditional MI, it can more finely distinguish the motor imagery actions of different joints, and is more in line with the actual BCI needs; the unilateral upper limb motor imagery decoding based on the integrated multi-band attention mechanism can effectively combine the EEG signal characteristics in the frequency domain, time domain and spatial domain, and can obtain a more accurate decoding effect for the FMI EEG; the transfer learning method is used to effectively establish a cross-subject model, reduce the training time of the new model, and accelerate the convergence speed of the model, so as to achieve a faster application deployment speed in practical applications; it solves the problem that the traditional motor imagery actions cannot distinguish the unilateral limbs and the FMI recognition accuracy is low, while reducing the decoding model training time and improving the robustness of the model.
[0041] In another implementation of the present invention, the four unilateral upper limb movements included in the unilateral upper limb task motor imagery EEG data include: imagining keeping the upper arm still and flexing and extending the elbow joint; imagining keeping the arm straight and rotating the shoulder joint; imagining keeping the wrist joint still and making a fist; imagining keeping the fist and rotating the wrist joint.
[0042] For example, Figure 2 As shown in the figure, before the experiment, the subjects were asked to sit comfortably on the chair, put their hands naturally on the armrests of the chair, relax their whole body, keep their attention during the experiment, avoid unnecessary limb movements, and try to reduce the number of blinks. Before the motor imagery experiment, there will be a short training to help the subjects familiarize themselves with the overall experimental process. After confirming with the subjects that the experiment can begin, the motor imagery experiment will officially begin.
[0043] A complete experiment consists of six groups, with a certain rest time between each group, and each group contains 30 trials. At the beginning of each experiment, a cross will appear in the center of the screen for 1 second to remind the subjects that the motor imagery task is about to begin. After the cross disappears, a specific motor imagery prompt picture will appear.
[0044] like Figure 3 As shown in the figure, the motor imagery prompt pictures include imagining keeping the upper arm still and flexing and extending the elbow; imagining keeping the arm straight and rotating the shoulder joint; imagining keeping the wrist joint still and making a fist; imagining keeping the fist and rotating the wrist joint. Each picture lasts for 4 seconds, followed by a short rest of 2 seconds before the next round of trials. The subjects were required to repeat the motor imagery of the action shown in the stimulus picture at a frequency of about 1.5 Hz within 4 seconds of the picture stimulation. Each subject conducted 3 experiments, for a total of 540 motor imagery.
[0045] During the experiment, the subjects' EEG data were collected throughout the experiment. The data collection equipment can use the NeuracleNeuSen W series wireless EEG signal acquisition system, with 59 effective leads, and the electrodes are placed according to the international standard 10-20 system. The sampling rate is set to 1000Hz. During the acquisition process, the impedance is kept below 10kΩ, and the EEG experiment is conducted in a silent environment. The motor imagery data of healthy people and stroke patients were collected for model training and verification, which proved the real usability of this classification model and laid the foundation for the subsequent development of BCI.
[0046] In another implementation of the present invention, the EEG data of unilateral upper limb task motor imagery are preprocessed to obtain multi-band EEG data, including: preprocessing includes filtering and frequency division, interpolation and decomposition, re-referencing, independent component analysis, segmentation and data enhancement; the EEG data of unilateral upper limb task motor imagery are filtered and frequency divided to obtain six frequency bands of 1-4Hz, 4-8Hz, 8-13Hz, 13-31Hz, 31-40Hz, and 1-40Hz; the EEG data of each frequency band are interpolated and decomposition, re-referencing, independent component analysis, segmentation and data enhancement to obtain multi-band EEG data; wherein, the independent component analysis is used to remove artifact signals such as electrooculogram, electromyography and electrocardiogram.
[0047] Exemplarily, the preprocessing process includes filtering and frequency division, interpolation and derivation, re-referencing, independent component analysis, segmentation, data enhancement and other operations.
[0048] Specifically, the collected EEG data is filtered and data of different frequency bands are distinguished. An eighth-order Butterworth digital filter can be used, and an SOS type representation is used to ensure the stability of the filter.
[0049] First, a 50Hz notch filter is used to remove the power frequency interference in the EEG signal, and then the EEG is divided into six frequency bands: 1-4Hz, 4-8Hz, 8-13Hz, 13-31Hz, 31-40Hz, and 1-40Hz.
[0050] Interpolate bad conductors and remove a bad conductor. If there is no bad conductor in the data, skip this step.
[0051] Whole-brain average re-referencing was used to eliminate effects due to specific reference electrodes.
[0052] Independent component analysis (ICA) was used to decompose the various EEG components, and artificial signals such as electrooculography, electromyography, and electrocardiography were manually inspected and removed.
[0053] According to the marks of the experimental paradigm, the data were segmented from -1 to 4 seconds, and the data from -1 to 0 seconds were used for baseline correction.
[0054] Using downsampling, the 1000Hz signal is reduced to 250Hz, and the data set is expanded by 4 times.
[0055] In another implementation of the present invention, the feature matrix of the multi-band EEG data obtained by weighting processing is expressed as:
[0056]
[0057] in, represents the weighted feature matrix of the nth channel; s n Indicates the weight of each frequency band; u n represents EEG data; F scale Represents the empowerment process.
[0058] For example, Figure 4 As shown in the figure, the frequency band attention network structure uses the SE network, which focuses on the relationship between channels and can well extract the data correlation characteristics between frequency bands. Therefore, the frequency band is used as a channel and input into the SE network.
[0059] The SE network first compresses the global spatial information into the channel through global average pooling, which is expressed as follows:
[0060]
[0061] Among them, z n Represents the overall information of the nth channel, u n Representing the two-dimensional EEG signal of the nth channel, it can be expressed as
[0062] The compressed channel information is adaptively activated to obtain the weight matrix s, which can be expressed as follows:
[0063] s=F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z))
[0064] Among them, δ is the activation function, W1 and W2 are fully connected matrices, r is a hyperparameter used to control feature dimensionality reduction.
[0065] Multiply the original EEG data by the weight s to get the feature matrix It can be expressed as follows:
[0066]
[0067] in, represents the weighted feature matrix of the nth channel, s n Represents the weight of the nth channel.
[0068] The weight network structure of each frequency band is shown in Table 1.
[0069] Table 1 Frequency band weight calculation attention network parameters
[0070]
[0071]
[0072] In another implementation of the present invention, the feature matrix of the EEG data features obtained by feature extraction is expressed as:
[0073]
[0074] in, Represents the characteristic matrix of the nth frequency band; Indicates the characteristic data of the nth frequency band; F feature Represents the feature extraction process.
[0075] For example, in order to further extract features from the data of each frequency band, After dimension reorganization, it can be expressed as
[0076] The deep separable convolutional structure of EEGNet is used to effectively and quickly extract feature information from EEG signals. The network structure parameters are shown in Table 2.
[0077] Table 2 Feature extraction network parameter structure
[0078]
[0079]
[0080] After feature extraction, the feature matrix of the nth frequency band can be expressed as As shown below:
[0081]
[0082] Among them, F feature represents the feature extraction process, Represents the characteristic data of the nth frequency band.
[0083] In another implementation of the present invention, the features of the spliced and fused EEG data are expressed as:
[0084]
[0085] in, Represents the feature matrix of the Nth frequency band.
[0086] Exemplarily, concatenate the features of all bands:
[0087]
[0088] The final feature weighting is performed through a multi-head self-attention mechanism module, and its output is used as the final feature input classifier for classification.
[0089] After the data passes through all feature extraction modules, the features are obtained. Regarding classification, after flattening all features, the network passes through the fully connected layer and the normalized exponential function (Softmax) layer to output the final classification result. The network structure is shown in Table 3.
[0090] Table 3 Classification module network structure
[0091]
[0092] It should be understood that since data collection for hemiplegic patients is more difficult than for healthy subjects and cannot be collected on a large scale for a long time, the patient's data set is much smaller than that of healthy subjects, and transfer learning can play a role in this case. The FMI-EEG data of healthy subjects is used as the source domain data, and the FMI-EEG data of hemiplegic patients is used as the target domain data to complete the training of the model. Using the original EEG as the model input, an end-to-end FMI decoding system is designed, which reduces manual intervention and greatly enhances the usability of the model. In another implementation of the present invention, the cross entropy loss function of the unilateral upper limb motor imagery decoding model is expressed as:
[0093]
[0094] Wherein, N represents the number of frequency bands.
[0095] Exemplarily, the optimizer uses Adam for back-propagation to update network parameters, the learning rate is set to 0.001, and the data segmentation uses five-fold cross validation.
[0096] The model was first iteratively trained 100 times on the source domain data to stabilize the accuracy of the classifier, and the model with the lowest loss value on the validation set was selected as the pre-trained model. Secondly, this model was applied to the FMI-EEG data of each patient. After the patient data was split into data sets, the pre-trained model was fine-tuned and the classification accuracy of the model was verified on the validation set.
[0097] With the help of professional rehabilitation doctors in a rehabilitation hospital, motor imagery EEG data of several stroke patients were collected for model verification. The data of all healthy subjects were integrated to establish a cross-subject model, train a unified motor imagery decoding model, and then fine-tune the model on the motor imagery EEG data of real patients through transfer learning to speed up the model convergence, reduce the decoding model training time for new subjects and reduce the number of patient data sets required, increase the robustness of the decoding model, and obtain more stable and accurate decoding results, so that it can be applied to new brain-computer interface systems. Another aspect of the present invention provides a unilateral upper limb motor imagery decoding system based on an integrated multi-band attention mechanism, comprising:
[0098] Data acquisition module: Data is collected through a wireless EEG signal acquisition system to obtain unilateral upper limb task motor imagery EEG data.
[0099] Data processing module: pre-process the EEG data of unilateral upper limb task motor imagery to obtain multi-band EEG data.
[0100] Attention mechanism module: The attention mechanism module is used to perform weighted processing on multi-band EEG data.
[0101] Feature extraction module: Use the deep learning network to extract features from the weighted multi-band EEG data to obtain EEG data features.
[0102] Feature fusion classification module: concatenates and fuses the weighted EEG data features and performs classification processing.
[0103] Model building module: transfer learning is performed based on the classified EEG data features to establish a unilateral upper limb motor imagery decoding model.
[0104] Decoding module: The unilateral upper limb motor imagery EEG data to be processed is decoded through the unilateral upper limb motor imagery decoding model.
[0105] The unilateral upper limb motor imagery decoding system based on the integrated multi-band attention mechanism of the present invention designs an FMI paradigm for different joints for unilateral limbs. Compared with traditional MI, it can more finely distinguish the motor imagery actions of different joints, and is more in line with the actual BCI needs; the unilateral upper limb motor imagery decoding based on the integrated multi-band attention mechanism can effectively combine the EEG signal characteristics in the frequency domain, time domain and spatial domain, and can obtain a more accurate decoding effect for the FMI EEG; the transfer learning method is used to effectively establish a cross-subject model, reduce the training time of the new model, and accelerate the convergence speed of the model, so as to achieve a faster application deployment speed in practical applications; it solves the problem that traditional motor imagery actions cannot distinguish unilateral limbs and the FMI recognition accuracy is low, while reducing the decoding model training time and improving the robustness of the model.
[0106] In another aspect of the present invention, an electronic device includes: a processor, a memory, a communication bus, and a communication interface.
[0107] in:
[0108] The processor, memory and communication interface communicate with each other through a communication bus.
[0109] Communication interface, used to communicate with other electronic devices or servers.
[0110] The processor is used to execute the program, and specifically can execute the steps of any one of the above-mentioned embodiments of the unilateral upper limb motor imagery decoding method based on the integrated multi-band attention mechanism.
[0111] Specifically, the program may include program codes including computer operation instructions.
[0112] The processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.
[0113] The memory is used to store programs. The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.
[0114] The program can be specifically used to enable the processor to execute to implement any of the steps of the unilateral upper limb motor imagery decoding method based on the integrated multi-band attention mechanism described in the embodiment. The specific implementation of each step in the program can refer to the corresponding description in the steps and units executed by any of the unilateral upper limb motor imagery decoding methods based on the integrated multi-band attention mechanism in the above steps, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described equipment and modules can refer to the corresponding process description in the aforementioned method embodiment.
[0115] The exemplary embodiments of the present application also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the methods of the various embodiments of the present application.
[0116] The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code that may be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded over a network and will be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, processor, or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.
[0117] Thus far, specific embodiments of the present invention have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired results. Additionally, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing may be advantageous.
[0118] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0119] In the description of the present invention, the terms "first" and "second" are only used to facilitate the description of different components or names, and cannot be understood as indicating or implying a sequential relationship, relative importance, or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.
[0120] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.
[0121] It should be noted that although the specific embodiments of the present invention are described in detail in conjunction with the accompanying drawings, it should not be understood as limiting the scope of protection of the present invention. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still belong to the scope of protection of the present invention.
[0122] The examples of the embodiments of the present invention are intended to concisely illustrate the technical features of the embodiments of the present invention so that those skilled in the art can intuitively understand the technical features of the embodiments of the present invention, and are not intended to be improper limitations of the embodiments of the present invention.
[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A unilateral upper limb motor imagery decoding method based on an integrated multi-band attention mechanism, characterized in that: include: The wireless EEG signal acquisition system is used to collect data and obtain the EEG data of motor imagery of unilateral upper limb tasks; Preprocessing the unilateral upper limb task motor imagery EEG data to obtain multi-band EEG data; Performing weighted processing on the multi-band EEG data through an attention mechanism module; The deep learning network is used to extract features from weighted multi-band EEG data to obtain EEG data features; The weighted EEG data features are spliced and fused, and then classified; Based on the classified EEG data features, transfer learning was performed to establish a unilateral upper limb motor imagery decoding model. The unilateral upper limb motor imagery EEG data to be processed is decoded by the unilateral upper limb motor imagery decoding model.
2. The method according to claim 1, characterized in that The four types of unilateral upper limb movements included in the unilateral upper limb task motor imagery EEG data include: Imagine keeping your upper arm still and your elbow flexed and extended; Imagine keeping your arms straight and your shoulders rotated; Imagine keeping your wrist still and making a fist; Imagine keeping your fist clenched and rotating your wrist.
3. The method according to claim 1, characterized in that The preprocessing of the unilateral upper limb task motor imagery EEG data to obtain multi-band EEG data includes: The preprocessing includes filtering and frequency division, interpolation and derivation, re-referencing, independent component analysis, segmentation and data enhancement; The unilateral upper limb task motor imagery EEG data is filtered and processed to obtain six frequency bands: 1-4 Hz, 4-8 Hz, 8-13 Hz, 13-31 Hz, 31-40 Hz, and 1-40 Hz; For each frequency band of EEG data, interpolation, re-reference, independent component analysis, segmentation and data enhancement are performed to obtain multi-band EEG data; Among them, independent component analysis processing is used to remove artifact signals such as electrooculography, electromyography and electrocardiography.
4. The method according to claim 3, characterized in that The characteristic matrix of the multi-band EEG data obtained by the weighted processing is expressed as: in, represents the weighted feature matrix of the nth channel; s n Indicates the weight of each frequency band; u n represents EEG data; F scale Represents the empowerment process.
5. The method according to claim 4, characterized in that The feature matrix of the EEG data features obtained by the feature extraction is expressed as: in, Represents the characteristic matrix of the nth frequency band; Indicates the characteristic data of the nth frequency band; F feature Represents the feature extraction process.
6. The method according to claim 5, characterized in that The characteristics of the EEG data after splicing and fusion are expressed as: in, Represents the feature matrix of the Nth frequency band.
7. The method according to claim 6, characterized in that The cross entropy loss function of the unilateral upper limb motor imagery decoding model is expressed as: Wherein, N represents the number of frequency bands.
8. A unilateral upper limb motor imagery decoding system based on an integrated multi-band attention mechanism, characterized in that: include: Data acquisition module: Data acquisition is performed through a wireless EEG signal acquisition system to obtain EEG data of motor imagery of unilateral upper limb tasks; Data processing module: pre-processing the unilateral upper limb task motor imagery EEG data to obtain multi-band EEG data; Attention mechanism module: weighting the multi-band EEG data through the attention mechanism module; Feature extraction module: Use the deep learning network to extract features from weighted multi-band EEG data to obtain EEG data features; Feature fusion classification module: splice and fuse the weighted EEG data features and perform classification processing; Model building module: transfer learning is performed based on the classified EEG data features to establish a unilateral upper limb motor imagery decoding model; Decoding module: Decoding the unilateral upper limb motor imagery EEG data to be processed through the unilateral upper limb motor imagery decoding model.
9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the unilateral upper limb motor imagery decoding method based on an integrated multi-band attention mechanism as described in any one of claims 1 to 7 are implemented.
10. A computer storage medium, characterized in that: The computer storage medium stores a computer program, which, when executed by a processor, implements the steps in the unilateral upper limb motor imagery decoding method based on an integrated multi-band attention mechanism as described in any one of claims 1 to 7.