Unilateral upper limb rehabilitation training method and system based on brain-computer interface
Through the unilateral upper limb rehabilitation training method based on brain-computer interface, using neural network and socket technology, the problem of low comfort and efficiency of exoskeleton rehabilitation training system is solved, and the motor recovery ability of stroke patients is improved.
Patent Information
- Application Number
- CN202411208050.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-24
AI Technical Summary
The existing exoskeleton rehabilitation training system has shortcomings in terms of comfort and rehabilitation efficiency, and it is difficult to effectively improve the unilateral upper limb motor recovery ability of stroke patients.
Using a one-sided upper limb rehabilitation training method based on brain-computer interface, collect and preprocess the electroencephalogram data of motor imagination, use neural networks to classify, output control instructions to control exoskeleton rehabilitation training tasks, and combine the electroencephalogram classification module and attention module. It improves the accuracy of action recognition and realizes online data interaction through socket technology.
It improves the accuracy of motion recognition of the electroencephalogram data of sports imagination, improves the comfort and efficiency of rehabilitation training, and enhances the patient's independent exercise ability.
Smart Images
Figure CN119181469A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of electroencephalogram signal processing, and in particular to a unilateral upper limb rehabilitation training method and system based on a brain-computer interface. Background Art
[0002] Currently, one of the most important symptoms after stroke is unilateral motor dysfunction, of which about 85% of stroke patients have motor dysfunction, which brings great inconvenience to patients' daily lives. Therefore, effective treatment equipment is an extremely important task to improve patients' motor recovery ability.
[0003] Motor imagery is actually the brain's intention to move without the limbs performing the movement. Motor imagery is used to activate and repair the patient's damaged brain neural network, thereby improving the patient's autonomous movement ability. Since motor imagery is not a real movement, but is closely related to movement, the unilateral upper limb exoskeleton rehabilitation training system based on motor imagery can effectively improve the rehabilitation effect. However, the exoskeleton rehabilitation training system in the prior art has low comfort and poor rehabilitation efficiency. Summary of the invention
[0004] In view of this, the present invention provides a unilateral upper limb rehabilitation training method and system based on brain-computer interface to solve the above problems.
[0005] The present invention provides a unilateral upper limb rehabilitation training method based on a brain-computer interface, comprising: collecting motor imagery EEG data of upper limb shoulder rotation, elbow flexion, and wrist rotation; preprocessing the motor imagery EEG data; training and classifying the preprocessed motor imagery EEG data through a constructed neural network to obtain classification results; outputting control instructions based on the classification results; and using the control instructions to control an exoskeleton to specify a rehabilitation training task.
[0006] In another implementation of the present invention, the preprocessing includes basic filtering, ICA artifact removal, multi-band division, segmentation and baseline correction, and manual inspection.
[0007] In another implementation of the present invention, the neural network is composed of an EEG classification module, an attention module, a feature fusion module, and a feature classification module; the training and classification of the pre-processed motor imagery EEG data by the neural network to obtain a classification result includes: extracting the time-frequency domain and spatial features of the pre-processed motor imagery EEG data by the EEG classification module; selecting the high-weighted part of the spatial features by the attention module; reorganizing the extracted features according to the high-weighted part of the spatial features by the feature fusion module to obtain a comprehensive feature; and classifying the comprehensive features using the feature classification module to obtain a classification result.
[0008] In another implementation of the present invention, selecting the high-weighted part of the spatial feature by the attention module includes: compressing the spatial feature into a vector by global average pooling, and calculating as follows:
[0009]
[0010] Among them, Z c is the global feature, H and W are the height and width of the feature map respectively.
[0011] Use a fully connected layer to map the compressed vector to a low-dimensional space and activate it through a nonlinear activation function; map it back to the original dimension through another fully connected layer, and learn the importance weight of each channel, which is calculated as follows:
[0012] s=σ(W2·δ(W1·z))
[0013] Among them, δ is the ReLU activation function, σ is the Sigmoid activation function, and W1 and W2 are the weights of the fully connected layer.
[0014] The original spatial features are weighted by channel, and the features of each channel are recalibrated as follows:
[0015]
[0016] Among them, s c is the weight of each channel obtained through the weighted operation process, is the adjusted output.
[0017] Another aspect of the present invention provides a unilateral upper limb rehabilitation training system based on a brain-computer interface, including: an EEG signal acquisition module: used to collect EEG data of motor imagery of upper limb shoulder rotation, elbow flexion, and wrist rotation; an online data processing module: used to preprocess the motor imagery EEG data; train and classify the preprocessed motor imagery EEG data through a constructed neural network to obtain classification results; output control instructions based on the classification results; and an exoskeleton control module: uses the control instructions to control the exoskeleton to specify rehabilitation training tasks.
[0018] Another aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned brain-computer interface-based unilateral upper limb rehabilitation training method are implemented.
[0019] Another aspect of the present invention provides a computer storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the above-mentioned brain-computer interface-based unilateral upper limb rehabilitation training method are implemented.
[0020] The unilateral upper limb rehabilitation training method based on brain-computer interface of the present invention adopts the fusion technology of EEG classification module and attention module in the data processing part, which improves the action recognition accuracy of motor imagery EEG data; and uses socket technology to realize the data interaction between online EEG signal feature analysis and upper limb rehabilitation exoskeleton; while ensuring the convenience and comfort of the trainee, the rehabilitation efficiency of the patient is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. By reading the detailed description of the following implementations, the advantages and benefits of the solutions become clear to those skilled in the art. The drawings are only used to illustrate the preferred implementations and are not to be considered as limiting the present invention.
[0022] In the attached picture:
[0023] Figure 1 This is a flow chart of a unilateral upper limb rehabilitation training method based on brain-computer interface according to an embodiment of the present invention.
[0024] Figure 2 This is a schematic diagram of the TCP / IP interaction process between the online EEG feature analysis model and the exoskeleton control module according to an embodiment of the present invention.
[0025] Figure 3 It is a schematic diagram of joint rehabilitation actions involved in one embodiment of the present invention.
[0026] Figure 4 A schematic diagram of an exoskeleton model designed for an embodiment of the present invention.
[0027] Figure 5 A schematic diagram of a structure capable of adjusting the telescopic length designed for an embodiment of the present invention.
[0028] Figure 6 This is a schematic diagram of a unilateral upper limb rehabilitation training system based on brain-computer interface according to an embodiment of the present invention. DETAILED DESCRIPTION
[0029] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and in detail below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.
[0030] Figure 1 A flow chart of a unilateral upper limb rehabilitation training method based on a brain-computer interface provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, this embodiment mainly includes:
[0031] S101. Collect EEG data of motor imagery of upper limb shoulder rotation, elbow flexion, and wrist rotation.
[0032] For example, the NeuSen W series wireless EEG acquisition system can be used for data acquisition. The wireless EEG acquisition system uses wet electrodes to collect EEG data, reduces impedance through conductive paste, and greatly improves the quality of data. It follows the international 10-20 standard, and the sampling rate can be set to 1000Hz. During the acquisition process, the impedance is reduced to less than 10kΩ. Figure 3 As shown, the EEG data of motor imagery of upper limb shoulder rotation, elbow flexion, and wrist rotation were collected.
[0033] S102, pre-processing the motor imagery EEG data.
[0034] S103, training and classifying the pre-processed motor imagery EEG data through the constructed neural network to obtain classification results.
[0035] S104: Output a control instruction based on the classification result.
[0036] S015. Use control instructions to control the exoskeleton to specify rehabilitation training tasks.
[0037] The unilateral upper limb rehabilitation training method based on brain-computer interface of the present invention adopts the fusion technology of EEG classification module and attention module in the data processing part, which improves the action recognition accuracy of motor imagery EEG data; and uses socket technology to realize the data interaction between online EEG signal feature analysis and upper limb rehabilitation exoskeleton; while ensuring the convenience and comfort of the trainee, the rehabilitation efficiency of the patient is improved.
[0038] In another implementation of the present invention, the preprocessing includes basic filtering, ICA artifact removal, multi-band division, segmentation and baseline correction, and manual inspection.
[0039] For example, according to the characteristics of the data, the EEG frequency bands of movement phenomena are mainly concentrated in the Delta, Theta, Alpha, Beta, and Gamma frequency bands, and 0.5-45 Hz is selected as the basic filtering frequency band to filter out the DC component and some ultra-low frequency noise and 50 Hz power frequency interference in the original EEG signal.
[0040] Use the ICA independent component analysis algorithm to segment multiple signals into independent components.
[0041] Subsequent EEG signal processing will use multi-frequency band data, and after the ICA processing is completed, the frequency bands will be further divided.
[0042] The data set was segmented according to the marking signals during the task and the specific tasks performed. The motor imagery execution time was 4s. The 1s before marking and the 4s after marking were retained as a signal segment. The EEG data before marking was used as the baseline for correction to reduce local EEG signal drift.
[0043] The final processed data was manually checked in the time domain to see if there were any obvious artifacts.
[0044] In another implementation of the present invention, the neural network is composed of an EEG classification module, an attention module, a feature fusion module, and a feature classification module; the training and classification of the pre-processed motor imagery EEG data by the neural network to obtain a classification result includes: extracting the time-frequency domain and spatial features of the pre-processed motor imagery EEG data by the EEG classification module; selecting the high-weighted part of the spatial features by the attention module; reorganizing the extracted features according to the high-weighted part of the spatial features by the feature fusion module to obtain a comprehensive feature; and classifying the comprehensive features using the feature classification module to obtain a classification result.
[0045] Exemplarily, by integrating the attention module (Squeeze-and-Excitation SE) with the EEG classification module (EEGNet), the EEGNet module is used to extract time-frequency domain and spatial features from the collected EEG data, the SE module is used to select the high-weighted parts of the features, and then through the feature fusion module, the extracted features are recombined to obtain comprehensive features, and finally the classifier is used to classify the comprehensive features to obtain the classification results.
[0046] The EEGNet module consists of: temporal convolution layer, depthwise separable convolution layer, first layer Dropout, first layer average pooling, separable convolution layer, second layer average pooling, and second layer Dropout. The temporal convolution layer is used to extract frequency features, the depthwise separable convolution layer is used to capture spatial features, and the average pooling layer is used to reduce feature dimensions.
[0047] In another implementation of the present invention, selecting the high-weighted part of the spatial feature by the attention module includes: compressing the spatial feature into a vector by global average pooling, and calculating as follows:
[0048]
[0049] Among them, Z cis the global feature, H and W are the height and width of the feature map respectively.
[0050] Use a fully connected layer to map the compressed vector to a low-dimensional space and activate it through a nonlinear activation function; map it back to the original dimension through another fully connected layer, and learn the importance weight of each channel, which is calculated as follows:
[0051] s=σ(W2·δ(W1·z))
[0052] Among them, δ is the ReLU activation function, σ is the Sigmoid activation function, and W1 and W2 are the weights of the fully connected layer.
[0053] The original spatial features are weighted by channel, and the features of each channel are recalibrated as follows:
[0054]
[0055] Among them, s c is the weight of each channel obtained through the weighted operation process, is the adjusted output.
[0056] Exemplarily, the SE module is used to improve the feature representation capability by learning the importance of each channel and weighting different feature channels to obtain valuable features and suppress unimportant features. The SE module consists of three steps: Squeeze, Excitation, and Recalibration.
[0057] Finally, after flattening all the features, they pass through the fully connected layer and the normalized exponential function (Softmax) layer to output the final classification result.
[0058] In another implementation of the present invention, the specific operation process is as follows:
[0059] (1) The subjects wore an EEG cap on their heads and an exoskeleton on their upper limbs. The exoskeleton model is as follows: Figure 4 shown.
[0060] (2) The experiment begins. The EEG cap collects real-time EEG data of the subjects during the experiment, transmits the data to the online data processing module, and uses network training to develop a feature classification model.
[0061] (3) TCP / IP protocol transmits the classification results to the exoskeleton control module.
[0062] (4) The exoskeleton control module makes corresponding joint movements based on the classification results to achieve the effect of rehabilitation training.
[0063] Another aspect of the present invention, as Figure 6 As shown, a unilateral upper limb rehabilitation training system based on a brain-computer interface is provided, comprising:
[0064] EEG signal acquisition module: used to collect EEG data of motor imagery of upper limb shoulder rotation, elbow flexion and wrist rotation.
[0065] Online data processing module: used for preprocessing the motor imagery EEG data; training and classifying the preprocessed motor imagery EEG data through the constructed neural network to obtain classification results.
[0066] For example, Figure 2 As shown in the figure, the online data processing module acts as the server, and the exoskeleton control module acts as the client. The two are connected using a fixed socket interface. The server is responsible for listening, then receiving the connection request from the client and successfully connecting. Then, after receiving and processing the EEG data, the data processing module sends instructions to the client. Then, the client completes the instruction and notifies the server of the completion. Finally, the server closes the connection and waits for the client to connect again.
[0067] Exoskeleton control module: outputs control instructions based on the classification results, and uses the control instructions to control the exoskeleton to specify rehabilitation training tasks.
[0068] For example, Figure 5 As shown, the exoskeleton module is designed using Solidworks modeling software. In order to adapt to users with different arm spans, a retractable connecting rod is used, which can achieve a length adjustment of 5-15cm. It can realize three joint movements: shoulder rotation, elbow flexion, and wrist rotation. After the design is completed, 3D printing technology is used for model printing and assembly.
[0069] In another implementation of the present invention, a data communication module is also included, and the data communication module uses the TCP / IP protocol to connect the online data processing module with the exoskeleton control module.
[0070] Exemplarily, the data communication module uses a socket interface to connect the online data processing module and the exoskeleton control module. The online data processing module is responsible for monitoring and receiving connection requests from the exoskeleton control module. After receiving the EEG data and processing it, the online data processing module sends instructions to the exoskeleton control module.
[0071] The unilateral upper limb rehabilitation training system based on brain-computer interface of the present invention adopts the fusion technology of EEG classification module and attention module in the data processing part, which improves the action recognition accuracy of motor imagery EEG data; and uses socket technology to realize the data interaction between online EEG signal feature analysis and upper limb rehabilitation exoskeleton; while ensuring the convenience and comfort of the trainees, the rehabilitation efficiency of the patients is improved.
[0072] In another aspect of the present invention, an electronic device includes: a processor, a memory, a communication bus, and a communication interface.
[0073] in:
[0074] The processor, memory and communication interface communicate with each other through a communication bus.
[0075] Communication interface, used to communicate with other electronic devices or servers.
[0076] The processor is used to execute the program, and specifically can execute the steps of any one of the brain-computer interface-based unilateral upper limb rehabilitation training methods in the above-mentioned embodiments.
[0077] Specifically, the program may include program codes including computer operation instructions.
[0078] The processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.
[0079] The memory is used to store programs. The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.
[0080] The program can be specifically used to enable the processor to execute to implement the steps of any one of the unilateral upper limb rehabilitation training methods based on brain-computer interface described in the embodiment. The specific implementation of each step in the program can refer to the corresponding description in the steps and units executed by any one of the unilateral upper limb rehabilitation training methods based on brain-computer interface in the above steps, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described equipment and modules can refer to the corresponding process description in the aforementioned method embodiment.
[0081] The exemplary embodiments of the present application also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the methods of the various embodiments of the present application.
[0082] The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code that may be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded over a network and will be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, processor, or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.
[0083] Thus far, specific embodiments of the present invention have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired results. Additionally, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing may be advantageous.
[0084] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0085] In the description of the present invention, the terms "first" and "second" are only used to facilitate the description of different components or names, and cannot be understood as indicating or implying a sequential relationship, relative importance, or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.
[0086] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.
[0087] It should be noted that although the specific embodiments of the present invention are described in detail in conjunction with the accompanying drawings, it should not be understood as limiting the scope of protection of the present invention. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still belong to the scope of protection of the present invention.
[0088] The examples of the embodiments of the present invention are intended to concisely illustrate the technical features of the embodiments of the present invention so that those skilled in the art can intuitively understand the technical features of the embodiments of the present invention, and are not intended to be improper limitations of the embodiments of the present invention.
[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A unilateral upper limb rehabilitation training method based on brain-computer interface, characterized in that: include: Collect EEG data of motor imagery of upper limb shoulder rotation, elbow flexion, and wrist rotation; Preprocessing the motor imagery EEG data; The pre-processed motor imagery EEG data are trained and classified through the constructed neural network to obtain the classification results; outputting a control instruction based on the classification result; The control instructions are used to control the exoskeleton to specify rehabilitation training tasks.
2. The method according to claim 1, characterized in that: The preprocessing includes basic filtering, ICA artifact removal, multi-band division, segmentation and baseline correction, and manual inspection.
3. The method according to claim 1, characterized in that The neural network consists of an EEG classification module, an attention module, a feature fusion module, and a feature classification module; The training and classification of the pre-processed motor imagery EEG data by the neural network to obtain the classification results includes: Extracting the time-frequency domain and spatial features of the pre-processed motor imagery EEG data through the EEG classification module; Selecting a high-weighted portion of the spatial feature through the attention module; The feature fusion module reorganizes the extracted features according to the parts with high weights in the spatial features to obtain comprehensive features; The comprehensive features are classified using a feature classification module to obtain a classification result.
4. The method according to claim 3, characterized in that The selecting a part with a high weight in the spatial feature by the attention module includes: The spatial features are compressed into a vector through global average pooling, which is calculated as follows: Among them, Z c is the global feature, H and W are the height and width of the feature map respectively; Use a fully connected layer to map the compressed vector to a low-dimensional space and activate it through a nonlinear activation function; Mapped back to the original dimension through another fully connected layer, the importance weight of each channel is learned and calculated as follows: s=σ(W2·δ(W1·z)) Among them, δ is the ReLU activation function, σ is the Sigmoid activation function, W1 and W2 are the weights of the fully connected layer; The original spatial features are weighted by channel, and the features of each channel are recalibrated as follows: Among them, s c is the weight of each channel obtained through the weighted operation process, is the adjusted output.
5. A unilateral upper limb rehabilitation training system based on brain-computer interface, characterized in that: include: EEG signal acquisition module: used to collect EEG data of upper limb shoulder rotation, elbow flexion, and wrist rotation; Online data processing module: used for preprocessing the motor imagery EEG data; The pre-processed motor imagery EEG data are trained and classified through the constructed neural network to obtain the classification results; outputting a control instruction based on the classification result; Exoskeleton control module: uses the control instructions to control the exoskeleton to specify rehabilitation training tasks.
6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the unilateral upper limb rehabilitation training method based on brain-computer interface as described in any one of claims 1 to 4 are implemented.
7. A computer storage medium, characterized in that: The computer storage medium stores a computer program, which, when executed by a processor, implements the steps in the unilateral upper limb rehabilitation training method based on brain-computer interface as described in any one of claims 1 to 4.