Mirror image control training method and system for hand rehabilitation robot based on electromyographic signals

Abstract

The invention discloses a hand rehabilitation robot mirroring control training method and system based on electromyographic signals, relates to the technical field of rehabilitation machine learning, and aims to solve the problem that in the prior art, during mirroring training, hand action recognition is inaccurate, and the method comprises the steps that real-time electromyographic signal data of the healthy side hand of a patient is obtained; performing feature extraction on the real-time electromyographic signal data of the healthy side hand of the patient to obtain time domain features and frequency domain features; the time domain features and the frequency domain features serve as input, and a gesture classification result is obtained based on output of a multi-layer perceptron; according to the gesture classification result, the hand rehabilitation robot is used for conducting mirror image control training on the hand on the affected side. The method can effectively improve the gesture recognition accuracy of the healthy side hand in mirror image training, improves the mirror image training effect, and promotes the rehabilitation of the hand motor nerves of the patient.

Description

Technical Field

[0001] The invention relates to a hand rehabilitation robot mirror control training method and system based on electromyographic signals, belonging to the technical field of rehabilitation machine learning. Background Art

[0002] According to statistics, stroke has become one of the main causes of disability among adults in my country. Although most stroke patients can survive after treatment, 80%-90% will have residual motor function defects, which seriously affect their daily lives. In addition, patients with hand trauma, fractures, tendon injuries, etc. also need hand rehabilitation training after wound healing, which increases the demand for hand rehabilitation treatment. In recent years, robot technology has made great progress, with the advantages of high precision, high stability, and repeatability, which can meet the requirements of hand rehabilitation training for movement accuracy and training intensity.

[0003] The training modes of hand rehabilitation robots are divided into passive training mode and active training mode. Compared with the passive training mode, the active training mode can better stimulate the patient's enthusiasm, promote the development of neuromuscular coordination, and stimulate the remodeling of brain neurons. Mirror training is a type of active training mode. For patients with hand injuries or functional impairments, the patient's healthy arm is used as an ideal movement pattern reference. The hand rehabilitation robot guides the injured hand to perform the same movement to achieve mirror training and ensure that the two hands maintain consistency in terms of movement amplitude, speed, strength, etc.

[0004] Current mirror training for hand rehabilitation robots is mostly based on leap motion sensors, but because it is easily affected by factors such as ambient light and object occlusion, recognition is inaccurate when hand movements are complex. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a mirror control training method and system for a hand rehabilitation robot based on electromyographic signals, which can effectively improve the gesture recognition accuracy of the healthy hand in mirror training, improve the effect of mirror training, and promote the rehabilitation of the patient's hand motor nerves.

[0006] To achieve the above object, the present invention is implemented by adopting the following technical solutions: On the one hand, the present invention provides a mirror control training method for a hand rehabilitation robot based on electromyographic signals, comprising: Obtain real-time electromyographic signal data of the patient's healthy hand; The real-time electromyographic signal data of the patient's healthy hand is extracted to obtain time domain features and frequency domain features; Taking the time domain features and frequency domain features as input, the gesture classification result is obtained based on the output of the multi-layer perceptron; According to the gesture classification results, the hand rehabilitation robot was used to conduct mirror control training on the patient's affected hand.

[0007] Furthermore, the acquisition of real-time electromyographic signal data of the patient's healthy hand is achieved through a MYO electromyographic acquisition armband, which includes a nine-degree-of-freedom motion sensor and 8 dry electrodes, specifically including: Eight dry electrodes were evenly placed on the patient's healthy forearm; The electromyographic signal of the patient's healthy hand forearm is collected by dry electrodes at a sampling frequency of 200 Hz and transmitted to a nine-degree-of-freedom motion sensor. The real-time electromyographic signal data of the patient's healthy hand is obtained through the output of the nine-degree-of-freedom motion sensor.

[0008] Furthermore, the feature extraction of the real-time electromyographic signal data of the healthy hand of the patient to obtain time domain features and frequency domain features includes: The real-time electromyographic signal data of the patient's healthy hand is filtered by a bandpass filter to remove the baseline drift below 10 Hz and the high-frequency noise above 500 Hz, and the electromyographic signal data after filtering is obtained; Amplifying and normalizing the electromyographic signal data after filtering to obtain processed electromyographic signal data; The processed electromyographic signal data is subjected to time domain feature extraction through the Hanning window function, and frequency domain feature extraction through short-time Fourier transform with a window length of 30ms to obtain time domain features and frequency domain features.

[0009] Furthermore, the time domain features include the average absolute value of the electromyographic signal voltage, the zero crossing rate of the electromyographic signal voltage, the slope sign change of the electromyographic signal voltage and the length of the electromyographic signal waveform; the frequency domain features include the power spectral density of the electromyographic signal voltage, the average power frequency of the electromyographic signal voltage and the median frequency of the electromyographic signal voltage.

[0010] Furthermore, the multilayer perceptron includes an input layer, a hidden layer and an output layer connected in sequence, the input layer uses 10 neurons; the number of the hidden layers is 3, each of which uses 50 neurons, and the output layer uses 6 neurons; The neurons in the hidden layer use the ReLU activation function, and the neurons in the output layer use the Softmax activation function.

[0011] Furthermore, it also includes pre-training the multi-layer perceptron, and the pre-training method includes: S1, collecting the electromyographic signals generated when the patient's healthy hand makes different gestures, and obtaining electromyographic signal data corresponding to different gestures; S2, extracting features from the electromyographic signal data corresponding to different gestures to obtain feature data corresponding to different gestures; S3, dividing the feature data corresponding to different gestures into training set data and test set data respectively; S4, taking the training set data of each gesture as input in turn, pre-training the multi-layer perceptron, during the training process, calculating the cross entropy loss function, and using the Adam optimizer to adjust the model weight according to the loss function; S5. Use the test set data of each gesture as input in turn to test the multilayer perceptron. If the accuracy of the gesture classification result output by the multilayer perceptron is higher than the preset value, a pre-trained multilayer perceptron is obtained. If the accuracy of the gesture classification result output by the multilayer perceptron is lower than the preset value, repeat S4 to S5 until a pre-trained multilayer perceptron is obtained.

[0012] Furthermore, the gestures include single thumb bending, pointed shape, tripod shape, cylindrical shape, half grip, and full grip.

[0013] Furthermore, the electromyographic signals generated when the healthy hand of the patient performs different gestures are collected to obtain electromyographic signal data corresponding to different gestures, including: Wear the MYO electromyography armband on the patient's healthy forearm; When the patient is making gestures, the MYO electromyographic acquisition armband is used to collect the electromyographic signals generated by different gestures to obtain the electromyographic signal data corresponding to different gestures. The acquisition parameters include collecting each gesture 10 times, with an interval of 10 seconds each, and stopping for 3 minutes before changing to the next gesture.

[0014] Furthermore, the hand rehabilitation robot is used to perform mirror control training on the affected hand of the patient according to the gesture classification result, including: The gesture classification structure is encoded into binary instructions by the host computer and transmitted to the hand rehabilitation robot; The binary instruction code is decoded by the hand rehabilitation robot, and mirror control training is performed on the affected hand according to the parsed instructions.

[0015] On the other hand, the present invention also provides a hand rehabilitation robot mirror control training system based on electromyographic signals, comprising: The electromyographic signal acquisition module is configured to acquire real-time electromyographic signal data of the patient's healthy hand; A feature extraction module is configured to extract features from the real-time electromyographic signal data of the healthy hand of the patient to obtain time domain features and frequency domain features; A gesture classification module is configured to take the time domain features and the frequency domain features as inputs and obtain a gesture classification result based on the output of the multi-layer perceptron; The mirror control training module uses a hand rehabilitation robot to perform mirror control training on the patient's affected hand based on the gesture classification results.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention uses electromyographic signal collection to achieve a non-invasive and relatively comfortable detection method to keenly capture muscle activity, thereby more accurately identifying gestures, quantitatively and objectively evaluating hand muscle strength, and effectively predicting and identifying the patient's movement intentions; applying the identified mobile phone to mirror control training can promote the repair and regeneration of the patient's motor neurons and accelerate the recovery of the motor function of the affected hand; The present invention performs mirror training based on gesture recognition, which can better realize active training, stimulate the patient's neural perception of force and movement control, and promote the patient's participation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a flow chart of a mirror control training method for a hand rehabilitation robot based on electromyographic signals in an embodiment of the present invention; Figure 2 Schematic diagram of gesture types of a mirror control training method for a hand rehabilitation robot based on electromyographic signals in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention. Example 1

[0019] like Figure 1 As shown, an embodiment of the present invention provides a mirror control training method for a hand rehabilitation robot based on electromyographic signals, including a training phase and an implementation phase.

[0020] Training phase: A multilayer perceptron (MLP) is constructed. In this embodiment, the multilayer perceptron includes an input layer, a hidden layer, and an output layer connected in sequence, wherein the number of hidden layers is 3, the input layer uses 10 neurons, the hidden layer uses 50 neurons, the neurons of the hidden layer use the ReLU activation function, the output layer uses 6 neurons, and the neurons of the output layer use the Softmax activation function.

[0021] Pre-train the multi-layer perceptron. The pre-training methods include: S1. Collect the electromyographic signals generated when the healthy hand of the patient makes different gestures. The healthy hand is the healthy hand of the patient. In this embodiment, the electromyographic signals are collected by the MYO electromyographic collection armband. The MYO electromyographic collection armband includes a nine-degree-of-freedom motion sensor and eight dry electrodes.

[0022] The acquisition operation includes: evenly placing 8 dry electrodes on the patient's healthy forearm to ensure good contact between the dry electrodes and the skin.

[0023] The electromyographic signal of the patient's healthy hand forearm is collected by dry electrodes at a sampling frequency of 200 Hz and transmitted to a nine-degree-of-freedom motion sensor. The nine-degree-of-freedom motion sensor outputs the electromyographic signal data generated when the patient's healthy hand makes different gestures. Each gesture is collected 10 times, with an interval of 10 seconds each time, and stop for 3 minutes before changing to the next gesture.

[0024] In this embodiment, combined with Figure 2 There are 6 types of gestures, namely single thumb bending, tip shape, tripod shape, cylinder shape, half grip, and full grip. Single thumb bending is a single thumb bending 110°, tip shape is a 130° index finger bending, and a 40° thumb bending, forming a "pinching" action. Tripod shape is a 50° thumb bending, a 120° index finger bending, and a 130° middle finger bending, and a certain force is maintained to maintain the shape. Cylinder shape is a 140° bending angle for the index finger, middle finger, ring finger, and little finger, and a 60° bending angle for the thumb. The position should be coordinated with other fingers, so that a hollow cylindrical space is formed in the center of the palm to surround a shape that is approximately cylindrical. Half grip is a 140° bending angle for the index finger, middle finger, ring finger, and little finger, and a 60° bending angle for the thumb. Full grip is a 185° bending angle for the index finger, middle finger, ring finger, and little finger, and a 90° bending angle for the thumb. S2. Extract features from the electromyographic signal data generated by different gestures. Specifically: A bandpass Butterworth filter was used to filter out baseline drift below 10 Hz and high-frequency noise above 500 Hz, and the signal was amplified by 500 times and the data was normalized to obtain processed electromyographic signal data.

[0025] The processed electromyographic signal data is subjected to time domain feature extraction through the Hanning window function, and frequency domain feature extraction through the short-time Fourier transform (STFT) with a window length of 30ms to obtain time domain features and frequency domain features, thereby obtaining feature data corresponding to different gesture actions.

[0026] In this embodiment, the time domain features include mean absolute value (MAV), zero crossing rate (ZC), slope sign change (SSC) and waveform length (WL); the frequency domain features include power spectral density (PSD), mean power frequency (MPF) and median frequency (MF).

[0027] S3. Divide the feature data corresponding to different gesture actions into training set data and test set data, that is, each gesture is divided into its own training set and test set.

[0028] S4. Select the training set data of a gesture action as input in turn, pre-train the multi-layer perceptron, calculate the cross entropy loss function during the training process, and use the Adam optimizer to adjust the model weight according to the loss function.

[0029] S5. Select the test set data of the gesture action as input and test the multilayer perceptron. If the accuracy of the gesture classification result output by the multilayer perceptron is higher than the preset value, a pre-trained multilayer perceptron is obtained. If the accuracy of the gesture classification result output by the multilayer perceptron is lower than the preset value, repeat S4 to S5 until a pre-trained multilayer perceptron is obtained.

[0030] Then, after one gesture training is completed, the next gesture training is carried out until all gesture training is completed, and finally a pre-trained multi-layer perceptron is obtained.

[0031] Implementation phase: Obtain real-time electromyographic signal data of the patient's healthy hand.

[0032] The real-time electromyographic signal data of the patient's healthy hand is extracted to obtain time domain features and frequency domain features. The time domain features and frequency domain features are used as input, and the gesture classification results are obtained based on the output of the pre-trained multi-layer perceptron.

[0033] The multi-layer perceptron sends the gesture classification results to the host computer, which encodes the gesture classification structure into binary instructions and sends them to the hand rehabilitation robot. In this embodiment, the instruction sending program is written in Python to convert the six gestures into corresponding binary instruction codes, single thumb bending: 000, pointed shape: 001, tripod shape: 010, cylindrical shape: 011, half grip shape: 100, full grip shape: 101, and then sent to the hand rehabilitation robot through the Serial class library of serial communication.

[0034] Finally, the hand rehabilitation robot parses the binary command code and performs mirror control training on the affected hand according to the parsed instructions. Example 2

[0035] On the basis of Example 1, this embodiment provides a hand rehabilitation robot mirror control training system based on electromyographic signals, including an electromyographic signal acquisition module, a feature extraction module, a gesture classification module and a mirror control training module.

[0036] The electromyographic signal acquisition module is configured to acquire real-time electromyographic signal data of the patient's healthy hand.

[0037] The feature extraction module is configured to extract features from the real-time electromyographic signal data of the healthy hand of the patient to obtain time domain features and frequency domain features.

[0038] The gesture classification module is configured to take the time domain features and the frequency domain features as inputs and obtain the gesture classification result based on the multi-layer perceptron output. In this embodiment, the gesture classification module includes a multi-layer perceptron.

[0039] The mirror control training module uses a hand rehabilitation robot to perform mirror control training on the affected hand based on the gesture classification results.

[0040] In this embodiment, the mirror control training module includes a host computer and a hand rehabilitation robot. The multilayer perceptron sends the gesture classification result to the host computer, the host computer encodes the gesture classification result, and then sends it to the hand rehabilitation robot. The hand rehabilitation robot decodes and performs mirror control training on the affected hand.

[0041] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A mirror control training method for a hand rehabilitation robot based on electromyographic signals, characterized in that: include: Obtain real-time electromyographic signal data of the patient's healthy hand; The real-time electromyographic signal data of the patient's healthy hand is extracted to obtain time domain features and frequency domain features; Taking the time domain features and frequency domain features as input, the gesture classification result is obtained based on the output of the multi-layer perceptron; According to the gesture classification results, the hand rehabilitation robot was used to conduct mirror control training on the patient's affected hand.

2. The mirror control training method for hand rehabilitation robot based on electromyographic signals according to claim 1 is characterized in that: The acquisition of real-time electromyographic signal data of the patient's healthy hand is achieved through the MYO electromyographic acquisition armband, which includes a nine-degree-of-freedom motion sensor and 8 dry electrodes, specifically including: Eight dry electrodes were evenly placed on the patient's healthy forearm; The electromyographic signal of the patient's healthy hand forearm is collected by dry electrodes at a sampling frequency of 200 Hz and transmitted to a nine-degree-of-freedom motion sensor. The real-time electromyographic signal data of the patient's healthy hand is obtained through the output of the nine-degree-of-freedom motion sensor.

3. The mirror control training method for hand rehabilitation robot based on electromyographic signals according to claim 1, characterized in that: The feature extraction of the real-time electromyographic signal data of the healthy hand of the patient to obtain time domain features and frequency domain features includes: The real-time electromyographic signal data of the patient's healthy hand is filtered by a bandpass filter to remove the baseline drift below 10 Hz and the high-frequency noise above 500 Hz, and the electromyographic signal data after filtering is obtained; Amplifying and normalizing the electromyographic signal data after filtering to obtain processed electromyographic signal data; The processed electromyographic signal data is subjected to time domain feature extraction through the Hanning window function, and frequency domain feature extraction through short-time Fourier transform with a window length of 30ms to obtain time domain features and frequency domain features.

4. The mirror control training method for hand rehabilitation robot based on electromyographic signals according to claim 3 is characterized in that: The time domain features include the average absolute value of the electromyographic signal voltage, the zero crossing rate of the electromyographic signal voltage, the slope sign change of the electromyographic signal voltage and the length of the electromyographic signal waveform; the frequency domain features include the power spectrum density of the electromyographic signal voltage, the average power frequency of the electromyographic signal voltage and the median frequency of the electromyographic signal voltage.

5. The mirror control training method for hand rehabilitation robot based on electromyographic signals according to claim 1, characterized in that: The multilayer perceptron includes an input layer, a hidden layer and an output layer connected in sequence, wherein the input layer uses 10 neurons; the number of the hidden layers is 3, each of which uses 50 neurons, and the output layer uses 6 neurons; The neurons in the hidden layer use the ReLU activation function, and the neurons in the output layer use the Softmax activation function.

6. The hand rehabilitation robot mirror control training method based on electromyographic signals according to claim 1, characterized in that: The method also includes pre-training a multi-layer perceptron. The pre-training method includes: S1, collecting the electromyographic signals generated when the patient's healthy hand makes different gestures, and obtaining electromyographic signal data corresponding to different gestures; S2, extracting features from the electromyographic signal data corresponding to different gestures to obtain feature data corresponding to different gestures; S3, dividing the feature data corresponding to different gestures into training set data and test set data respectively; S4, taking the training set data of each gesture as input in turn, pre-training the multi-layer perceptron, during the training process, calculating the cross entropy loss function, and using the Adam optimizer to adjust the model weight according to the loss function; S5. Use the test set data of each gesture as input in turn to test the multilayer perceptron. If the accuracy of the gesture classification result output by the multilayer perceptron is higher than the preset value, a pre-trained multilayer perceptron is obtained. If the accuracy of the gesture classification result output by the multilayer perceptron is lower than the preset value, repeat S4 to S5 until a pre-trained multilayer perceptron is obtained.

7. The mirror control training method for hand rehabilitation robot based on electromyographic signals according to claim 6 is characterized in that: The gestures include single thumb bend, pointed shape, tripod shape, cylindrical shape, half grip, and full grip.

8. The mirror control training method for hand rehabilitation robot based on electromyographic signals according to claim 6, characterized in that: The step of collecting electromyographic signals generated when the patient's healthy hand performs different gestures to obtain electromyographic signal data corresponding to different gestures includes: Wear the MYO electromyography armband on the patient's healthy forearm; When the patient is making gestures, the MYO electromyographic acquisition armband is used to collect the electromyographic signals generated by different gestures to obtain the electromyographic signal data corresponding to different gestures. The acquisition parameters include collecting each gesture 10 times, with an interval of 10 seconds each, and stopping for 3 minutes before changing to the next gesture.

9. The mirror control training method for hand rehabilitation robot based on electromyographic signals according to claim 1, characterized in that: The method of using a hand rehabilitation robot to perform mirror control training on the affected hand of the patient according to the gesture classification result comprises: The gesture classification structure is encoded into binary instructions by the host computer and transmitted to the hand rehabilitation robot; The binary instruction code is decoded by the hand rehabilitation robot, and mirror control training is performed on the affected hand according to the parsed instructions.

10. A hand rehabilitation robot mirror control training system based on electromyographic signals, characterized in that: include: The electromyographic signal acquisition module is configured to acquire real-time electromyographic signal data of the patient's healthy hand; A feature extraction module is configured to extract features from the real-time electromyographic signal data of the healthy hand of the patient to obtain time domain features and frequency domain features; A gesture classification module is configured to take the time domain features and the frequency domain features as inputs and obtain a gesture classification result based on the output of the multi-layer perceptron; The mirror control training module uses a hand rehabilitation robot to perform mirror control training on the patient's affected hand based on the gesture classification results.

Images