Hand continuous motion estimation method and rehabilitation training robot control method

Abstract

The invention discloses a hand continuous motion estimation method and a rehabilitation training robot control method, and relates to the technical field of biological signal processing. The motion estimation method comprises the following steps: acquiring a surface electromyogram signal of a joint part and a hand joint angle signal for preprocessing to generate training data; a hand continuous motion estimation model combining a parallel module and an external attention feature extraction mechanism is constructed, the surface myoelectricity features serve as model input, the hand joint angle signals serve as model output, and the hand continuous motion estimation model is trained; collecting to-be-estimated surface electromyogram signals, performing feature extraction, and inputting the to-be-estimated surface electromyogram signals into the trained hand continuous motion estimation model; and acquiring a hand joint angle signal output by the hand continuous motion estimation model, and smoothing the hand joint angle signal so as to acquire a hand continuous motion estimation result. According to the invention, the recognition precision and recognition efficiency of human hand multi-joint angles can be improved by using the surface electromyogram signals.

Description

Technical Field

[0001] The invention belongs to the field of biological signal processing, and in particular relates to a hand continuous motion estimation method and a rehabilitation training robot control method. Background Art

[0002] In recent years, both artificial intelligence technology and neuroscience have shown a trend of rapid development. Against this background, the scientific problem of analyzing the bioelectric signals corresponding to human movement intentions has ushered in new opportunities, and the possibility of its realization has been significantly improved. Among them, surface electromyography signals, as a type of bioelectric signal with great research value, have received widespread and high attention in the scientific community due to their outstanding characteristics such as rich motion information connotations, easy collection convenience, and non-invasive safety. From the perspective of scientific research applications, continuous motion estimation technology based on surface electromyography has gradually become an important research direction and key technical route for achieving breakthroughs in the next generation of human-computer interaction technology. In-depth exploration of this technical route has far-reaching significance for expanding the application boundaries of human-computer interaction.

[0003] In the current scientific research progress, the research on the complex scientific problem of motion intention recognition mainly focuses on two core parts. One is the classification of discrete actions and the recognition of motion patterns, and the other is the direct decoding of the movement intention information of the nerve center (including key kinematic parameters such as joint angles, angular velocities, and torques). In the research process, it was found that although the current research methods for classification problems are relatively mature in theory and practice, their limitations are also obvious, that is, they can only predict a limited number of discrete limb movements. When these prediction results are applied to the control scenario of rehabilitation robots, due to the essential difference between discrete motion prediction and continuous natural movement of the human body, rehabilitation robots cannot achieve natural and smooth movement like human joints. From the perspective of the intersection of rehabilitation medicine and ergonomics, the continuous matching of human-machine motion is the core prerequisite for ensuring the safe and effective control of rehabilitation robots. Therefore, from the perspective of combining scientific research innovation with practical application, accurately estimating the continuous movement of human joints through surface electromyography signals has an important positive effect on improving the rehabilitation training effect and promoting the rehabilitation process of patients. Summary of the invention

[0004] In view of the technical problems existing in the prior art such as the estimation results of the continuous estimation method are not accurate enough and the movements are not rich enough, the present invention provides a hand continuous motion estimation method and a rehabilitation training robot control method to solve the problems existing in the prior art.

[0005] The technical means adopted by the present invention are as follows:

[0006] A method for estimating continuous hand motion comprises the following steps:

[0007] Acquire surface electromyographic signals of joints and hand joint angle signals, and preprocess the surface electromyographic signals and hand joint angle signals to generate one-to-one corresponding surface electromyographic features and hand joint angle signals as training data;

[0008] Constructing a hand continuous motion estimation model combining a parallel module and an external attention feature extraction mechanism, and training the hand continuous motion estimation model by using surface electromyography features in training data as model input and hand joint angle signals as model output;

[0009] The surface electromyographic signal to be estimated is collected in real time, and after feature extraction, it is input into the trained hand continuous motion estimation model;

[0010] The hand joint angle signal output by the hand continuous motion estimation model is obtained, and the hand joint angle signal is smoothed to obtain the hand continuous motion estimation result.

[0011] Furthermore, the hand continuous motion estimation model includes: an attention calculation module and a parallel module, which fuses the features extracted by multiple layers of attention with the features in the external memory, fuses the fused features with the data extracted by the parallel module, and then maps them to the target joint angles through a linear layer.

[0012] Furthermore, the attention calculation module includes a one-dimensional convolution module and a multi-layer perceptron module, and each layer of the perceptron module uses an attention head to focus on input features in different ways.

[0013] Furthermore, the parallel module extracts local features of the feature map through two two-dimensional convolutions, compresses the feature information using maximum pooling, and then introduces nonlinear factors through an activation function.

[0014] Furthermore, the surface electromyography signal and the hand joint angle signal are preprocessed, including:

[0015] Firstly, a filter is used to denoise the acquired surface electromyographic signal;

[0016] Then the hand joint angle signals are resampled to synchronize them with the surface electromyography signals;

[0017] Then, the sliding window technology is used to extract the surface electromyography features in the denoised surface electromyography signal, and the surface electromyography features are normalized to generate one-to-one corresponding surface electromyography features and hand joint angle signals.

[0018] Furthermore, a filter is used to perform denoising on the acquired surface electromyography signal, including:

[0019] Firstly, the collected surface electromyography signal is processed by notch filtering to extract 50Hz and its multiple frequency components;

[0020] Then, Butterworth high-pass filtering is used to remove motion artifacts in the notch-filtered signal.

[0021] Furthermore, a sliding window technique is used to extract electromyographic features in the denoised surface electromyographic signal, including: performing root mean square extraction on the surface electromyographic signal in the sliding window, and using the extracted root mean square as the surface electromyographic feature.

[0022] The present invention also discloses a rehabilitation training robot control method, comprising the following steps:

[0023] Estimating the continuous hand motion based on any of the above methods;

[0024] The estimated results are obtained as control reference input for the rehabilitation training robot.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] The present invention uses surface electromyography signals to identify the angles of multiple joints of the human hand. The estimated results can be used as key reference inputs for controlling robots, thereby significantly improving the robot's autonomous adaptability in rehabilitation training scenarios. This not only creates a safe and comfortable training environment for patients, but also provides strong scientific support and practical basis for promoting the development of human-computer interaction in a more intelligent and humanized direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0028] Figure 1 The present invention is a flowchart of a method for estimating continuous hand motion in an embodiment of the present invention.

[0029] Figure 2 It is a schematic diagram of estimating hand joint angles based on preprocessed surface electromyography signals in an embodiment of the present invention.

[0030] Figure 3 4 is a diagram of the architecture of a continuous hand motion estimation model in an embodiment of the present invention.

[0031] Figure 4 4 is a structural diagram of a parallel module in an embodiment of the present invention.

[0032] Figure 5 This is a comparison chart of the prediction results of the continuous hand motion estimation method of the present invention and other algorithms in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] A method for estimating continuous hand motion comprises the following steps:

[0035] S1. Acquire surface electromyographic signals of joints and hand joint angle signals, and preprocess the surface electromyographic signals and hand joint angle signals to generate one-to-one corresponding surface electromyographic features and hand joint angle signals as training data.

[0036] In the specific implementation, the human joint part to be predicted is first determined, and then the electromyographic signal and joint angle signal of the part are collected. The Delsys Trigno wireless system equipped with 12 wireless sEMG electrodes is used to record sEMG, and the Cyber ​​Glove II with 22 sensors is used to measure hand movement. The sampling rate of Cyber ​​Glove II is 20Hz, and the sampling rate of Delsys is 2000Hz.

[0037] Secondly, the surface electromyography signal is filtered, feature extracted, normalized and a series of operations are carried out in sequence, and the surface electromyography signal is synchronized with the hand joint angle signal, such as Figure 2 As shown, the details are as follows:

[0038] S101. First, all surface electromyographic signals are oversampled to the highest sampling frequency using linear interpolation (real-valued data stream) or nearest neighbor interpolation (discrete data stream).

[0039] S102, then use a filter to remove noise in the surface electromyography signal.

[0040] Filtering processing is implemented on the sEMG signal, and the surface EMG signal denoising operation is as follows: first, the collected EMG signal is notched filtered to remove the components of 50 Hz and its integer multiples, and then Butterworth high-pass filtering is used to remove motion artifacts in the notch filtered signal, thereby eliminating the influence of low-frequency motion of the object to be predicted on the signal.

[0041] S103, using the sliding window technology to extract the features of the surface electromyography signal after denoising, specifically including extracting the long exposure root mean square features of the surface electromyography signal in the window, and then performing logarithmic normalization calculation on the extracted features to obtain the surface electromyography features.

[0042] The sEMG signal is feature extracted, and the root mean square feature of the surface electromyographic signal is calculated using the sliding window technology. The sliding window size is set to 100ms, and the sliding step is set to 0.5ms to minimize the delay of human-computer interaction while retaining the original information of the electromyographic signal. The root mean square calculation formula is as follows:

[0043]

[0044] Where n represents the length of the sliding window, y i , i = 1, 2, 3, ..., n represents the surface electromyography signal with a time length of n.

[0045] Since the sampling rates of the surface electromyographic signal and the joint angle signal after feature extraction are different, the present application adopts a linear interpolation method to interpolate the joint angle signal so that it corresponds one-to-one with the surface electromyographic feature in time.

[0046] S2. Construct a hand continuous motion estimation model that combines a parallel module and an external attention feature extraction mechanism, and use the surface electromyography features in the training data as the model input and the hand joint angle signal as the model output to train the hand continuous motion estimation model.

[0047] like Figure 3 As shown, the continuous hand motion estimation model in the present application includes: an attention calculation module and a parallel module, which fuses the features extracted by multiple layers of attention with the features in the external memory, fuses the fused features with the data extracted by the parallel module, and then maps them to the target joint angle through a linear layer. The attention calculation module includes a one-dimensional convolution module and a multi-layer perceptron module. Each layer of the perceptron module uses an attention head to focus on the input features in different ways. The parallel module extracts the local features of the feature map through two two-dimensional convolutions, compresses the feature information using maximum pooling, and then introduces nonlinear factors through activation functions. As shown in FIG. Figure 4 Shown is the parallel module structure diagram.

[0048] In the implementation case of this application, the synchronized surface electromyography features and joint angle signals are divided into small segments of 100ms (200 sample points) as samples and input into the prediction model, which are processed in turn by the normalization module, the attention calculation feature extraction module, the parallel calculation local feature extraction module, and the smoothing module. The specific steps are as follows:

[0049] S201, extracting the root mean square operation of the surface electromyographic signal divided into small segments, and then sending it to the normalization module to normalize the feature signal. The normalized data are sent to the attention calculation feature extraction module and the parallel calculation local feature extraction module respectively.

[0050] S202. Before sending the normalized surface electromyography signal to the attention calculation feature extraction module, we first map it into a query tensor for attention calculation through a fully connected network and process the query tensor separately.

[0051] Furthermore, the processed split query tensor is input into the attention calculation module for feature extraction. First, a one-dimensional convolution is used to extract the local information of the feature signal in the time dimension, and the convolution kernel size is 1×3, and then the extracted feature signal is sorted through the double normalization module. The double normalization module contains a batch normalization and a Softmax process. Then the feature will be extracted in the time dimension by the same one-dimensional convolution, and then the output is mapped by a fully connected network to simplify the output of the attention. After calculation by multiple such attention modules, the output of the attention module is fused with the features stored in a global memory to obtain the output of the entire attention module.

[0052] S203, in the parallel computing module, the normalized surface electromyographic signal is first subjected to three two-dimensional convolutions to extract features from the time dimension and the space dimension, and the convolution kernels are all 3×3. Then the spatiotemporal feature information extracted is simplified using maximum pooling. Subsequently, the spatiotemporal feature information is processed by an integrated linear mapping network to be converted into output information. This mapping network contains two linear layers, a GELU activation function, and a Dropout module. After a linear interpolation, an output of the same shape and size as the input is obtained. Finally, a two-dimensional convolution network identical to the above is used to merge the dimensions of the feature information into one, and then the output of the parallel computing module is obtained.

[0053] S204, the output of the attention calculation feature extraction module is merged with the output of the parallel calculation feature extraction module, and after being mapped by a fully connected network, the joint angle data to be predicted is finally obtained.

[0054] S3. Collect the surface electromyography signal to be estimated in real time, extract the root mean square feature and generate the surface electromyography feature to be estimated, and input it into the trained hand continuous motion estimation model.

[0055] S4. Obtain the hand joint angle signal output by the hand continuous motion estimation model, and smooth the hand joint angle signal to obtain the hand continuous motion estimation result.

[0056] Specifically, the predicted joint angle signals are smoothed. A small amount of historical joint angles can be used to process some predicted values ​​with large errors, making them more consistent with the actual movement of the human body and improving the robustness of the model.

[0057] In order to verify the effect of the continuous motion information prediction model obtained by this embodiment, an experimental verification was carried out. The experiments were all implemented using the PyTorch framework and verified on the public dataset Ninapro.

[0058] Specifically, ten representative subjects in Ninapro were selected. The Pearson correlation coefficient between the predicted curve and the actual curve of each subject's test set was calculated. Figure 5 As shown, the prediction model (PET) proposed in this embodiment is significantly better than other existing algorithms, such as TCN and LSTM models.

[0059] The embodiment of the present invention also discloses a rehabilitation training robot control method, comprising the following steps:

[0060] A1. Estimate the continuous motion of the hand based on the above method;

[0061] A2. Obtain the estimated result as the control reference input of the rehabilitation training robot.

[0062] The present invention uses surface electromyography signals to identify the multi-joint angles of the human hand. The estimated results obtained can be used as key reference inputs for controlling the robot, thereby significantly improving the robot's autonomous adaptability in rehabilitation training scenarios.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for estimating continuous hand motion, characterized in that: The following steps are involved: Acquire surface electromyographic signals of joints and hand joint angle signals, and preprocess the surface electromyographic signals and hand joint angle signals to generate one-to-one corresponding surface electromyographic features and hand joint angle signals as training data; Constructing a hand continuous motion estimation model combining a parallel module and an external attention feature extraction mechanism, and training the hand continuous motion estimation model by using surface electromyography features in training data as model input and hand joint angle signals as model output; The surface electromyographic signal to be estimated is collected in real time, and after feature extraction, it is input into the trained hand continuous motion estimation model; The hand joint angle signal output by the hand continuous motion estimation model is obtained, and the hand joint angle signal is smoothed to obtain the hand continuous motion estimation result.

2. A method for estimating continuous hand motion according to claim 1, characterized in that: The hand continuous motion estimation model includes: an attention calculation module and a parallel module, which fuses the features extracted by multiple layers of attention with the features in the external memory, fuses the fused features with the data extracted by the parallel module, and then maps them to the target joint angle through a linear layer.

3. A method for estimating continuous hand motion according to claim 2, characterized in that: The attention calculation module includes a one-dimensional convolution module and a multi-layer perceptron module, and each layer of the perceptron module uses an attention head to focus on input features in different ways.

4. A method for estimating continuous hand motion according to claim 2, characterized in that: The parallel module extracts local features of the feature map through two two-dimensional convolutions, compresses the feature information using maximum pooling, and then introduces nonlinear factors through an activation function.

5. The method for estimating continuous hand motion according to claim 1, characterized in that: Preprocessing of surface electromyography signals and hand joint angle signals, including: Firstly, a filter is used to denoise the acquired surface electromyography signal; Then the hand joint angle signals are resampled to synchronize them with the surface electromyography signals; Then, the sliding window technology is used to extract the surface electromyography features in the denoised surface electromyography signal, and the surface electromyography features are normalized to generate one-to-one corresponding surface electromyography features and hand joint angle signals.

6. A method for estimating continuous hand motion according to claim 5, characterized in that: The acquired surface electromyography signal is denoised using a filter, including: Firstly, the collected surface electromyography signal is processed by notch filtering to extract 50Hz and its multiple frequency components; Then, Butterworth high-pass filtering is used to remove motion artifacts in the notch-filtered signal.

7. A method for estimating continuous hand motion according to claim 5, characterized in that: The sliding window technology is used to extract the electromyographic features in the surface electromyographic signal after denoising, including: extracting the root mean square of the surface electromyographic signal in the sliding window, and using the extracted root mean square as the surface electromyographic feature.

8. A rehabilitation training robot control method, characterized in that: The following steps are involved: Estimating continuous hand motion based on the method described in any one of claims 1 to 7; The estimated results are obtained as control reference input for the rehabilitation training robot.

Images