Hand continuous motion estimation method and rehabilitation training robot control method
By constructing a continuous hand motion estimation model with parallel modules and an external attention feature extraction mechanism, the problem of inaccurate surface electromyography signal estimation in existing technologies is solved, and accurate estimation of hand joint motion is achieved, thereby improving the autonomous adaptability of rehabilitation training robots and the level of intelligence in human-computer interaction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DALIAN MARITIME UNIVERSITY
- Filing Date
- 2024-12-26
- Publication Date
- 2026-07-24
AI Technical Summary
In existing technologies, continuous motion estimation methods based on surface electromyography signals are not accurate enough, which prevents rehabilitation robots from achieving natural and smooth joint movements, thus limiting the effectiveness and safety of rehabilitation training.
By constructing a continuous hand motion estimation model that combines parallel modules and external attention feature extraction mechanisms, surface electromyography signals and joint angle signals are preprocessed, features are collected and extracted in real time, and smoothed results of hand joint angle signals are obtained, thus achieving accurate estimation of continuous hand motion.
It significantly improves the autonomous adaptability of rehabilitation training robots, creates a safe and comfortable training environment, and promotes the intelligent and humanized development of human-computer interaction.
Smart Images
Figure CN119949812B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of biological signal processing, specifically relating to a method for estimating continuous hand movements and a control method for a rehabilitation training robot. Background Technology
[0002] In recent years, both artificial intelligence technology and neuroscience have experienced rapid development. Against this backdrop, the scientific challenge of analyzing bioelectrical signals corresponding to human movement intentions has encountered new opportunities, with its feasibility significantly improved. Among these, surface electromyography (EMG) signals, as a type of bioelectrical signal with significant research value, have received widespread and high attention in the scientific community due to their rich motion information content, ease of acquisition, and non-invasive safety. From a research application perspective, continuous motion estimation technology based on surface EMG has gradually become an important research direction and key technological route for achieving breakthroughs in next-generation human-computer interaction technology. In-depth exploration of this technological route has profound significance for expanding the application boundaries of human-computer interaction.
[0003] Current research on the complex scientific problem of motion intention recognition mainly focuses on two core aspects: the classification of discrete movements and the identification of movement patterns, and the direct decoding of motor intention information from the central nervous system (including key kinematic parameters such as joint angles, angular velocities, and torques). During the research process, it was found that although current methods for classification problems are relatively mature both theoretically and practically, their limitations are also significant, namely, they can only predict a limited number of discrete limb movements. When these prediction results are applied to the control scenarios of rehabilitation robots, due to the fundamental difference between discrete movement prediction and continuous natural human movement, rehabilitation robots cannot achieve the natural and smooth movements of human joints. From the interdisciplinary perspective of rehabilitation medicine and ergonomics, continuous matching of human-machine movements is a core prerequisite for ensuring the safe and effective control of rehabilitation robots. Therefore, from the perspective of combining scientific innovation and practical application, accurately estimating the continuous movement of human joints through surface electromyography (EMG) signals has an undeniable positive effect on improving the effectiveness of rehabilitation training and advancing the rehabilitation process for patients. Summary of the Invention
[0004] In view of the technical problems of insufficient accuracy and limited range of motion in existing continuous estimation methods, this invention provides a method for estimating continuous hand movements and a control method for a rehabilitation training robot to solve the problems existing in the prior art.
[0005] The technical means employed in this invention are as follows:
[0006] A method for estimating continuous hand movements includes the following steps: Acquire surface electromyography (EMG) signals at joint sites and hand joint angle signals, and preprocess the EMG signals and hand joint angle signals to generate one-to-one corresponding EMG features and hand joint angle signals as training data; A continuous hand motion estimation model combining parallel modules and external attention feature extraction mechanism is constructed. The surface electromyography features in the training data are used as the model input and the hand joint angle signal is used as the model output to train the continuous hand motion estimation model. The surface electromyography signals to be estimated are collected in real time, and after feature extraction, they are input into the trained continuous hand motion estimation model. The hand joint angle signal output by the hand continuous motion estimation model is obtained, and the hand joint angle signal is smoothed to obtain the hand continuous motion estimation result.
[0007] Furthermore, the hand continuous motion estimation model includes an attention calculation module and a parallel module, which fuses the features extracted through multiple attention layers with the features in the external memory, fuses the fused features with the data extracted by the parallel module, and then maps them to the target joint angle through a linear layer.
[0008] Furthermore, the attention calculation module includes a one-dimensional convolution module and a multilayer perceptron module, with each layer perceptron module using an attention head to focus on the input features in different ways.
[0009] Furthermore, the parallel module extracts local features from the feature map through three two-dimensional convolutional networks, compresses the feature information using max pooling, and then introduces nonlinear factors through an activation function.
[0010] Furthermore, the surface electromyography signals and hand joint angle signals are preprocessed, including: First, a filter is used to denoise the acquired surface electromyography signals; Then, the hand joint angle signal is resampled to synchronize it with the surface electromyography signal; The sliding window technique is then used to extract the surface electromyography (EMG) features from the denoised EMG signals, and the EMG features are normalized to generate one-to-one corresponding EMG features and hand joint angle signals.
[0011] Furthermore, the acquired surface electromyography signals are denoised using filters, including: First, the acquired surface electromyography signals are subjected to notch filtering to extract the 50Hz and its harmonic components; Next, Butterworth high-pass filtering is used to remove motion artifacts from the notch-filtered signal.
[0012] Furthermore, the sliding window technique is used to extract electromyographic features from the denoised surface electromyographic signal, including: performing root mean square extraction on the surface electromyographic signal in the sliding window, and using the extracted root mean square as the surface electromyographic feature.
[0013] This invention also discloses a control method for a rehabilitation training robot, comprising the following steps: Estimate continuous hand movements based on any of the methods described above; The estimated results are used as the control reference input for the rehabilitation training robot.
[0014] Compared with the prior art, the present invention has the following advantages: This invention utilizes surface electromyography (EMG) signals to identify the angles of multiple joints in the human hand. The resulting estimates can serve as a crucial reference input for controlling a robot, thereby significantly improving the robot's autonomous adaptability in rehabilitation training scenarios. This not only creates a safe and comfortable training environment for patients but also provides strong scientific support and practical evidence for promoting the development of human-computer interaction towards a more intelligent and human-centered direction. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a flowchart of a method for estimating continuous hand movements according to an embodiment of the present invention.
[0017] Figure 2 This is a schematic diagram of hand joint angle estimation using preprocessed surface electromyography signals in an embodiment of the present invention.
[0018] Figure 3 This is a diagram of the architecture of the continuous hand motion estimation model in an embodiment of the present invention.
[0019] Figure 4 This is a structural diagram of the parallel module in an embodiment of the present invention.
[0020] Figure 5 This is a comparison chart of the prediction results of the hand continuous motion estimation method of the present invention and other algorithms in an embodiment of the present invention. Detailed Implementation
[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0022] A method for estimating continuous hand movements includes the following steps: S1. Acquire surface electromyography (EMG) signals and hand joint angle signals at the joint sites, and preprocess the EMG signals and hand joint angle signals to generate one-to-one corresponding EMG features and hand joint angle signals as training data.
[0023] In practice, the target joint location is first identified, followed by the acquisition of electromyography (EMG) signals and joint angle signals at that location. sEMG is recorded using a Delsys Trigno wireless system equipped with 12 wireless sEMG electrodes, and hand movements are measured using a Cyber Glove II with 22 sensors. The Cyber Glove II has a sampling rate of 20Hz, while the Delsys system has a sampling rate of 2000Hz.
[0024] Secondly, the surface electromyography (EMG) signal undergoes a series of operations, including filtering, feature extraction, and normalization, and is synchronized with the hand joint angle signal, such as... Figure 2 As shown, the details are as follows: S101. First, use linear interpolation (real data stream) or nearest neighbor interpolation (discrete data stream) to oversample all surface electromyography signals to the highest sampling frequency.
[0025] S102. Then, a filter was used to remove noise from the surface electromyography signal.
[0026] To perform filtering on the sEMG signal, the surface electromyography signal denoising operation is as follows: First, notch filtering is performed on the acquired electromyography signal to remove the components of 50Hz and its integer multiples. Then, Butterworth high-pass filtering is used to remove motion artifacts in the notch filtered signal, thereby eliminating the influence of low-frequency motion of the object to be predicted on the signal.
[0027] S103. Then, the sliding window technique is used to extract the surface electromyography (EMG) signal features after denoising. Specifically, this includes extracting long-exposure root mean square features from the EMG signals in the window, and then performing logarithmic normalization on the extracted features to obtain the surface EMG features.
[0028] Feature extraction was performed on the sEMG signal, and the root mean square (RMS) feature of the surface electromyography (sEMG) signal was calculated using the sliding window technique. The sliding window size was set to 100 ms, and the sliding step size was set to 0.5 ms to minimize the latency of human-computer interaction while preserving the original information of the EMG signal. The formula for calculating the RMS is as follows:
[0029] Where n represents the length of the sliding window. Let i = 1, 2, 3, ..., n to represent surface electromyography signals with a time length of n.
[0030] Since the sampling rates of the surface electromyography (EMG) signal and the joint angle signal after feature extraction are different, this application uses a linear interpolation method to interpolate the joint angle signal so that it corresponds one-to-one with the surface EMG features in time.
[0031] S2. Construct a continuous hand motion estimation model that combines parallel modules and external attention feature extraction mechanism. Use surface electromyography features in the training data as model input and hand joint angle signals as model output to train the continuous hand motion estimation model.
[0032] like Figure 3 As shown, the hand continuous motion estimation model in this application includes an attention calculation module and a parallel module. The module fuses the features extracted through multi-layer attention with features stored in external memory, then fuses the fused features with the data extracted by the parallel module, and finally maps them to the target joint angle through a linear layer. The attention calculation module includes a one-dimensional convolutional module and a multi-layer perceptron module. Each perceptron module uses an attention head to focus on the input features in different ways. The parallel module extracts local features from the feature map through three two-dimensional convolutional networks, compresses the feature information using max pooling, and then introduces nonlinear factors through an activation function. Figure 4 The diagram shows the structure of the parallel module.
[0033] In the implementation example of this application, the synchronized surface electromyography features and joint angle signals are divided into 100ms (200 sample points) segments as samples and input into the prediction model. These segments are then processed sequentially through a normalization module, an attention-based feature extraction module, a parallel module, and a smoothing module. The specific steps are as follows: S201. The surface electromyography (EMG) signals, divided into small segments, are subjected to root mean square (RMS) extraction, and then fed into the normalization module to normalize the feature signals. The normalized data are then fed into the attention calculation feature extraction module and the parallel module, respectively.
[0034] S202. Before sending the normalized surface electromyography signal into the attention calculation feature extraction module, we first map it to the query tensor of attention calculation through a fully connected network, and then process the query tensor separately.
[0035] Furthermore, the processed split query tensor is input into the attention computation module for feature extraction. First, a one-dimensional convolution with a kernel size of 1×3 is used to extract local information of the feature signal in the temporal dimension. Then, a double normalization module is used to refine the extracted feature signal. This double normalization module includes batch normalization and a softmax process. The features are then extracted again in the temporal dimension by the same one-dimensional convolution, and subsequently mapped by a fully connected network to produce a simplified attention output. After computation by multiple such attention modules, the outputs of the attention modules are fused with the features stored in a global memory to obtain the final output of the attention module.
[0036] In the parallel module (S203), the normalized surface electromyography (EMG) signal is first processed by two 2D convolutional networks to extract features from the temporal and spatial dimensions, with 3×3 kernels in each network. Then, max pooling is used to refine the extracted spatiotemporal features. Subsequently, the spatiotemporal features are processed by an integrated linear mapping network to transform them into output information. This mapping network contains two linear layers, a GELU activation function, and a Dropout module. After a linear interpolation, an output with the same shape and size as the input is obtained. Finally, the output is processed by a 2D convolutional network with the same structure as the aforementioned 2D convolutional networks to merge the dimensions of the feature information into one, resulting in the output of the parallel module.
[0037] S204. The output of the attention calculation feature extraction module is fused with the output of the parallel module, and then mapped through a fully connected network to finally obtain the joint angle data to be predicted.
[0038] S3. Real-time acquisition of surface electromyography (EMG) signals to be estimated, extraction of root mean square features to generate EMG features to be estimated, and input into the trained hand continuous motion estimation model.
[0039] S4. Obtain the hand joint angle signal output by the hand continuous motion estimation model, and smooth the hand joint angle signal to obtain the hand continuous motion estimation result.
[0040] Specifically, the predicted joint angle signals are smoothed. A small number of historical joint angles can be used to process some predictions with larger errors, making them more consistent with actual human movement and improving the robustness of the model.
[0041] To verify the effectiveness of the continuous motion information prediction model obtained in this embodiment, experimental verification was conducted. All experiments were implemented using the PyTorch framework and validated on the publicly available Ninapro dataset.
[0042] Specifically, ten representative subjects from Ninapro were selected. The Pearson correlation coefficient between the predicted curve and the actual curve for each subject on the test set was calculated. For example... Figure 5 As shown, the prediction model (PET) proposed in this embodiment is significantly better than other existing algorithms, such as TCN and LSTM models.
[0043] This invention also discloses a control method for a rehabilitation training robot, comprising the following steps: A1. Estimate the continuous motion of the hand based on the above method; A2. Obtain the estimation results as the control reference input for the rehabilitation training robot.
[0044] This invention uses surface electromyography (EMG) signals to identify the angles of multiple joints in the human hand. The obtained estimation results can serve as a key reference input for controlling the robot, thereby significantly improving the robot's autonomous adaptability in rehabilitation training scenarios.
[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for estimating continuous hand movements, characterized in that, Includes the following steps: Acquire surface electromyography (EMG) signals at joint sites and hand joint angle signals, and preprocess the EMG signals and hand joint angle signals to generate one-to-one corresponding EMG features and hand joint angle signals as training data; A continuous hand motion estimation model combining parallel modules and external attention feature extraction mechanism is constructed. The surface electromyography features in the training data are used as the model input and the hand joint angle signal is used as the model output to train the continuous hand motion estimation model. The hand continuous motion estimation model includes an attention calculation module and a parallel module. The attention calculation module fuses features extracted through multiple attention layers with features stored in external memory. The fused features are then fused with data extracted by the parallel module, and finally mapped to the target joint angle through a linear layer. The attention calculation module includes a one-dimensional convolutional module and a multi-layer perceptron module. Each perceptron module uses an attention head to focus on the input features in different ways. The parallel module extracts local features from the feature map through a two-dimensional convolutional network, compresses the feature information using max pooling, and then introduces nonlinear factors through an activation function. Real-time acquisition of surface electromyography (EMG) signals to be estimated, followed by feature extraction and input into a trained continuous hand motion estimation model, including: The surface electromyography signals, divided into small segments, are subjected to root mean square (RMS) extraction and then fed into a normalization module to normalize the feature signals. The normalized data are then fed into the attention-based feature extraction module and the parallel local feature extraction module, respectively. Before the normalized surface electromyography signal is fed into the attention computation feature extraction module, it is first mapped to the query tensor of attention computation through a fully connected network, and the query tensor is processed separately. The processed split query tensor is input into the attention calculation module for feature extraction. First, a one-dimensional convolution is used to extract local information of the feature signal in the time dimension, with a convolution kernel size of 1×3. Then, the extracted feature signal is processed by a double normalization module, which includes batch normalization and softmax processing. The features are then extracted again in the time dimension by the same one-dimensional convolution, and then mapped by a fully connected network to produce a simplified attention output. After calculation by multiple such attention modules, the output of the attention module is fused with the features stored in a global memory to obtain the output of the entire attention module. In the parallel module, the normalized surface electromyography (EMG) signal is first processed by two 2D convolutional networks to extract features from the temporal and spatial dimensions, with 3×3 kernels for each network. Then, max pooling is used to simplify the extracted spatiotemporal features. Subsequently, the spatiotemporal features are processed by an integrated linear mapping network to convert them into output information. This mapping network contains two linear layers, a GELU activation function, and a Dropout module. After a linear interpolation, an output with the same shape and size as the input is obtained. Finally, the feature information is merged into one dimension by a 2D convolutional network with the same structure as the two-dimensional convolutional network mentioned above, and then the output of the parallel module is obtained. The output of the attention-based feature extraction module is fused with the output of the parallel feature extraction module, and then mapped through a fully connected network to finally obtain the joint angle data to be predicted. The hand joint angle signal output by the hand continuous motion estimation model is obtained, and the hand joint angle signal is smoothed to obtain the hand continuous motion estimation result.
2. The method for estimating continuous hand movements according to claim 1, characterized in that, Preprocessing of surface electromyography signals and hand joint angle signals includes: First, a filter is used to denoise the acquired surface electromyography signals; Then, the hand joint angle signal is resampled to synchronize it with the surface electromyography signal; The sliding window technique is then used to extract the surface electromyography (EMG) features from the denoised EMG signals, and the EMG features are normalized to generate one-to-one corresponding EMG features and hand joint angle signals.
3. The method for estimating continuous hand movements according to claim 2, characterized in that, The acquired surface electromyography (EMG) signals are denoised using filters, including: First, the acquired surface electromyography signals are subjected to notch filtering to extract the 50Hz and its harmonic components; Next, Butterworth high-pass filtering is used to remove motion artifacts from the notch-filtered signal.
4. The method for estimating continuous hand movements according to claim 3, characterized in that, The method of extracting electromyographic features from the denoised surface electromyographic signal using the sliding window technique includes: performing root mean square extraction on the surface electromyographic signal in the sliding window, and using the extracted root mean square as the surface electromyographic feature.
5. A control method for a rehabilitation training robot, characterized in that, Includes the following steps: Estimation of continuous hand movements based on the method described in any one of claims 1-4; The estimated results are used as the control reference input for the rehabilitation training robot.