A method for human upper limb posture estimation based on progressive unscented Kalman filter network
By combining deep learning and the progressive unscented Kalman filter network, the problem of insufficient description of nonlinear models by the Kalman filter network in the existing technology is solved, and high-precision human upper limb posture and joint angle estimation is achieved in small sample data sets.
Patent Information
- Application Number
- CN202211500262.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-28
- Publication Date
- 2025-09-05
- Estimated Expiration
- 2042-11-28
AI Technical Summary
In the existing sEMG-based joint continuous motion estimation methods, the Kalman filter network has limited ability to describe nonlinear models, resulting in insufficient accuracy and robustness. Deep neural networks have poor modeling effects under small sample data sets and cannot accurately estimate the human upper limb posture and joint angles.
A human upper limb posture estimation method based on a progressive unscented Kalman filter network is adopted, which combines deep learning and progressive unscented Kalman filtering. The system noise and measurement noise are learned through the LSTM neural network, and the progressive UKF filtering method with adaptive measurement update is used to improve the nonlinear description ability and stability of the model.
It significantly improves the accuracy and robustness of human upper limb posture estimation in small data sets, outperforms the traditional learning-based KF model, and improves the accuracy of joint angle estimation.
Smart Images
Figure CN116129464B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of human body posture estimation, in particular to a method for estimating human upper limb posture based on a progressive unscented Kalman filter network. Background Art
[0002] With the continuous maturity and development of human-computer interaction technology based on bioelectrical signals, this technology has been widely used in prosthetic control, clinical medicine, virtual reality, robotics, and sports biomechanics. Surface electromyography (sEMG) is a superimposed electrical signal formed on the surface of human skin by the motor unit action potential (MUAP) of movement-related muscles propagating along the muscle fibers. This signal reflects the muscle contraction state that causes limb movement and contains rich information such as muscle contraction force and joint torque. It can be used to decode the intention directly related to movement and is widely used in human posture estimation and human behavior analysis. Compared with the recognition of discrete limb movements, continuous motion estimation based on sEMG signals can extract more human motion information and has applications in intelligent prostheses, medical rehabilitation robots, and motion assessment.
[0003] Currently, two common approaches are used to estimate continuous joint motion based on sEMG. The first approach combines muscle physiology and mechanics to establish a joint dynamics model using sEMG as input, and then calculates continuous quantities such as joint torque, angular acceleration, and angular velocity. This approach has the advantage of being able to explain the process of motion generation. The most widely used muscle force model is the Hill model, a physiological phenomenological model that incorporates multiple physiological parameters that cannot be directly measured. Therefore, preliminary experiments are required to identify effective parameters, which limits its practical application. The second approach directly establishes a regression model linking sEMG and continuous joint motion. This approach has the advantage of a straightforward modeling process and unrestricted use of sEMG. The most widely used direct modeling methods are divided into two categories. One is a deep learning-based approach that utilizes deep neural networks to establish a regression model linking sEMG and joint motion. This typically requires preprocessing the sEMG signal, which is then used to train the neural network to ultimately obtain the corresponding regression model. However, deep learning-based methods are data-driven networks, and the modeling process inevitably ignores some important prior knowledge. Furthermore, deep neural network training requires a large number of samples as training data. sEMG, as a non-stationary signal, is subject to individual differences, muscle fatigue, and other variations or interference during the acquisition phase, resulting in significantly different distributions between the training and test sets, ultimately leading to inaccurate regression models. Another approach, such as Kalman filtering (KF), uses time-series filters to improve regression accuracy under small sample sizes. However, in real systems, KF measurement and state functions are often difficult to obtain directly. Artificially designed measurement and state models are often crude approximations of complex models, reducing KF performance. In particular, in the task of estimating continuous human motion using sEMG, limb posture and joint angles do not follow a simple motion model, and the physical relationship between sEMG generated by muscle contraction and limb posture cannot be directly derived. In this context, KF modeling becomes extremely difficult. Furthermore, linear state-space models, such as KF, do not adequately describe the nonlinear relationship between sEMG and joint motion, and their generalization capabilities need to be further improved.
[0004] To overcome these limitations, some researchers have attempted to use fully connected neural networks or long short-term memory to learn motion models directly from training data. The model obtained by this method can combine the prior knowledge in the KF with the motion model learned by the deep neural network from the data, thereby improving the accuracy of continuous human motion estimation with a smaller data set. However, the KF is a filtering method for linear systems and has limited ability to describe nonlinear models. Even various progressive Kalman methods have limited ability to describe the nonlinear models of deep neural networks, which also affects the accuracy and robustness of the model. In summary, among the existing methods for continuous joint motion estimation based on sEMG, there is no Kalman filter network that can fully and robustly describe nonlinear models.
[0005] Therefore, the present invention proposes a method for estimating the posture of human upper limbs based on a progressive unscented Kalman filter network, which can more fully describe the nonlinearity of sEMG and joint motion when using a smaller data set, and improve the accuracy and robustness in estimating continuous motion of the human body using sEMG signals. Summary of the Invention
[0006] In order to overcome the difficulties in Kalman filter modeling and the inaccuracy of deep neural network regression models in the task of estimating continuous human motion using sEMG, the present invention provides a method for estimating human upper limb posture based on a progressive unscented Kalman filter network, which effectively improves the accuracy and robustness of continuous human motion estimation by fusion of deep learning and progressive unscented Kalman filter.
[0007] The technical solution adopted by the present invention to solve its technical problem is:
[0008] A method for estimating the posture of human upper limbs based on a progressive unscented Kalman filter network, the method comprising the following steps:
[0009] Step 1) Data acquisition: Build a data acquisition system to obtain the angle information of the human upper limb motion joints through a visual capture system, and use an electromyographic wristband to collect the electromyographic signals of the forearm during movement;
[0010] Step 2) Build a model between sEMG signals and upper limb joint angles:
[0011] x k =f(x k-1 )+w k-1 (1)
[0012] z k =h(x k )+v k (2)
[0013] Step 3) An unscented Kalman filter system with LSTM estimation parameters is constructed, and the LSTM neural network is used to learn the system's measurement equation h(·), system noise Q k and measurement noise R k ; Using UT changes, weighted statistical linear regression is used to approximate the posterior mean of the nonlinear equation and variance P k|k-1 .
[0014] Step 4) Transform and measure the noise R according to UT k , and obtain the one-step prediction of the measured variable Variance P zz，k|k-1 and the cross-covariance P xz，k|k-1 .
[0015] Step 5) Calculate the Kalman gain according to the UKF calculation steps, and calculate the Kalman gain according to the measurement value z at time k k , find the state estimate at time k and the estimated variance P k|k .
[0016] Furthermore, in step 1), A, B, and C represent the numbers of the joint points of the human elbow, wherein AB represents the upper arm of the experimenter, BC represents the forearm of the experimenter, and the joint angle of the upper limb elbow joint is the angle θ.
[0017] Furthermore, in step 1), the visual capture system is composed of 12 cameras, and the sampling frequency of the myoelectric bracelet is 200 Hz.
[0018] Further, in the step 2), the x k and z k are the n-dimensional upper limb joint state vector and the m-dimensional sEMG signal measurement vector at time k respectively.
[0019] In the step 2), the f(·) and h(·) are the nonlinear state equation and measurement equation of the system.
[0020] In the step 2), the w k and v k are the system noise and measurement noise with zero mean, and the corresponding covariance Q k and R k is uncorrelated Gaussian white noise.
[0021] Furthermore, in step 3), the unscented Kalman filter system whose parameters are estimated by LSTM is constructed, and the UKF partial model is learned using the LSTM neural network to directly obtain the measurement noise, system noise and measurement function from the training data.
[0022] Furthermore, in step 4), the measurement equation h(·) is directly used with an LSTM h(·) The module is replaced by UT transformation and LSTM R The resulting measurement noise R k , and obtain the one-step prediction of the measured variable Variance P zz，k|k-1 and the cross-covariance P xz，k|k-1 .
[0023] Furthermore, in step 5), a progressive UKF filtering method with adaptive measurement update is added to the UKF to improve the stability of the system; the covariance of the measurement noise is artificially increased to solve the contradiction between stability and filtering accuracy, and the measurement value z at time k is calculated. k , find the state estimate at time k and the estimated variance P k|k .
[0024] Based on the UKF computation graph, the present invention utilizes an LSTM model to obtain the unknown parameters of the UKF. Furthermore, a progressive filtering approach is used to address the Kalman gain instability issue caused by the addition of the UKF to the LSTM. This model combines the advantages of the UKF and LSTM, enabling application in nonlinear systems and achieving high regression accuracy on smaller datasets. It outperforms other learning-based KF models in upper limb joint angle estimation based on sEMG. The present invention provides a method for upper limb posture estimation based on a progressive unscented Kalman filter network, effectively improving the accuracy and robustness of upper limb posture estimation.
[0025] The beneficial effects of the present invention are mainly manifested in: providing a method for estimating human posture based on progressive unscented Kalman filter network fusion, s In the task of continuous motion estimation of the human body using EMG, it is very difficult to use Kalman filtering to model limb posture and joint angles. A long short-term memory neural network is used to directly learn the motion model from training data, and a progressive UKF filtering method with adaptive measurement update is combined to solve the problem of Kalman gain instability, thereby improving the accuracy and robustness of upper limb posture estimation. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 This is a flow chart of upper limb posture estimation based on LSTM-pUKF fusion of the present invention.
[0027] Figure 2 This is the algorithm framework diagram of the upper limb posture fusion estimation based on LSTM-pUKF of the present invention.
[0028] Figure 3 a is the LSTM of the present invention QThe module network diagram, Figure 3 b is LSTM R The module network diagram, Figure 3 c is LSTM h Module network diagram. DETAILED DESCRIPTION
[0029] The present invention will be further described below with reference to the accompanying drawings.
[0030] Reference Figure 1 、 Figure 2 、 Figure 3 A method for estimating the posture of human upper limbs based on a progressive unscented Kalman filter network is provided, the method comprising the following steps:
[0031] Step 1) Data Acquisition: Build a data acquisition system that can obtain the angle information of the human upper limb motion joints through the visual capture system, and use the electromyography acquisition bracelet to collect the electromyography signals of the forearm during movement;
[0032] Step 2) Build a model between sEMG signals and upper limb joint angles:
[0033] x k =f(x k-1 )+w k-1 (1)
[0034] z k =h(x k )+v k (2)
[0035] Step 3) An unscented Kalman filter system with LSTM estimation parameters is constructed, and the LSTM neural network is used to learn the system's measurement equation h(·), system noise Q k and measurement noise R k , using UT changes, using weighted statistical linear regression to approximate the posterior mean of the nonlinear equation and variance P k|k-1 .
[0036] Step 4) Transform and measure the noise R according to UT k , and obtain the one-step prediction of the measured variable Variance P zz，k|k-1 and the cross-covariance P xz，k|k-1 .
[0037] Step 5) Calculate the Kalman gain according to the UKF calculation steps, and calculate the Kalman gain according to the measurement value z at time k k , find the state estimate at time k and the estimated variance P k|k .
[0038] The flow chart of human upper limb posture estimation based on progressive unscented Kalman filter network fusion is as follows: Figure 1 As shown. First, we built an acquisition system and used an electromyographic bracelet to acquire electromyographic signals during forearm movement. At the same time, we used a motion capture system to synchronously acquire upper limb elbow joint angle data. We conducted experiments on multiple different testers, collecting ten sets of data for each specified action. Then, we performed five sets of exercises with random postures and angles. There was sufficient rest time between each experiment to prevent muscle fatigue. The model between sEMG signals and upper limb joint angles was established as follows:
[0039] x k =f(x k-1 )+w k-1 (1)
[0040] z k =h(x k )+v k (2)
[0041] in, and are the n-dimensional upper limb joint state vector and the m-dimensional sEMG signal measurement vector at time k, respectively. f(·) and h(·) are the nonlinear state equation and measurement equation of the system, and the system noise w k and measurement noise v k They have a mean of zero and a covariance of Q k and R k uncorrelated Gaussian white noise.
[0042] Secondly, an unscented Kalman filter system with LSTM parameter estimation is constructed. In the model between the sEMG signal and the upper limb joint angle, the state equation f(·) is known, the measurement equation h(·), the system noise Q k and measurement noise R k are unknown, since the state noise covariance matrix Q k and the measurement noise covariance matrix R k It is difficult to estimate, so LSTM is used to learn Q directly from the state vector and noise vector k and R k :
[0043]
[0044]
[0045] Among them, LSTM Q and LSTM R They are used to learn Q k and R k LSTM module, x k-1is the n-dimensional sEMG signal state vector at time k, z k is the m-dimensional sEMG signal measurement vector at time k, It is k-1 time LSTM Q The output hidden unit, It is k-1 time LSTM R In the training of all networks, the preprocessed s The EMG data was root mean square undersampled and then normalized; the normalized dataset was divided into a training set and a test set according to a ratio of 7:3; 70% of the training set data was input into the network model for training, and 30% of the test set data was used to verify the effectiveness of the model. In the network initialization stage, for all LSTM cell units, all LSTM weight matrices were initialized using a uniform distribution of [0.01, 0.01], and all fully connected layer weight matrices were initialized using Xavier. Except for the LSTM forget bias, all biases were initialized to zero. The initial learning rate was set to 0.01, and then the ADAM optimizer was used for 100 epochs of training in a batch size of 32. The R 2 And root mean square error (RMSE) to evaluate network performance. 2 It represents the correlation between the estimated result and the true state, and RMSE represents the difference in magnitude between the state estimate and the measurement.
[0046] Then, according to the core idea of UKF algorithm, a symmetric sampling strategy is adopted, using 2n+1 Sigma sampling points {χ′ i}, i = 1, 2, ..., 2n, to approximate the posterior mean and variance of the nonlinear state equation, and the obtained 2 n +1 sampling point is:
[0047]
[0048]
[0049]
[0050] Sigma sampling point Substitute the set into the nonlinear state equation to obtain:
[0051]
[0052] Symmetric sampling corresponding mean weight W i m and variance weight W i c for:
[0053]
[0054] Where K is the Sigma sampling point spacing scale factor of the random variable x, and n is the dimension of the variable x. The mean and variance of the one-step state prediction can be obtained by weighted statistical linear regression technology:
[0055]
[0056]
[0057] According to the calculated state prediction estimate and the prediction variance P k|k-1 Substitute into the sampling formula to get a new set of sampling points and the corresponding mean weight W i (m) and variance weight W i (c)
[0058] Next, use an LSTM directly h(·) Module replacement measurement equation h(·):
[0059]
[0060] According to UT transformation and LSTM R The learned measurement noise R k , and further obtain the one-step prediction of the measured variable Variance P zz，k|k-1 , mutual covariance P xz，k|k-1 And the Kalman gain:
[0061]
[0062]
[0063]
[0064] K k =P xz，k (P zz，k|k-1 ) -1 (16)
[0065] Here, N indicates that the measurement noise covariance increases to N times with each progressive measurement update, and N steps are required to achieve the measurement update, thereby achieving the purpose of adaptive measurement update and progressive filtering, thereby improving the stability of the system and the accuracy of filtering.
[0066] Finally, according to the measurement value zk at time k, the state estimate at time k is obtained and the estimated variance P k|k :
[0067]
[0068]
Claims
1. A method for estimating human upper limb posture based on a progressive unscented Kalman filter network, characterized by: The following steps are involved: Step 1) Data acquisition: Build a data acquisition system to obtain the angle information of the human upper limb motion joints through a visual capture system, and use an electromyographic wristband to collect the electromyographic signals of the forearm during movement; Step 2) Build a model between sEMG signals and upper limb joint angles: x k =f(x k-1 )+w k-1 (1) z k =h(x k )+v k (2) Step 3) Construct an unscented Kalman filter system with LSTM estimation parameters, and use the LSTM neural network to learn the system's measurement equation h(·), system noise Q k and measurement noise R k ; Using UT changes, weighted statistical linear regression is used to approximate the posterior mean of the nonlinear equation and variance P k|k-1 ; Step 4) Transform and measure the noise R according to UT k , and obtain the one-step prediction of the measured variable Variance P zz,k|k-1 and the cross-covariance Px z,k|k-1 ; Step 5) Calculate the Kalman gain according to the UKF calculation steps, and calculate the Kalman gain according to the measured value z at time k k , find the state estimate at time k and the estimated variance P k|k ; A progressive UKF filtering method with adaptive measurement update is added to the UKF to improve the stability of the system; the covariance of the measurement noise is artificially increased to solve the contradiction between stability and filtering accuracy. k , find the state estimate and estimated variance at time k.
2. The method for estimating upper limb posture based on a progressive unscented Kalman filter network according to claim 1, wherein: In step 1), A, B, and C are used to represent the numbers of the human elbow joints, wherein AB represents the upper arm of the experimenter, BC represents the lower arm of the experimenter, and the joint angle of the upper limb elbow joint is the angle θ.
3. The method for estimating upper limb posture based on a progressive unscented Kalman filter network according to claim 1, wherein: In step 1), the visual capture system is composed of 12 cameras, and the sampling frequency of the myoelectric wristband is 200 Hz.
4. The method for estimating upper limb posture based on a progressive unscented Kalman filter network according to claim 1 or 2, characterized in that: In the step 2), and are the n-dimensional upper limb joint state vector and the m-dimensional sEMG signal measurement vector at time k, respectively. f(·) and h(·) are the nonlinear state equation and measurement equation of the system, and w k and v k are the system noise and measurement noise with zero mean, and the corresponding covariance Q k and R k is uncorrelated Gaussian white noise.
5. The method for estimating upper limb posture based on a progressive unscented Kalman filter network according to claim 1, wherein: In the step 3), the unscented Kalman filter system of the LSTM estimation parameters uses the LSTM neural network to learn the UKF part model, and directly obtains the measurement noise, system noise and measurement function from the training data.