A mirror image assist control method of an exoskeleton for estimating lower limb joint torque in real time

Abstract

The application discloses a kind of real-time estimation of lower limb joint torque exoskeleton mirror image assistance control method, belong to exoskeleton robot control technical field.The real-time estimation of lower limb joint torque exoskeleton mirror image assistance control method of the application carries out pre-training on open source dataset using Raindrop graph neural network, then again on self-built dataset migration learning is carried out, to realize the estimation of lower limb joint torque, static information is added during training, improve accuracy and individual difference adaptability;Using mirror image assistance control method, according to the difference between healthy side torque and affected side torque and the last step assistance torque iteration, the real-time assistance torque of affected side is obtained to realize the method of individual real-time mirror image assistance.The exoskeleton mirror image assistance control method of the application can improve the accuracy and individual difference adaptability of lower limb exoskeleton assistance.

Description

TECHNICAL FIELD

[0001] The application relates to a kind of exoskeleton mirror image assistance control method for estimating lower limb joint torque in real time, and belongs to the technical field of exoskeleton robot control. BACKGROUND

[0002] With the aggravation of population aging, the sharp increase in the number of patients with walking disorders has become a worldwide problem. Stroke patients often have unilateral walking disorders after surgery, and during rehabilitation, the affected side muscle spasm and no step ability. Lower extremity exoskeletons are replacing traditional rehabilitation training due to their labor-saving and accurate movement advantages. Studies have shown that wearing a unilateral exoskeleton helps patients with hemiplegia recover gait.

[0003] Traditional lower limb joint torque estimation includes two methods based on surface electromyography (sEMG) and lower limb inverse dynamics model. Among them, the sEMG-based method needs to establish a complex human skeletal muscle activation and contraction dynamics model. On the other hand, sEMG signals are often disturbed by various factors and have poor stability. The inverse dynamics model-based method estimates joint torque by combining human motion posture, plantar pressure and human dynamics model. Compared with the sEMG-based method, this method has more advantages in stability and reliability. However, this method requires a large amount of data and more rigorous preprocessing, and the estimation result of human joint torque is always lagging behind the movement intention generated in the human brain, which makes it impossible to provide timely and accurate assistance.

[0004] In view of the shortcomings of the above two traditional lower limb joint torque estimation methods, researchers have proposed a lower limb joint torque estimation method based on neural network. This method requires collecting a large amount of data in advance to train the model, and can compensate for the shortcomings of the above two traditional methods in actual use: it does not need to use sEMG signals with poor stability, nor does it need complete lower limb joint angles, plantar pressure signals, and it does not need to pre-calibrate the human body model, and it has the ability to estimate joint torque in real time. In the case of lag in computing power, it can still predict the joint torque at the next moment or the joint torque at multiple moments in the future to make up for the time delay caused by calculation and data transmission. For example, Chinese patent CN115416003A discloses "a demand-assisted control method for lower limb exoskeletons for the elderly", in which "statistical methods are used to select reference trajectories and reference torques", i.e. joint torques calculated from a large amount of gait data of healthy people, and reference torques of joints are obtained through data processing. By means of the dynamics model of the lower limb exoskeleton system, a single-hidden-layer feedforward neural network is trained using the back propagation (BP) algorithm to obtain a knee joint torque estimation model based on the human body center of mass. Then, according to the estimated torque and the reference torque, the demand-assisted torque is output in combination with gait recognition, and the body weight, friction, and inertia torque of the exoskeleton robot are compensated. For another example, Chinese patent CN115006206A discloses "a control method for double lower limb muscle force compensation and gait mirror adjustment based on an exoskeleton robot", which realizes double lower limb muscle force compensation and gait adjustment for the wearer through mirror control strategy. Specifically, the method first records the initial electromyographic intensity as a reference, and then calculates the muscle force parameters of both sides from the real-time collected sEMG signals. When the difference between the muscle force parameters of both sides reaches a certain threshold, the controller will adjust the size of the impedance parameter. The larger the impedance parameter of one side, the smaller the allowed trajectory error, and the larger the correction torque of that side, i.e. the passive adjustment trajectory position control mode of that side. For example, when the muscle force of the left leg is greater than that of the right leg, the left leg adopts an active adjustment strategy, and the right leg adopts a passive adjustment strategy, and the theoretical trajectory of the right leg is calculated based on the actual trajectory of the left leg.

[0005] As described above, although the existing neural network-based lower limb joint torque estimation method overcomes the shortcomings of traditional methods, it still has some deficiencies that cannot be ignored: for example, most exoskeletons need to collect the joint torque of healthy people as the reference curve for assistance before assisting. Although this method is simple and easy to implement, the exoskeleton wearer is usually a group of people with certain walking disorders, and the joint torque of healthy people cannot adapt to the current wearer's recovery and walking habits, and cannot provide personalized assistance to patients. And the current torque estimation model does not consider the influence of patient static data, i.e. patient height and weight, on joint torque estimation, which is not conducive to improving torque estimation accuracy and individual difference adaptability. And the method of adjusting the impedance parameter by presetting the threshold of the difference between the bilateral muscle strength parameters, because its strategy is to modify the assistance parameter only when the muscle strength parameter difference exceeds the fixed threshold, and the threshold size is fixed. When the patient's muscle strength parameter difference is between the two thresholds, the assistance parameter is fixed and unchanged, i.e. it cannot provide personalized mirror assistance for patients with muscle strength parameter differences within the set threshold range, making it difficult to achieve precise mirror assistance. And by setting the impedance parameter when the affected side muscle strength is less than the healthy side, the affected side is forced to track the healthy side trajectory, which is very uncomfortable for patients. SUMMARY

[0006] In view of the low accuracy and poor individual difference adaptability of the current lower limb exoskeleton control method, the present application provides a real-time estimation of lower limb joint torque exoskeleton mirror assistance control method, which is applied to a lower limb exoskeleton assistance device to provide assistance to the wearer, and the method comprises:

[0007] Step 1, constructing a joint torque estimation model based on a graph neural network, and pre-training the joint torque estimation model, wherein the database used for training contains static data, dynamic data and joint torque;

[0008] Step 2, collecting the static data and dynamic data of the wearer of the lower limb exoskeleton assistance device to construct an exclusive database, and using the exclusive database to perform transfer learning on the pre-trained joint torque estimation model to obtain an individual model for the wearer of the lower limb exoskeleton assistance device;

[0009] Step 3, using the individual model to estimate the torque of the wearer of the lower limb exoskeleton assistance device to obtain bilateral torque and mirror the healthy side torque to the affected side; when mirroring the healthy side torque to the affected side, an iterative process is used to establish the relationship between the current step assistance curve and the previous step assistance torque, healthy side torque and affected side torque to obtain the current step assistance torque curve;

[0010] Step 4, determining the mirror assistance torque to be provided according to the current step assistance torque curve, and using the lower limb exoskeleton assistance device to provide the corresponding assistance.

[0011] In an embodiment, the step 3 of establishing the relationship between the current step assistance curve and the assistance torque of the previous step, the healthy side torque and the affected side torque in an iterative process to obtain the assistance torque curve of the current step, comprises:

[0012] initial first step assistance torque

[0013] In the iterative process, the current step assistance torque the assistance torque of the previous step the healthy side torque and the affected side torque The relationship is as follows:

[0014]

[0015]

[0016] wherein, indicates the phase.

[0017] In an embodiment, the calculation of the phase in the step 3 is obtained by a phase angle oscillator, ranging from 0 to 100%, and the horizontal coordinate of the phase angle oscillator output phase curve is time, and the vertical coordinate is gait phase;

[0018] In an embodiment, the torque curve of the step 3 is constructed with the phase as the horizontal coordinate and the estimated joint torque as the vertical coordinate, and the torque curves of the healthy side and the affected side are constructed.

[0019] In an embodiment, the graph neural network of the step 1 is a Raindrop graph neural network.

[0020] In an embodiment, the static data of the lower extremity exoskeleton assistance device wearer in the step 2 is collected, including collecting the height, gender, weight and age information of the lower extremity exoskeleton assistance device wearer.

[0021] In an embodiment, the dynamic data of the lower extremity exoskeleton assistance device wearer in the step 2 is collected, including: using a motion analysis platform to collect the dynamic data of the lower extremity exoskeleton assistance device wearer, including electromyographic signals, ground reaction forces, joint angle signals and joint torques calculated by the motion analysis platform; the treadmill of the motion analysis platform has a force plate for measuring plantar pressure, and has a matching electromyographic sensor with 7 on one side, a total of 14, which are: rectus femoris RF, vastus lateralis VL, tibialis anterior TA, soleus SO, semitendinosus ST, biceps femoris BF and gastrocnemius GL.

[0022] In an embodiment, in the step 2, when the pre-trained joint torque estimation model is subjected to transfer learning by using the exclusive database, a full connection layer is added to the last layer of the pre-trained joint torque estimation model before the transfer learning is performed.

[0023] In an embodiment, in the step 3, when the individual model is used to estimate the torque of the lower extremity exoskeleton wearer, only the wearable motion information acquisition device integrated with the exoskeleton is used to acquire the real-time dynamic information of the lower extremity exoskeleton wearer.

[0024] In an embodiment, in the step 1, when the joint torque estimation model is pre-trained, a public database is used.

[0025] In an embodiment, the device calculates the mirror assistance torque to be provided by using the above method, and provides assistance to the lower extremity exoskeleton wearer according to the calculated mirror assistance torque.

[0026] In an embodiment, the device comprises a controller, a fixed waistband, an inertial measurement unit (IMU), a contralateral assistance leg rod and a driving unit; wherein the controller calculates the mirror assistance torque to be provided by using the above method and controls the contralateral assistance leg rod and the driving unit to provide assistance to the lower extremity exoskeleton wearer; the inertial measurement unit (IMU) is used to acquire real-time dynamic information of the lower extremity exoskeleton wearer, and the fixed waistband is used to fix the entire assistance device on the wearer's body.

[0027] The above-mentioned lower extremity exoskeleton assistance device is applied to gait recovery of a lower extremity hemiplegic patient.

[0028] The beneficial effects of the present application are as follows:

[0029] 1) The present application adopts a Raindrop graph neural network model based on Transformer, the weights of the nodes and all connection edges of the graph neural network can simulate the implicit relationship between sensors and can be optimized by computer data training. The model not only solves the problem of poor feature extraction ability of RNN, but also solves the problem of using different types of sensors and different sensor acquisition frequencies, and can realize real-time, accurate and convenient lower extremity joint torque estimation.

[0030] 2) The present application uses a public database on the Internet as basic data for model training, then uses the collected data for transfer learning, and uses static data as neural network model input, which is beneficial to realize personalized mirror assistance.

[0031] 3) On the basis of real-time estimation of bilateral joint torque, the application proposes a method of obtaining real-time assistive torque of the affected side by iteration of the difference between the torque of the healthy side and the torque of the affected side and the assistive torque of the previous step to realize individualized real-time mirror assistance, which can simply and accurately realize individualized mirror assistance and has great advantages in adapting to the individual walking habits of patients and improving gait symmetry. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0033] Figure 1 Structure diagram of lower extremity exoskeleton;

[0034] Figure 2 Training flowchart of Raindrop graph neural network;

[0035] Figure 3A Structure diagram of Raindrop graph neural network;

[0036] Figure 3B Schematic diagram of Raindrop graph neural network training process, in which the values of different sampling frequency sensors are input in time sequence;

[0037] Figure 3C Schematic diagram of Transformer architecture;

[0038] Figure 4 Schematic diagram of motion analysis platform (GRAIL, CAREN);

[0039] Figure 5 Schematic diagram of electromyographic sensor positions (RF, VL, TA, SO, ST, BF, GL) and joint angles (A1-A6);

[0040] Figure 6 Schematic diagram of established sample database;

[0041] Figure 7 Schematic diagram of bilateral gait phase examples;

[0042] Figure 8 Schematic diagram of real-time mirror assistance torque calculation method;

[0043] Figure 9 Instance diagram of walking with exoskeleton. DETAILED DESCRIPTION

[0044] In order to make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0045] Introduction to basic knowledge:

[0046] Lower extremity exoskeleton, such as Figure 1 As shown in the figure, the lower extremity exoskeleton is a power-assisted device composed of a power supply and a controller 1, a fixed waistband 2, an IMU (Inertial Measurement Unit) 3, a diseased side power-assisted leg rod 4 and a driving unit 5. The entire power-assisted device is fixed on the wearer's body through the fixed waistband 2. The IMU measures the biomechanical signals of the wearer and transmits them to the controller 1. The controller 1 estimates the power assistance information needed to be provided according to the wearer's biomechanical signals, and then applies the corresponding power assistance to the wearer's leg through the driving unit 5 and the diseased side power-assisted leg rod 4, so as to realize the power assistance to the wearer and assist the wearer in rehabilitation training and other content.

[0047] Embodiment one:

[0048] A lower extremity exoskeleton mirror image power assistance control method for real-time estimation of joint torque is provided, which is applied to a lower extremity exoskeleton power assistance device to provide power assistance to the wearer. The method comprises:

[0049] Step 1, constructing a joint torque estimation model based on a graph neural network, and pre-training the joint torque estimation model, wherein the database used for training contains static data, dynamic data and joint torque;

[0050] Step 2, collecting the static data and dynamic data of the wearer of the lower extremity exoskeleton power assistance device to construct a special database, and using the special database to perform transfer learning on the pre-trained joint torque estimation model to obtain a personalized model for the wearer of the lower extremity exoskeleton power assistance device;

[0051] Step 3, using the personalized model to estimate the torque of the wearer of the lower extremity exoskeleton power assistance device to obtain bilateral torque and mirror the healthy side torque to the diseased side; when mirroring the healthy side torque to the diseased side, the relationship between the current step power curve and the last step power torque, the healthy side torque and the diseased side torque is established in an iterative process to obtain the power torque curve of the current step;

[0052] Specifically, the initial first step power torque In the iterative process, the current step power torque The last step power torque The healthy side torque And the diseased side torque The relationship is as follows:

[0053]

[0054]

[0055] wherein, represents the gait phase, which is obtained by a phase angle oscillator, and ranges from 0 to 100%. The horizontal coordinate of the phase curve output by the phase angle oscillator is time, and the vertical coordinate is gait phase; the torque curve of the healthy side and the affected side is constructed by taking phase as the horizontal coordinate and the estimated joint torque as the vertical coordinate.

[0056] Step 4, determining the mirror assistance torque to be provided according to the assistance torque curve of the current step, and providing the corresponding assistance by using the lower limb exoskeleton assistance device.

[0057] Embodiment Two:

[0058] The embodiment provides an exoskeleton mirror assistance control method for real-time estimation of lower limb joint torque, and the method comprises the following steps:

[0059] First, construct a joint torque estimation model based on a graph neural network, and pre-train the generalization model.

[0060] Download the open source database, pre-train the generalization model, and the schematic diagram of training the model is as shown in Figure 2 , and the established graph neural network structure is as shown in Figures 3A-3C .

[0061] The open source database used in the embodiment comprises:

[0062] Database 1, HuMoD (paper: Sci-Hub | HuMoD-A versatile and open database for the investigation, modeling and simulation of human motion dynamics on actuation level. 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids) | 10.1109 / HUMANOIDS.2015.7363534. Database website: wojtusch / HuMoD at 3c2926764c4e7e68bf6d7ad55640a418986f0fc7 (github.com)), which contains raw and processed biomechanical measurement data from a three-dimensional motion capture system, an instrumented treadmill system and an electromyography measurement system, including 8 different motion tasks performed by healthy female and male subjects, and anthropometric parameters of two subjects.

[0063] Database 2 (Paper: A comprehensive, open-source dataset of lower limb biomechanics in multiple conditions of stairs, ramps, and level-ground ambulation and transitions - ScienceDirect. Database URL: A comprehensive, open-source dataset of lower limb biomechanics in multiple conditions and of stairs, ramps, and level-ground ambulation and transitions - (gatech.edu)). This database contains three-dimensional biomechanics and wearable sensor data from 22 able-bodied adults for multiple movement patterns (level-ground / treadmill walking, stair ascent / descent, and ramp ascent / descent) and multiple terrain conditions for each pattern (walking speed, stair height, and ramp incline).

[0064] Database 3 (Paper: IMU and Smartphone Camera Fusion for Knee Adduction and Knee Flexion Moment Estimation During Walking | IEEE Journals & Magazine | IEEE Xplore. Database URL: TheOne-1 / KAM_and_KFM_Estimation: KAM and KFM Estimation via IMU-Camera Fusion (github.com)), which records the walking data of 17 subjects with 8 IMUs and 2 smartphone cameras.

[0065] The present application aims to meet the demand for providing a personalized lower limb exoskeleton control method, and constructs a joint moment estimation model based on a graph neural network as shown in Figures 3A-3C The estimation model is jointly constituted by static data, dynamic data, and joint moment to form a joint moment database of the joint moment estimation model based on the graph neural network.

[0066] Secondly, data is collected to establish a dedicated database.

[0067] The motion analysis platform used is Gait Real-time Analysis Interactive Lab (GRAIL), as shown in Figure 4 The dynamic signals collected include electromyography signals, ground reaction forces, and joint angle signals. The treadmill of the motion analysis platform has force plates to measure plantar pressures, and has 14 electromyography sensors on one side, including rectus femoris (RF), vastus lateralis (VL), tibialis anterior (TA), soleus (SO), semitendinosus (ST), biceps femoris (BF), and gastrocnemius (GL). The wearing positions of the sensors are shown in Figure 5 In addition, the GRAIL platform is equipped with a camera. After the white collection points are pasted on the leg, the joint angles of the lower limbs can be obtained through machine vision, as shown in Figure 5 A1-A6 represent the hip, knee, and ankle joint angles of the left and right sides, respectively. The static data includes the patient's height, gender, weight, and age. The established database is shown in Figure 6

[0068] Step 3: Transfer learning.

[0069] The pre-trained Raindrop graph neural network model obtained in the first step is used. A fully connected layer is added to the last layer of the original model, and fine-tuning is performed on the database obtained in the second step.

[0070] Step 4: Real-time joint torque estimation.

[0071] In actual use, the wearable motion information collection device (WT901C-TTL, Shenzhen Weite Intelligent Technology Co., Ltd.) integrated with the exoskeleton is used, and the collected motion information is transmitted to the trained neural network model in real time to output the estimated bilateral joint torque.

[0072] Step 5: Real-time gait phase calculation.

[0073] The gait phase obtained in the example is shown in Figure 7 The horizontal axis is time, and the vertical axis is gait phase, which is dimensionless.

[0074] The horizontal axis of the torque curve is the gait phase (range: 0-100%), which is obtained by continuous gait phase estimation (phase angle oscillator); the vertical axis is the joint torque, which is obtained by real-time joint torque estimation.

[0075] Step 6: Real-time mirror-assisted torque calculation.

[0076] As shown in Figure 8 ​As shown, the bilateral torques are estimated using the model and the healthy side torque is mirrored to the affected side. Specifically, the neural network model inputs the current gait motion data and patient information, and outputs bilateral torque estimation data. Due to the timing difference between the healthy side and the affected side motion, the abscissa (phase) of the healthy side torque curve is shifted by 50%. The assist torque curve designed by the present application Initial last step assist torque After the assist is completed And in the iteration process, the new step assist torque curve With the last step assist torque Healthy side torque And the affected side torque Related to, as shown in the following formula:

[0077]

[0078]

[0079] The resulting mirrored assist torque On the basis of compensating for the gravity and inertial torque of the exoskeleton itself, it can be transmitted to the exoskeleton motor controller in real time for torque tracking, so as to accurately apply to the human-exoskeleton whole

[0080] Step 7, output the assist torque.

[0081] The calculated real-time mirrored assist torque is compensated for the gravity and inertial torque of the exoskeleton itself, and is output to the motor end for bottom-level torque control, and is applied to the human-exoskeleton system, as Figure 9 Shown.

[0082] Some steps in the embodiments of the present application can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk, etc.

[0083] The above only describes the preferred embodiments of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A lower extremity exoskeleton assistance device, characterized by, The device adopts a mirror image assistance control method of an exoskeleton to estimate joint torque of a lower limb in real time, calculates a mirror image assistance torque to be provided, and provides assistance to a wearer of a lower limb exoskeleton assistance device according to the calculated mirror image assistance torque. The method comprises: Step 1, constructing a joint torque estimation model based on a graph neural network, and pre-training the joint torque estimation model, wherein a database used for training contains static data, dynamic data and joint torque; Step 2, collecting static data and dynamic data of a wearer of a lower limb exoskeleton assistance device to construct a special database, and performing transfer learning on the pre-trained joint torque estimation model by using the special database to obtain a personalized model for the wearer of the lower limb exoskeleton assistance device; Step 3, using the personalized model to estimate torque of the wearer of the lower limb exoskeleton assistance device to obtain bilateral torque and mirror the healthy side torque to the affected side; when the healthy side torque is mirrored to the affected side, a relationship between a current step assistance curve and an assistance torque of a previous step, the healthy side torque and the affected side torque is established in an iterative process to obtain an assistance torque curve of the current step; Step 4, determining a mirror image assistance torque to be provided according to the assistance torque curve of the current step, and providing corresponding assistance by using the lower limb exoskeleton assistance device; In the step 3, the relationship between the current step assistance curve and the assistance torque of the previous step, the healthy side torque and the affected side torque is established in the iterative process to obtain the assistance torque curve of the current step, which comprises: initial first step assist torque ; Current step assist torque Previous step assist torque Healthy side torque Affected side torque Relationships, as follows: wherein, The gait phase is represented by a phase angle oscillator, which is in the range of 0-100%. The horizontal coordinate of the phase angle oscillator output phase curve is time, and the vertical coordinate is gait phase. The torque curve of the healthy side and the affected side is constructed with the phase as the horizontal coordinate and the estimated joint torque as the vertical coordinate.

2. The apparatus of claim 1, wherein, In the step 2, the static data of the wearer of the lower limb exoskeleton assistance device is collected, which comprises collecting height, gender, weight and age information of the wearer of the lower limb exoskeleton assistance device.

3. The apparatus of claim 1, wherein, In the step 2, the dynamic data of the wearer of the lower limb exoskeleton assistance device is collected, which comprises collecting dynamic data of the wearer of the lower limb exoskeleton assistance device by using a motion analysis platform, including electromyographic signals, ground reaction forces, joint angle signals and joint torques calculated by the motion analysis platform; the running platform of the motion analysis platform has a force plate for measuring plantar pressure and has 14 electromyographic sensors on one side, including 7 sensors for rectus femoris RF, vastus lateralis VL, tibialis anterior TA, soleus SO, semitendinosus ST, biceps femoris BF and gastrocnemius GL.

4. The apparatus of claim 1, wherein, In the step 2, when performing transfer learning on the pre-trained joint torque estimation model by using the special database, a fully connected layer is added to the last layer of the pre-trained joint torque estimation model before performing the transfer learning.

5. The apparatus of claim 1, wherein, In the step 3, when the personalized model is used to estimate torque of the wearer of the lower limb exoskeleton assistance device, only wearable motion information collection devices integrated with the exoskeleton are used to collect real-time dynamic information of the wearer of the lower limb exoskeleton assistance device.

6. The apparatus of claim 1, wherein, In the step 1, the joint torque estimation model is pre-trained by using a public database.

7. The apparatus of claim 1, wherein, The device comprises a controller, a fixed waist belt, an inertial measurement unit (IMU), a paretic side assistive leg rod and a driving unit; wherein the controller calculates the mirror assistive torque to be provided by steps 1-4 and controls the paretic side assistive leg rod and the driving unit to provide assistance to the lower extremity exoskeleton assistance device wearer; the inertial measurement unit (IMU) is used to collect real-time dynamic information of the lower extremity exoskeleton assistance device wearer, and the fixed waist belt is used to fix the entire assistance device on the wearer's body.

Images