Human walking lower limb muscle force solving device and method based on generative adversarial network and reinforcement learning

Abstract

The application discloses a human walking lower limb muscle force solving device and method based on a generative adversarial network and reinforcement learning, which comprises the following steps: introducing model joint angles and human walking joint angles into a generative adversarial network for training, and outputting part of a reward function required by reinforcement learning from the generative adversarial network; obtaining part of a reward function related to human motion effects through kinematics data and dynamics data generated by a muscle-skeleton model; obtaining part of a reward function related to muscle coordination by analyzing surface electromyogram signals and dynamics data obtained by the muscle-skeleton model; and introducing the three parts of reward functions into a policy network for training, so as to obtain muscle force calculation results capable of reflecting muscle coordination. The application can calculate muscle forces of main lower limb muscles in the walking process of different individuals, and provides a theoretical basis for understanding of human motion rules, motion rehabilitation and development of medical equipment, control of humanoid robots and the like.

Description

Technical Field

[0001] This invention belongs to the field of human dynamics calculation technology, specifically relating to a device and method for solving the muscle force of the lower limbs during human walking based on generative adversarial networks and reinforcement learning. Background Technology

[0002] Calculating muscle force during human walking is crucial for understanding the laws governing human movement. However, calculating muscle force during multi-degree-of-freedom human movements remains a challenge, primarily due to the following reasons:

[0003] (1) The shape and structure of muscles are very complex, making it difficult to perform mechanical analysis and to directly measure muscle force using force sensors.

[0004] (2) In multi-degree-of-freedom human motion, the problem of calculating muscle force from joint torque is a multi-solution problem, and it is difficult to solve muscle force by solving dynamic equations.

[0005] (3) The human muscular system is a whole, and there is a certain synergistic relationship between muscles. It is difficult to calculate muscle force by measuring the data of each muscle during exercise.

[0006] (4) Different individuals have different movement styles, which means that the muscle strength calculated by general methods is not applicable to other individuals and is not universal.

[0007] The calculation of muscle force during human walking can be broadly categorized into three methods: electromyography (EMG) driven method, forward dynamics method, and inverse dynamics method. The EMG driven method primarily measures the surface electromyography (sEMG) signals of relevant muscles during movement and establishes a mathematical model to calculate muscle force from the sEMG signals. The forward dynamics method inputs neural signals into a musculoskeletal model, aiming to minimize energy dissipation during walking, enabling the model to generate various human movements. The muscle force of this model is then used as the corresponding muscle force during movement. The inverse dynamics method first calculates the joint torques of each joint during movement using kinematic data and plantar pressure, and then uses optimization methods to address the redundancy problem in translating joint torques into muscle force.

[0008] Most existing methods focus on ensuring that each muscle satisfies certain kinetic relationships or using optimization methods to determine muscle forces that meet certain goals and constraints, without considering the synergistic relationships between muscles. Furthermore, due to differences in habits and physiological parameters, each individual often develops their own unique movement style, a point that previous studies have not taken into account. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a device and method for solving the lower limb muscle force during human walking using generative adversarial networks and reinforcement learning, which is used to calculate the muscle force of the main muscles of the lower limbs during the walking process of different individuals.

[0010] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0011] A method for solving lower limb muscle forces during human walking based on generative adversarial networks and reinforcement learning:

[0012] Muscle activity and plantar pressure are imported into a musculoskeletal model to generate kinematic and dynamic data for the model. The kinematic data includes joint angles and joint angular velocities, and the dynamic data includes joint torques and muscle forces.

[0013] The joint angles of the model and the joint angles of human walking are imported into the generative adversarial network for training, and the generative adversarial network outputs the part of the reward function needed for reinforcement learning.

[0014] Using kinematic and dynamic data generated from musculoskeletal models, a partial reward function related to human movement performance is obtained:

[0015]

[0016] Where J(t) is the Jacobian matrix from the muscle connection point to the generalized coordinates at the current time, T is the transpose sign, and k f Here are the weighting coefficients, f(t) represents the muscle force vector, and τ(t) represents the joint torque vector. i (t) represents the muscle force of muscle i;

[0017] By analyzing surface electromyography signals and kinetic data obtained from musculoskeletal models, a partial reward function related to muscle synergy was derived:

[0018]

[0019] Among them, u x u y These are two sets of muscle electromyographic signal data with a synergistic relationship, where ρ is the correlation coefficient and k is the coefficient. xy This indicates that muscles x and y have a synergistic relationship;

[0020] The three reward functions are imported into the policy network for training to obtain muscle force calculation results that reflect muscle synergy.

[0021] Furthermore, the muscle activity is calculated by using an activation kinetics model to analyze the processed surface electromyography signal u(t).

[0022]

[0023] Where: A is the nonlinear shape coefficient, a i (t) is the activity of muscle i calculated from surface electromyography signals.

[0024] Furthermore, the kinematic data is generated using a dynamic optimization method, and the dynamic data is calculated using a forward dynamics method.

[0025] Furthermore, the loss function of the generative adversarial network is:

[0026]

[0027] Wherein, the letter V represents the form of cross-entropy; This represents all collected data, P. c When, the expected value of logD(q); P represents data generated from a musculoskeletal model. m When, the expectation of log(1-D(q)) is given; q represents the joint angle.

[0028] Furthermore, the musculoskeletal model selects 14 muscles involved in human sagittal plane walking, 7 for each leg, including: tibialis anterior, gastrocnemius, soleus, rectus femoris, hamstrings, gluteus maximus, and psoas major. The tibialis anterior, soleus, rectus femoris, gluteus maximus, and psoas major are single-joint muscles, while the gastrocnemius and hamstrings are double-joint muscles.

[0029] Furthermore, the partial reward function required for reinforcement learning of generative adversarial networks is:

[0030] r style = -log[1-D(q)]

[0031] Where D(q) is the discriminator of the generative adversarial network.

[0032] Furthermore, the complete reward function, composed of the three reward functions, is as follows:

[0033] r = ω model r model +ω style r style +ω synergy r synergy

[0034] Where, ω model ω style ω synergy The weights of the three reward functions.

[0035] A device for solving lower limb muscle forces during human walking based on generative adversarial networks and reinforcement learning, comprising:

[0036] The human body data acquisition module is used to acquire muscle activity, plantar pressure, joint angles and joint angular velocities during human walking.

[0037] The motion style generation module generates a portion of the reward function needed for reinforcement learning by outputting the adversarial network.

[0038] The muscle synergy extraction module analyzes surface electromyography signals and dynamic data obtained from musculoskeletal models to obtain partial reward functions related to muscle synergy.

[0039] The policy network training module imports the three reward functions into the policy network for training, and obtains muscle force calculation results that reflect the muscle synergy relationship.

[0040] In the above technical solution, the data acquisition module includes a motion capture camera and an electromyography (EMG) signal acquisition device; the motion capture camera acquires the coordinates of marker points on the human body and sends them to a PC to generate the joint angles and angular velocities of the human body during walking; the EMG signal acquisition device acquires the surface EMG signals of the human body during walking and sends them to a PC for processing to generate muscle activity.

[0041] In the above technical solution, the plantar pressure is obtained by measuring a force plate set on the treadmill.

[0042] The beneficial effects of this invention are as follows: This method trains different individuals to develop specific walking styles through generative adversarial networks. These walking styles are then combined with the dynamic data of a musculoskeletal model and muscle synergy relationships to generate a reward function for a reinforcement learning policy network, thereby obtaining lower limb muscle strength data for human walking with specific styles. This invention provides a biomechanical basis for understanding the laws of human movement, sports rehabilitation training and guidance, and motion control of humanoid robots. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the musculoskeletal model and the set muscle force described in this invention;

[0044] Figure 2 This is a schematic diagram of the muscle force solution process described in this invention;

[0045] Figure 3 This is a diagram showing the muscle force calculation results described in this invention. Detailed Implementation

[0046] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0047] Figure 1This is a schematic diagram of the human lower limb muscles studied in this invention, and a musculoskeletal model established by combining the set muscles with information on human limbs and joints.

[0048] This invention selected 14 muscles that are mainly involved in human sagittal plane walking, 7 in each leg, including: tibialis anterior (TA), gastrocnemius (GAS), soleus (SOL), rectus femoris (RF), hamstrings (HAM), gluteus maximus (GLU), and psoas major (ILPOS). Among them, the tibialis anterior, soleus, rectus femoris, gluteus maximus, and psoas major are single-joint muscles; the gastrocnemius and hamstrings are double-joint muscles, spanning two joints.

[0049] Muscle force can be considered as a vector, originating at the junction of the muscle and skeleton, and directed from the junction to the midpoint of the muscle. Since muscles only contract during movement, muscle force only travels in the direction of muscle contraction. This invention uses a musculoskeletal model as the agent trained in reinforcement learning.

[0050] like Figure 2 As shown, this invention discloses a device for solving lower limb muscle force during human walking using generative adversarial networks and reinforcement learning. The device includes a human data acquisition module, a movement style generation module, a muscle coordination extraction module, and a policy network training module. The human data acquisition module includes a motion capture camera and an electromyography (EMG) signal acquisition device (consisting of electrodes mounted on the human body and an EMG signal receiving device). The module acquires human data while walking on a treadmill. A force plate is also mounted on the treadmill to measure the plantar pressure during walking, and the data is processed by a PC.

[0051] Within the framework of reinforcement learning, the process begins by acquiring surface electromyography (sEMG) signals of the human body during walking using an electromyography (EMG) acquisition device. This data is then sent to a PC for processing. A kinetic model is activated to generate muscle activity 'a'. A motion capture camera is used to capture the coordinates of marker points on the human body, which are then sent to the PC to generate joint angles and angular velocities for walking. Next, the muscle activity and plantar pressure are imported into a musculoskeletal model to generate kinematic data (joint angles and angular velocities) and dynamic data (including joint torques and muscle forces). The joint angles of the model and the human body during walking are then fed into a generative adversarial network (GAN) for training. The GAN's generator produces a specific human movement style and outputs a partial reward function representing this style, necessary for reinforcement learning. Simultaneously, the kinematic and dynamic data generated from the musculoskeletal model yield a partial reward function related to the human movement effect. Furthermore, by analyzing the acquired SEMG signals and the dynamic data from the musculoskeletal model, a reward function related to muscle coordination is obtained. Finally, these three reward functions are fed into a policy network for training, resulting in muscle force calculations that reflect specific muscle coordination relationships.

[0052] The specific steps are as follows:

[0053] Step (1) Collect surface electromyography signals, kinematic data and plantar pressure during human walking process, and import them into a PC for filtering and normalization processing;

[0054] Step (2) involves calculating the muscle activity of the relevant muscles using the processed surface electromyography (EMG) signal u(t) through an activation kinetic model. The process of obtaining muscle activity from the EMG signal is as follows:

[0055]

[0056] Where: A is the nonlinear shape coefficient, representing the degree of nonlinearity between neural signals and muscle activation, a i (t) is the activity of muscle i calculated from surface electromyography signals;

[0057] Step (3) import muscle activity and plantar pressure into the established musculoskeletal model, calculate the muscle force and joint torque of the model through positive dynamics, and generate kinematic data of the entire walking process, including joint angles and joint angular velocities, through dynamic optimization.

[0058] Step (4) involves importing the joint angles generated by the model and the acquired joint angles into a Generative Adversarial Network (GAN) for training to generate the motion style of a specific individual; the loss function of the GAN network is:

[0059]

[0060] Wherein, the letter V represents the form of cross-entropy; This represents all collected data, P. c When, the expected value of logD(q); P represents data generated from a musculoskeletal model. m At that time, log(1-D(q)) represents the expectation; q represents the joint angle; D(q) is the discriminator of the GAN network, which is related to the kinematic data collected and the kinematic data generated by the musculoskeletal model, and can represent the movement style of a specific individual. D(q) is continuously updated throughout the training process. The output of the trained discriminator D(q) is used as the input of the policy network, that is, as part of the reward function of the reinforcement learning policy network. The specific expression of this reward function is:

[0061] r style = -log[1-D(q)]

[0062] Step (5): The reward function of the musculoskeletal model is obtained by positive dynamics calculation and input to the policy network. The expression of the reward function of the model is:

[0063]

[0064] Where J(t) is the Jacobian matrix from the muscle connection point to the generalized coordinates at the current moment, and T is the transpose; the first term of the reward function represents minimizing the L2 norm of the absolute value of the difference between the current muscle force vector multiplied by the transpose of the Jacobian matrix and the current joint torque vector, i.e., the error between the joint torque that the current muscle force can provide and the actual joint torque; the second term represents minimizing the sum of the muscle forces at the current moment, which is physiologically based on the fact that when people walk, they tend to perform rhythmic movements in a way that minimizes energy dissipation. In terms of muscle force, this can be approximated as minimizing the sum of muscle forces at all times during walking; k f For weighting coefficients, adjust k f The weights of the two optimization objectives can be adjusted; f(t) represents the muscle force vector, τ(t) represents the joint torque vector, f i (t) represents the muscle force of muscle i.

[0065] Step (6): Calculate the correlation coefficients of all acquired surface electromyography signals, select the correlation coefficients of muscles with synergistic relationships, and generate a partial reward function r for the input policy network. synergy r synergy The expression is:

[0066]

[0067] In this invention, the method for examining muscle synergy is based on measuring two sets of electromyographic (EMG) signal data of muscles with a synergistic relationship. x u y The correlation coefficient ρ was used for analysis. The muscle electromyography (EMG) signal data were derived from 14 sets of collected muscle EMG signal data.

[0068] Where m represents the number of muscle synergies during the entire walking process, x and y are the subscripts of the electromyographic signals of the two sets of muscles with a synergistic relationship, and k is the boundary of the correlation coefficient between the two sets of muscle electromyographic signals with a synergistic relationship, that is, the correlation coefficient between the two sets of muscle electromyographic signals is greater than k. xy This means that the x and y muscles are considered to have a synergistic relationship.

[0069] Step (7) involves inputting the generated three partial reward functions into the policy network for training. The complete reward function expression is as follows:

[0070] r = ω model r model +ω style r style +ω synergy r synergy

[0071] Where, ω model ω style ω synergy The weights of the three reward functions.

[0072] The policy network consists of two identical fully connected layers connected by an activation function. In addition to the reward function, surface electromyography (SEMG) signals, model-generated kinematic data, and collected kinematic data are also input into the policy network as state inputs for reinforcement learning. During training, the parameters of the policy network are continuously updated until the reward function reaches its maximum value under the distribution of input states.

[0073] Based on the above methods, this invention can calculate the muscle strength of the main muscles of the lower limbs when a person with a specific walking style walks. Figure 3 This is a graph showing the muscle force calculated according to the present invention. The horizontal axis represents the percentage of gait cycle time, and the vertical axis represents the percentage of body weight. Analysis of the calculated results, combined with knowledge of human kinesiology and muscle synergy, reveals that the muscle force calculation results basically conform to the laws of human movement.

[0074] The embodiments described above are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for solving lower limb muscle force in human walking based on generative adversarial networks and reinforcement learning, characterized in that: Muscle activity and plantar pressure are imported into a musculoskeletal model to generate kinematic and dynamic data for the model. The kinematic data includes joint angles and joint angular velocities, and the dynamic data includes joint torques and muscle forces. The joint angles of the model and the joint angles of human walking are imported into the generative adversarial network for training, and the generative adversarial network outputs the part of the reward function needed for reinforcement learning. Using kinematic and dynamic data generated from musculoskeletal models, a partial reward function related to human movement performance is obtained: Where J(t) is the Jacobian matrix from the muscle connection point to the generalized coordinates at the current time, T is the transpose sign, and k f Here are the weighting coefficients, f(t) represents the muscle force vector, and τ(t) represents the joint torque vector. i (t) represents the muscle force of muscle i; By analyzing surface electromyography signals and kinetic data obtained from musculoskeletal models, a partial reward function related to muscle synergy was derived: Among them, u x u y These are two sets of muscle electromyographic signal data with a synergistic relationship, where ρ is the correlation coefficient and k is the coefficient. xy This indicates that muscles x and y have a synergistic relationship; The three reward functions are imported into the policy network for training to obtain muscle force calculation results that reflect muscle synergy.

2. The method for solving the lower limb muscle force of human walking based on generative adversarial networks and reinforcement learning according to claim 1, characterized in that, The muscle activity is calculated by using a processing surface electromyography signal u(t) through an activation kinetic model: Where: A is the nonlinear shape coefficient, a i (t) is the activity of muscle i calculated from surface electromyography signals.

3. The method for solving the lower limb muscle force of human walking based on generative adversarial networks and reinforcement learning according to claim 1, characterized in that, The kinematic data is generated using a dynamic optimization method, and the dynamic data is calculated using a forward dynamics method.

4. The method for solving the lower limb muscle force of human walking based on generative adversarial networks and reinforcement learning according to claim 1, characterized in that, The loss function of the generative adversarial network is: Wherein, the letter V represents the form of cross-entropy; This represents all collected data, P. c When, the expected value of logD(q); P represents data generated from a musculoskeletal model. m When, the expectation of log(1-D(q)) is given; q represents the joint angle.

5. The method for solving the lower limb muscle force of human walking based on generative adversarial networks and reinforcement learning according to claim 1, characterized in that, The musculoskeletal model selected 14 muscles involved in human sagittal plane walking, 7 for each leg, including: tibialis anterior, gastrocnemius, soleus, rectus femoris, hamstrings, gluteus maximus, and psoas major. The tibialis anterior, soleus, rectus femoris, gluteus maximus, and psoas major are single-joint muscles, while the gastrocnemius and hamstrings are double-joint muscles.

6. The method for solving the lower limb muscle force of human walking based on generative adversarial networks and reinforcement learning according to claim 1, characterized in that, The partial reward function required for reinforcement learning of generative adversarial networks is: r style =-log[1-D(q)] Where D(q) is the discriminator of the generative adversarial network.

7. The method for solving the lower limb muscle force of human walking based on generative adversarial networks and reinforcement learning according to claim 6, characterized in that, The complete reward function, consisting of the three reward functions, is as follows: r=ω model r model +oh style r style +oh synergy r synergy Where, ω model ω style ω synergy The weights of the three reward functions.

8. An apparatus for implementing the method for solving the lower limb muscle force of human walking based on generative adversarial networks and reinforcement learning as described in any one of claims 1-7, characterized in that, include: The human body data acquisition module is used to acquire muscle activity, plantar pressure, joint angles and joint angular velocities during human walking. The motion style generation module generates a portion of the reward function needed for reinforcement learning by outputting the adversarial network. The muscle synergy extraction module analyzes surface electromyography signals and dynamic data obtained from musculoskeletal models to obtain partial reward functions related to muscle synergy. The policy network training module imports the three reward functions into the policy network for training, and obtains muscle force calculation results that reflect the muscle synergy relationship.

9. The apparatus according to claim 8, characterized in that, The data acquisition module includes a motion capture camera and an electromyography signal acquisition device; the motion capture camera acquires the coordinates of marker points on the human body and sends them to a PC to generate the joint angles and joint angular velocities of the human body during walking. The electromyography (EMG) signal acquisition device collects surface EMG signals from the human body during walking and sends them to a PC for processing to generate muscle activity data.

10. The apparatus according to claim 8, characterized in that, The plantar pressure is obtained by measuring a force plate set on the treadmill.

Images