Muscle strength prediction method and device based on fusion of physical model and deep learning

Abstract

The invention discloses a muscle strength prediction method and device based on fusion of a physical model and deep learning, and the method comprises the steps: taking the physical information of neuromusculoskeletal as the soft constraint of a data-driven model, fully considering the dynamic time relation of muscle strength distribution, converting the time series input data into time slice data, and carrying out the prediction of the muscle strength. Kinematic data and EMG signals are used as input, advanced features are automatically extracted through a deep learning algorithm, the relation between the input signals and muscle force is established, muscle force prediction meeting a kinematic equation is achieved, and the defect that an existing model for muscle force calculation generally does not contain definite physical modeling for the potential neuromechanical process, and the calculation accuracy is poor is overcome. And the physical mechanism in the joint movement process is not reflected.

Description

Technical Field

[0001] The present invention relates to the technical field of muscle strength assessment, and in particular to a muscle strength prediction method and device based on a physical model integrated with deep learning. Background Art

[0002] Skeletal muscle and joint torque are important components of human biomechanics. Skeletal muscle is the main driving force of human movement, and joint torque provides basic information about the neuromotor control mechanism during movement. In the field of biomechanics, musculoskeletal models are often used to approximate the mapping from motion space to force space through the framework of classical mechanics. In previous studies, whether for simple repetitive movements (such as gait) or complex movements (such as jumping), most of the musculoskeletal models are based on physics-based modeling techniques to explain the conversion between neural excitation, muscle dynamics, and joint kinematics and dynamics. By using experimental records such as electromyography (EMG), ground reaction force (GRF), and segmental body kinematics, these musculoskeletal models can achieve non-invasive estimation of physiological quantities such as muscle force and joint torque.

[0003] Since the number of muscles involved in the movement process is greater than the degrees of freedom of the joints, optimization-driven schemes are usually established to estimate skeletal muscle force by optimizing multiple objective functions. Electromyography (EMG)-driven optimization has also been developed to more accurately describe the behavior of skeletal muscles during dynamic movement. However, for complex models in high-dimensional space, the above methods cannot match the speed of complex movements due to the computational complexity and slowness of static optimization. Supervised learning methods of artificial intelligence algorithms have gradually been developed to map joint time series to muscle tension by using interconnected artificial neural networks that mimic the functions of the central nervous system instead of mathematical equations in musculoskeletal modeling.

[0004] However, these models were built without explicit physical modeling of the underlying neuromechanical processes, and the intermediate functional relationships cannot reflect the physical mechanisms of the observed variables. Summary of the invention

[0005] Based on the problems raised by the above background technology, the purpose of the present invention is to provide a muscle strength prediction method and device that integrates physical models with deep learning, fully consider the dynamic behavior of time series in the process of solving muscle strength distribution during dynamic joint movement, build a physics-based deep learning framework for musculoskeletal modeling, develop a deep learning algorithm suitable for neuromusculoskeletal model modeling, use physical information as soft constraints of data-driven models, and provide a method for mapping joint kinematics to motion muscle space, which solves the problem that the model established by the existing muscle strength calculation method lacks consideration of the underlying neuromechanical process, making the model unable to reflect the physical movement mechanism of muscle strength and thus leading to inaccurate muscle strength calculation.

[0006] The present invention is achieved through the following technical solutions:

[0007] The first aspect of the present invention provides a muscle strength prediction method integrating physical model with deep learning, comprising the following steps:

[0008] constructing a neuromusculoskeletal model, and obtaining EMG signals using the neuromusculoskeletal model;

[0009] Constructing a deep neural network model including a recursive module and a convolutional module, obtaining the joint angle, inputting the joint angle and the EMG signal into the deep neural network model for prediction, and obtaining the predicted joint torque;

[0010] A total loss function of a deep neural network model is constructed, the deep neural network model is trained using the total loss function, and the trained deep neural network model is used to calculate muscle strength; the total loss function includes a neural network loss and a physical information loss, wherein the physical information loss is constructed using the predicted joint torque.

[0011] In the above technical solution, a neuromusculoskeletal model is constructed, and the physical information of the neuromusculoskeletal system is used as the soft constraint of the data-driven model. The neuromusculoskeletal model is used to simulate the dynamic movement of the joints. By simulating the dynamic movement of the joints, EMG signals are generated, which reflect the neuromotor control mechanism during the dynamic movement of the joints.

[0012] The EMG signal and the joint angle are combined as input data and input into the deep neural network model, which includes a recursive module and a convolutional module, wherein the recursive module is used to convert the input data into time slice data, and the convolutional module is used to read the characteristics of the neuromotor control mechanism in the dynamic movement of the joint in the EMG signal, so as to establish the relationship between kinematic information and EMG signals and muscle strength, and realize the deep mapping to skeletal muscle strength. The EMG signal and the joint angle are input, and the nonlinear characteristics of the sequence data can be effectively learned through the recursive module, and the temporal relationship between the EMG signal and the joint angle is effectively processed. The convolutional module reduces the complexity of data reconstruction through weight sharing, and improves the ability of feature extraction and the speed of classification.

[0013] In order to ensure that the muscle force calculation results meet the physical motion equations, while constructing the muscle force solution model, the physics-based domain knowledge is introduced as a soft constraint into the data-driven model to standardize the data-driven model. By comparing the actual joint torque with the predicted joint torque, the target torque that meets the dynamic balance condition and boundary condition is found, and the target torque and the joint angle corresponding to the target torque are used to train the deep neural network model. The trained deep neural network model is used to calculate muscle force, and the results obtained are closer to the musculoskeletal situation in the real movement process under the neuromotor control mechanism in the dynamic movement of the joint. This method overcomes the problem that the current models used for muscle force calculation usually do not contain explicit physical modeling of the underlying neuromechanical process, lack reflection of the physical mechanism in the joint movement process, and therefore cannot fully reflect the physical mechanism behind the observed variables.

[0014] In an optional embodiment, constructing a neuromusculoskeletal model includes:

[0015] The muscle-tendon is simplified into a contraction element, a series elastic element and a parallel elastic element, and the contraction element, the series elastic element and the parallel elastic element are used to establish a muscle-tendon execution unit force relationship; wherein the muscle-tendon execution unit force relationship is established as follows:

[0016] F T =F CE =(F CE +F PE )cosα

[0017] In the above formula, F T is the main force of the tendon, F CE is the contraction force of the active elastic contraction element, F PE is the parallel elastic element contraction force, and α is the feathering angle.

[0018] In an optional embodiment, the recursive module includes n recursive layers;

[0019] Among them, when there is only a single subject and the motion state is unique, n is 1;

[0020] When the motion state is unique but there are multiple subjects, n is 2;

[0021] When there are multiple subjects or the motion state is not unique, n is 3;

[0022] When there are multiple subjects and the motion state is not unique, n is 4.

[0023] In an optional embodiment, the total loss function includes muscle force loss and joint angle loss;

[0024] Wherein, the expression of muscle force loss is as follows:

[0025] L F =MSE(F)

[0026]

[0027] In the above formula, L F Represents the loss of muscle force, F t n represents the true value of the corresponding muscle force at time t, represents the predicted muscle force; N represents the total number of muscles at the target joint;

[0028] The expression of the joint angle loss is as follows:

[0029] L θ =MSE(θ)

[0030]

[0031] In the above formula, L θ represents the loss of joint angle, θ t n represents the true value of the corresponding joint angle at time t, represents the predicted joint angle.

[0032] In an optional embodiment, constructing the physical information loss using the predicted joint torque includes:

[0033] The actual joint torque and the predicted joint torque are combined to obtain a set of motion equations, and the set of motion equations is iteratively solved to obtain torque data;

[0034] Under the condition of the torque data, the joint angle and boundary reaction force data are calculated using the weighted least square method, and the target joint torque at each joint is calculated based on the joint angle and boundary reaction force data.

[0035] In an optional embodiment, inputting the joint angle and the EMG signal into the deep neural network model for prediction comprises the following steps:

[0036] The joint angle and the EMG signal are converted into time slice data using a recursive module; wherein each time step in each time slice in the time slice data includes the electromyographic signal of each muscle in the key muscle group and the joint angle corresponding to the key muscle group;

[0037] The convolution module performs layer-by-layer and step-by-step convolution on the time slice data to obtain muscle strength characteristics;

[0038] The muscle strength features are processed in turn through the fully connected layer and the dropout layer, and the predicted joint torque is output through the output layer.

[0039] A second aspect of the present invention provides a muscle strength prediction system integrating physical model and deep learning, comprising:

[0040] A neuromusculoskeletal model construction module, wherein the neuromusculoskeletal model construction module is used to construct a neuromusculoskeletal model and obtain EMG signals using the neuromusculoskeletal model;

[0041] A prediction module, wherein the prediction module is used to construct a deep neural network model including a recursive module and a convolutional module, obtain the joint angle, input the joint angle and the EMG signal into the deep neural network model for prediction, and obtain the predicted joint torque;

[0042] A training module, wherein the training module is used to construct a total loss function of a deep neural network model, use the total loss function to train the deep neural network model, and use the trained deep neural network model to calculate muscle strength; wherein the total loss function includes neural network loss and physical information loss.

[0043] In an optional embodiment, the prediction module includes:

[0044] A conversion unit, the conversion unit is used to convert the joint angle and the EMG signal into time slice data using a recursive module; wherein each time step under each time slice in the time slice data includes the electromyographic signal of each muscle in the key muscle group and the joint angle corresponding to the key muscle group;

[0045] A feature extraction unit, wherein the feature extraction unit is used for the convolution module to perform layer-by-layer and step-by-step convolution on the time slice data to obtain muscle strength features;

[0046] An output unit is used to process the muscle strength feature through a fully connected layer and a dropout layer in sequence, and output the predicted joint torque through the output layer.

[0047] A third aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a muscle strength prediction method integrating a physical model with deep learning is implemented.

[0048] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a muscle strength prediction method based on a physical model fused with deep learning.

[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0050] 1. The physics information of neuromusculoskeletal system is used as the soft constraint of the data-driven model, and the dynamic time relationship of muscle force distribution is fully considered. The time series input data is converted into time slice data. With kinematic data and EMG signals as input, the deep learning algorithm automatically extracts high-level features and establishes the relationship between input signals and muscle force, so as to achieve muscle force prediction that satisfies the kinematic equation. This overcomes the problem that the current models for muscle force calculation usually do not include explicit physical modeling of the underlying neuromechanical process and lack reflection of the physical mechanism during joint movement.

[0051] 2. Input EMG signals and joint angles. The recursive module can effectively learn the nonlinear characteristics of sequence data and effectively process the temporal relationship between EMG signals and joint angles. The convolution module reduces the complexity of data reconstruction through weight sharing, thereby improving the ability of feature extraction and the speed of classification.

[0052] 3. The total loss function is constructed by using the mean square error loss of deep learning integration and the error loss based on torque calculation to achieve accurate solution and model verification of dynamic muscle strength. The skeletal muscle force calculation model built by the deep learning network has stronger generalization ability and can provide useful model support for analyzing biomechanical characteristics with minimal motion data in different situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0054] Figure 1 A schematic diagram of the structure of the Hill muscle model provided in Example 1 of the present invention;

[0055] Figure 2 A schematic diagram of a process for constructing the overall loss function of the model provided in Example 1 of the present invention;

[0056] Figure 3 Schematic diagram of the process of calculating joint torque based on EMG drive provided in Example 1 of the present invention

[0057] Figure 4 A schematic diagram of the structure of a deep neural network model that integrates a physical model and calculates skeletal muscle force provided in Example 1 of the present invention;

[0058] Figure 5 A schematic diagram of the structure of an electronic device provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0060] Example 1

[0061] Embodiment 1 of the present invention provides a muscle strength prediction method integrating physical model with deep learning, and the muscle strength calculation method comprises the following steps:

[0062] constructing a neuromusculoskeletal model, and obtaining EMG signals using the neuromusculoskeletal model;

[0063] Constructing a deep neural network model including a recursive module and a convolutional module, obtaining the joint angle, inputting the joint angle and the EMG signal into the deep neural network model for prediction, and obtaining the predicted joint torque;

[0064] Construct a total loss function of a deep neural network model, use the total loss function to train the deep neural network model, and use the trained deep neural network model to calculate muscle strength; wherein the total loss function includes neural network loss and physical information loss.

[0065] It should be noted that the current calculation of muscle strength lacks modeling of potential neuromechanics, which makes it impossible to reflect the neuromotor control mechanism during joint dynamic movement when using neural network models to extract features from electromyograms, which in turn leads to large errors in the calculated muscle strength.

[0066] Based on the defects of the existing technology, this method proposes to build a neuromusculoskeletal model, use the physical information of the neuromusculoskeletal system as the soft constraint of the data-driven model, use the neuromusculoskeletal model to simulate the dynamic movement of the joints, and generate EMG signals by simulating the dynamic movement of the joints. The signals reflect the neuromotor control mechanism during the dynamic movement of the joints.

[0067] The EMG signal and the joint angle are combined as input data and input into the deep neural network model, which includes a recursive module and a convolutional module, wherein the recursive module is used to convert the input data into time slice data, and the convolutional module is used to read the characteristics of the neuromotor control mechanism in the dynamic movement of the joint in the EMG signal, so as to establish the relationship between kinematic information and EMG signals and muscle strength, and realize the deep mapping to skeletal muscle strength. The EMG signal and the joint angle are input, and the nonlinear characteristics of the sequence data can be effectively learned through the recursive module, and the temporal relationship between the EMG signal and the joint angle is effectively processed. The convolutional module reduces the complexity of data reconstruction through weight sharing, and improves the ability of feature extraction and the speed of classification.

[0068] In order to ensure that the muscle force calculation results meet the physical motion equations, while constructing the muscle force solution model, the physics-based domain knowledge is introduced as a soft constraint into the data-driven model to standardize the data-driven model. By comparing the actual joint torque with the predicted joint torque, the target torque that meets the dynamic balance condition and boundary condition is found, and the target torque and the joint angle corresponding to the target torque are used to train the deep neural network model. The trained deep neural network model is used to calculate muscle force, and the results obtained are closer to the musculoskeletal situation in the real movement process under the neuromotor control mechanism in the dynamic movement of the joint. This method overcomes the problem that the current models used for muscle force calculation usually do not contain explicit physical modeling of the underlying neuromechanical process, lack reflection of the physical mechanism in the joint movement process, and therefore cannot fully reflect the physical mechanism behind the observed variables.

[0069] In an optional embodiment, constructing a neuromusculoskeletal model includes:

[0070] The muscle-tendon is simplified into a contraction element, a series elastic element and a parallel elastic element, and the contraction element, the series elastic element and the parallel elastic element are used to establish a muscle-tendon execution unit force relationship; wherein the muscle-tendon execution unit force relationship is established as follows:

[0071] F T =F CE =(F CE +F PE )cosα

[0072] In the above formula, F T is the main force of the tendon, F CE is the contraction force of the active elastic contraction element, F PE is the parallel elastic element contraction force, and α is the feathering angle.

[0073] It should be noted that there are usually two types of interactions between bones and muscles: muscles change the posture of bones through force; at the same time, the posture of bones determines the overall length of muscles, and muscle length in turn affects muscle contraction force.

[0074] The embodiment of the present invention takes the simulation of the first-order dynamic Hill muscle model between excitation and activation as an example, and the Hill muscle force three-element model is as follows: Figure 1 As shown in , it considers the essential structure of muscles and simplifies muscle tendons into three elements, namely contractile element (CE), serial elastic element (SE) and parallel elastic element (PE) to reflect the function of muscles. Figure 1 As shown, the length between muscles is L T +L M cosα, where L T is the tendon length, L M cosα is the muscle length, α is the feather angle, and based on this figure, the force of the muscle tendon can be calculated using the contraction element, series elastic element, and parallel elastic element. The dynamic motion process of the joint is simulated using the neuromusculoskeletal model. When the EMG signal is generated, the EMG signal contains the force of the muscle tendon. The force is predicted by the deep neural network model to obtain the predicted joint torque.

[0075] In an optional embodiment, the recursive module includes n recursive layers;

[0076] Among them, when there is only a single subject and the motion state is unique, n is 1;

[0077] When the motion state is unique but there are multiple subjects, n is 2;

[0078] When there are multiple subjects or the motion state is not unique, n is 3;

[0079] When there are multiple subjects and the motion state is not unique, n is 4.

[0080] It should be noted that by adjusting the number of recursive layers to adapt to different subject scenarios, the method is applicable to a variety of scenarios.

[0081] In an optional embodiment, the total loss function includes muscle force loss and joint angle loss;

[0082] Wherein, the expression of muscle force loss is as follows:

[0083] L F =MSE(F)

[0084]

[0085] In the above formula, L F Represents the loss of muscle force, F t n represents the true value of the corresponding muscle force at time t, represents the predicted muscle force; N represents the total number of muscles at the target joint;

[0086] The expression of the joint angle loss is as follows:

[0087] L θ =MSE(θ)

[0088]

[0089] In the above formula, L θ represents the loss of joint angle, θ t n represents the true value of the corresponding joint angle at time t, represents the predicted joint angle.

[0090] It should be noted that the deep neural network model fully considers the changes in muscle strength related to musculoskeletal biomechanics under dynamic time relationships by adding physical information penalty / regularization soft constraints of the deep model, which helps to determine the final node weights of the model and can detect the overall causal relationship between the input data joint angle (kinematic data), EMG signal and the output data joint torque of skeletal muscle force.

[0091] like Figure 2 As shown in Figure 1, the total loss function includes a deep neural network part and a physical information loss part. Through the construction of the total loss function, along with the loss of the data-driven part, the loss based on the physical equation plays a regularization role in the prediction of the muscle force model, enhancing the robustness of the constructed model, and encoding physical information into the deep neural network can also enhance the generalization performance of the framework.

[0092] Among them, the total loss function L T The build is as follows:

[0093] L T =L F +L θ +L P

[0094] L F =MSE(F),L θ =MSE(θ),L P =f(F,θ)

[0095] In the above formula, LF Indicates the loss of muscle strength, L θ represents the loss of joint angle, L P Represents the loss function imposed by the laws of physics. Among them, the loss of muscle force and the loss of joint angle are the losses of the deep neural network part, the loss function imposed by the laws of physics is the physical information loss part, the loss of the deep neural network part is to minimize the MSE of the ground truth and prediction, and the physics-based loss can preserve physical constraints during human motion.

[0096] In an optional embodiment, the construction process of the physical information loss includes:

[0097] The actual joint torque and the predicted joint torque are combined to obtain a set of motion equations, and the set of motion equations is iteratively solved to obtain torque data;

[0098] Under the condition of the torque data, the joint angle and boundary reaction force data are calculated using the weighted least square method, and the target joint torque at each joint is calculated based on the joint angle and boundary reaction force data.

[0099] It should be noted that the control law based on the physical model reflects the potential relationship between muscle force and joint torque in human movement. In the loss training stage of deep learning, the relationship between myoelectricity and torque is converted into corresponding physical constraints. Figure 3 The principle shown is to use joint angles and EMG signals as inputs, and to implement physical model calculation of joint torque based on tendon kinematics, muscle contraction dynamics, and muscle activation dynamics. The difference between the calculated value and the predicted value is taken as the physical loss.

[0100] Specifically, the skeletal muscle force is back-calculated through the physical equation of the neuromusculoskeletal model to obtain the joint torque. In this embodiment, the tendon active force obtained by the Hill muscle force three-element model is converted into a vector representation of the force generated by the muscle to calculate the joint torque, and the torque at the joint generated by the muscle force is obtained as the predicted joint torque. The calculation process is as follows:

[0101] τ=J T F

[0102] Where τ represents the moment at the joint generated by the muscle force; J T is the transpose of the Jacobian matrix, describing the effect of muscle force on joint position; F is the vector representation of the force generated by the muscle.

[0103] The actual measured joint torque is calculated using inverse dynamics calculations by solving the dynamic equations of motion as follows:

[0104]

[0105] τ is the target moment vector, q, are the generalized position, velocity, and acceleration vectors, respectively; M(q) is the mass matrix, describing the inertial properties of the system; is the Coriolis and centrifugal force matrix, which reflects the dynamic force generated by the change of joint position and velocity; G(q) is the gravity matrix that describes the influence of gravity on each joint.

[0106] The above two formulas are combined to obtain a set of motion equations, and the motion equations are iteratively solved to obtain a certain torque data.

[0107] Then, based on the inverse kinematics equation, the weighted least squares problem is used to calculate the joint angle and the boundary reaction force data during the test. The formula is as follows:

[0108]

[0109] Among them, x i ,x i (q) represent the target position and the calculated position respectively, q j represent the expected angle and the actual angle respectively;

[0110] ∑ i∈maekers ω i ||x i -x i (q) 2 Indicates the position error of the marking point, Represents the preset joint coordinate error, which minimizes the sum of the weighted Euclidean distances between the target position and the calculated position by adjusting the joint angles.

[0111] Then, the target joint torque at each joint is calculated based on the joint angle and boundary reaction force data. The target joint torque satisfies the dynamic balance condition and boundary condition. The calculation formula is as follows:

[0112] minimize F obj

[0113] subjet to R(q)F TM =F MT

[0114]

[0115] Where q represents the joint angles of n joints, R(q) represents the muscle moment arm matrix (m×n), and F TM is the m×1 muscle force matrix, F MT represents the n×1 joint torque matrix, is the peak isometric force per cross-sectional area of ​​the muscle.

[0116] The physical information loss is constructed through the above process, which in turn reflects the neuromotor control mechanism during the dynamic movement of the joint.

[0117] In an optional embodiment, inputting the joint angle and the EMG signal into the deep neural network model for prediction comprises the following steps:

[0118] The joint angle and the EMG signal are converted into time slice data using a recursive module; wherein each time step in each time slice in the time slice data includes the electromyographic signal of each muscle in the key muscle group and the joint angle corresponding to the key muscle group;

[0119] The convolution module performs layer-by-layer and step-by-step convolution on the time slice data to obtain muscle strength characteristics;

[0120] The muscle strength features are processed in turn through the fully connected layer and the dropout layer, and the predicted joint torque is output through the output layer.

[0121] It should be noted that in the recursive module, the time series input data is converted into time slice data. Each time step under each time slice includes the electromyographic signal of each muscle in the key muscle group and the corresponding joint angle. The convolution module further reads the features in the kinematics and electromyographic signals through layer-by-layer and step-by-step convolution to achieve deep mapping to skeletal muscle force.

[0122] like Figure 4 As shown in the figure, first the joint angle and EMG signal are converted into time slice data through the recursive module, then pass through 3 convolutional modules with increasing number of neurons, and finally the fully connected layer and dropout layer. The fully connected layer obtains the output from the recursive layer to help the regression task of skeletal muscle force prediction, while the dropout layer is used to prevent the neural network from overfitting. Finally, the skeletal muscle force is predicted in the output layer. The hyperparameters of the deep model are determined by "coarse to fine" random search.

[0123] Example 2

[0124] Embodiment 2 of the present invention provides a muscle strength prediction system integrating physical model and deep learning, including:

[0125] A neuromusculoskeletal model construction module, wherein the neuromusculoskeletal model construction module is used to construct a neuromusculoskeletal model and obtain EMG signals using the neuromusculoskeletal model;

[0126] A prediction module, wherein the prediction module is used to construct a deep neural network model including a recursive module and a convolutional module, obtain the joint angle, input the joint angle and the EMG signal into the deep neural network model for prediction, and obtain the predicted joint torque;

[0127] A training module, wherein the training module is used to construct a total loss function of a deep neural network model, use the total loss function to train the deep neural network model, and use the trained deep neural network model to calculate muscle strength; wherein the total loss function includes neural network loss and physical information loss.

[0128] In an optional embodiment, the prediction module includes:

[0129] A conversion unit, the conversion unit is used to convert the joint angle and the EMG signal into time slice data using a recursive module; wherein each time step under each time slice in the time slice data includes the electromyographic signal of each muscle in the key muscle group and the joint angle corresponding to the key muscle group;

[0130] A feature extraction unit, wherein the feature extraction unit is used for the convolution module to perform layer-by-layer and step-by-step convolution on the time slice data to obtain muscle strength features;

[0131] An output unit is used to process the muscle strength feature through a fully connected layer and a dropout layer in sequence, and output the predicted joint torque through the output layer.

[0132] Example 3

[0133] Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention is shown in FIG. Figure 5 As shown, the electronic device includes a processor 21, a memory 22, an input device 23 and an output device 24; the number of processors 21 in the computer device can be one or more. Figure 5 A processor 21 is taken as an example; the processor 21, the memory 22, the input device 23 and the output device 24 in the electronic device can be connected by a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0134] The memory 22 is a computer-readable storage medium that can be used to store software programs, computer executable programs, and modules. The processor 21 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 22, that is, a physical model fusion deep learning muscle strength prediction method of embodiment 1 is implemented.

[0135] The memory 22 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 22 may further include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0136] The input device 23 can be used to receive the ID and password input by the user, etc. The output device 24 is used to output the network configuration page.

[0137] Example 4

[0138] Embodiment 4 of the present invention further provides a computer-readable storage medium, wherein the computer-executable instructions, when executed by a computer processor, are used to implement a muscle strength prediction method that integrates a physical model with deep learning as provided in Embodiment 1.

[0139] An embodiment of the present invention provides a storage medium containing computer executable instructions, and its computer executable instructions are not limited to the method operations provided in Example 1, and can also execute related operations in a muscle strength prediction method integrating physical model and deep learning provided in any embodiment of the present invention.

[0140] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A muscle strength prediction method integrating physical model with deep learning, characterized in that: The steps include: constructing a neuromusculoskeletal model, and obtaining EMG signals using the neuromusculoskeletal model; Constructing a deep neural network model including a recursive module and a convolutional module, obtaining the joint angle, inputting the joint angle and the EMG signal into the deep neural network model for prediction, and obtaining the predicted joint torque; A total loss function of a deep neural network model is constructed, the deep neural network model is trained using the total loss function, and the trained deep neural network model is used to calculate muscle strength; the total loss function includes a neural network loss and a physical information loss, wherein the physical information loss is constructed using the predicted joint torque.

2. The muscle strength prediction method of physical model fusion deep learning according to claim 1 is characterized in that: Constructing a neuromusculoskeletal model includes: The muscle-tendon is simplified into a contraction element, a series elastic element and a parallel elastic element, and the contraction element, the series elastic element and the parallel elastic element are used to establish a muscle-tendon execution unit force relationship; wherein the muscle-tendon execution unit force relationship is established as follows: F T =F CE =(F CE +F PE )cosα In the above formula, F T is the main force of the tendon, F CE is the contraction force of the active elastic contraction element, F PE is the parallel elastic element contraction force, and α is the feathering angle.

3. The muscle strength prediction method of physical model fusion deep learning according to claim 1 is characterized in that: The total loss function includes muscle force loss and joint angle loss; Wherein, the expression of muscle force loss is as follows: L F =MSE(F) In the above formula, L F Indicates loss of muscle strength. represents the true value of the corresponding muscle force at time t, represents the predicted muscle force; N represents the total number of muscles at the target joint; The expression of the joint angle loss is as follows: L θ =MSE(θ) In the above formula, L θ represents the loss of joint angle, represents the true value of the corresponding joint angle at time t, represents the predicted joint angle.

4. The muscle strength prediction method of physical model fusion deep learning according to claim 1 is characterized in that: Using the predicted joint torque to construct the physical information loss includes: The actual joint torque and the predicted joint torque are combined to obtain a set of motion equations, and the set of motion equations is iteratively solved to obtain torque data; Under the condition of the torque data, the joint angle and boundary reaction force data are calculated using the weighted least square method, and the target joint torque at each joint is calculated based on the joint angle and boundary reaction force data.

5. The muscle strength prediction method of physical model fusion deep learning according to claim 1 is characterized in that: Inputting the joint angle and the EMG signal into the deep neural network model for prediction comprises the following steps: The joint angle and the EMG signal are converted into time slice data using a recursive module; wherein each time step in each time slice in the time slice data includes the electromyographic signal of each muscle in the key muscle group and the joint angle corresponding to the key muscle group; The convolution module performs layer-by-layer and step-by-step convolution on the time slice data to obtain muscle strength characteristics; The muscle strength features are processed in turn through the fully connected layer and the dropout layer, and the predicted joint torque is output through the output layer.

6. The muscle strength prediction method of physical model fusion deep learning according to claim 5 is characterized in that: The recursive module includes n recursive layers; Among them, when there is only a single subject and the motion state is unique, n is 1; When the motion state is unique but there are multiple subjects, n is 2; When there are multiple subjects or the motion state is not unique, n is 3; When there are multiple subjects and the motion state is not unique, n is 4.

7. A muscle strength prediction system integrating physical model and deep learning, characterized in that: include: A neuromusculoskeletal model construction module, wherein the neuromusculoskeletal model construction module is used to construct a neuromusculoskeletal model and obtain EMG signals using the neuromusculoskeletal model; A prediction module, wherein the prediction module is used to construct a deep neural network model including a recursive module and a convolutional module, obtain the joint angle, input the joint angle and the EMG signal into the deep neural network model for prediction, and obtain the predicted joint torque; A training module, wherein the training module is used to construct a total loss function of a deep neural network model, use the total loss function to train the deep neural network model, and use the trained deep neural network model to calculate muscle strength; wherein the total loss function includes neural network loss and physical information loss.

8. The muscle strength prediction system of physical model fusion deep learning according to claim 7 is characterized in that: The prediction module comprises: A conversion unit, the conversion unit is used to convert the joint angle and the EMG signal into time slice data using a recursive module; wherein each time step under each time slice in the time slice data includes the electromyographic signal of each muscle in the key muscle group and the joint angle corresponding to the key muscle group; A feature extraction unit, wherein the feature extraction unit is used for the convolution module to perform layer-by-layer and step-by-step convolution on the time slice data to obtain muscle strength features; An output unit is used to process the muscle strength feature through a fully connected layer and a dropout layer in sequence, and output the predicted joint torque through the output layer.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, a muscle strength prediction method integrating physical model with deep learning as described in claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements a muscle strength prediction method integrating physical model and deep learning as described in claims 1 to 6.

Images