Lower limb biomechanical data estimation method based on data expansion technology
Through the combination of TimeGAN model and Bert model, the stable training problem of deep generative adversarial networks in biomechanical data is solved, efficient and accurate data expansion and estimation is achieved, and the effectiveness of exercise evaluation and medical rehabilitation is improved.
Patent Information
- Application Number
- CN202510340127.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-21
- Publication Date
- 2025-07-11
AI Technical Summary
The existing deep-generating adversarial models are difficult to directly apply to biomechanical data, especially due to the problems of gradient disappearance and pattern collapse, and the complex probability distribution types of biomechanical data are included in multiple modes, making it difficult for the generative model to be stablely trained to the optimal state.
The TimeGAN model based on the TCN-space-time attention mechanism is used to expand the data of ground reaction force, inertial sensing data and lower limb joint torque, and the expansion data set is generated through the Bert model fusion, and feature extraction and estimation are combined with inertial sensing data, dynamically adjust the ratio of generated data to real data, and data correction is performed using the mixed coefficient γ.
An efficient and accurate motion evaluation method has been established, which has improved the accuracy and diversity of samples generated, and improved the accuracy of GRF, EMG and joint torque estimation. It is suitable for sports evaluation, medical rehabilitation, biomechanics and exoskeleton control fields.
Smart Images
Figure CN120296341A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of bioinformatics perception, and particularly to a method for estimating lower limb biomechanical data based on data augmentation technology. Background Art
[0002] In recent years, with the rapid development of artificial intelligence technology, many deep learning methods have been applied to the fields of biomechanics and medical diagnosis. Deep learning models often require a large amount of training data to perform well. In the fields of biomechanics and medical diagnosis, due to reasons such as the privacy of patient data, the difficulty of data collection and processing, and the complexity of the experimental environment, it is often difficult to obtain a large amount of training data. Therefore, it is necessary to augment such small-scale data sets.
[0003] With the development of deep learning technology, generative models generate data by fitting the probability distribution of training data and have become a relatively advanced data augmentation method. As an important research field of unsupervised learning in machine learning, generative models generally refer to establishing a joint probability distribution model of data observation values based on randomly generated observation data. GAN is one of the current popular deep generative models and has achieved good research results in image data generation. However, there are still problems such as gradient disappearance and mode collapse, which make it difficult to train the generative model stably to the optimal state. In addition, compared with image data, the probability distribution type of each dimension of biomechanical data is more complex and contains multiple modalities. Existing deep generative adversarial models are difficult to be directly applied. Summary of the Invention
[0004] Based on this, in view of the problem that existing deep generative adversarial models are difficult to be directly applied to biomechanical data, it is necessary to provide a method for estimating lower limb biomechanical data based on data augmentation technology.
[0005] To solve the above technical problems, the technical solution adopted by the present invention is:
[0006] A method for estimating lower limb biomechanical data based on data augmentation technology, which includes the following steps:
[0007] Synchronously collect the motion data of a subject during the motion process; the motion data includes inertial sensing data, ground reaction force, and electromyogram signals;
[0008] Filter the ground reaction force and inertial sensing data, and filter, rectify, and normalize the electromyogram signals;
[0009] Deduce the lower limb joint torque during the motion process through inverse dynamics with the processed data, and normalize it according to the subject's body weight;
[0010] The processed ground reaction force, inertial sensing data, and lower limb joint moments are subjected to data augmentation through a pre-constructed TimeGAN model based on the TCN-spatiotemporal attention mechanism. The augmented data is fused with the original data to generate an augmented dataset, and the Bert model is trained with this dataset.
[0011] Inertial sensing data is obtained in real time, its features are extracted to generate input features, and the input features are input into the trained Bert model to estimate the ground reaction force, electromyogram signal, and lower limb joint moments.
[0012] Furthermore, the input features generated by extracting the features of the inertial sensing data include the x-axis accelerometer data a x , the y-axis accelerometer data a y , the z-axis accelerometer data a z , the combined acceleration a of the three-axis acceleration data, the average acceleration a of the three-axis acceleration data average , the x-axis gyroscope data b x , the y-axis gyroscope data b y , the z-axis gyroscope data b z , the combined value b of the three-axis gyroscope data, the average value b of the three-axis gyroscope data average .
[0013] Furthermore, the specific steps for data augmentation of the processed ground reaction force, inertial sensing data, and lower limb joint moments through the pre-constructed TimeGAN based on the TCN-spatiotemporal attention mechanism are as follows:
[0014] The processed ground reaction force, inertial sensing data, and lower limb joint moments are segmented by a sliding window, and the frequency domain features of each window are extracted.
[0015] The frequency domain features are input into the TimeGAN model based on the TCN-spatiotemporal attention mechanism to output generated data, and the mixing coefficient γ is calculated in real time by calculating the difference between the generated data and the input real data: where k represents the scaling factor, D real represents the confidence of the real data, and D gen represents the confidence of the generated data;
[0016] The ratio of the generated data to the real data is dynamically adjusted through the mixing coefficient γ to obtain the augmented data.
[0017] Furthermore, when the difference between D real and D gen is greater than the preset upper limit value, the ratio of the real data is adjusted to 99%; when D gen is less than the preset lower limit value, all real data is adopted.
[0018] Further, the specific steps for normalizing the lower limb joint moments during the movement process according to the subject's body weight are as follows:
[0019] Convert the lower limb joint moments during the movement process into dimensionless parameters where τ j represents the lower limb joint moments during the movement process, BW represents the subject's body weight, BH represents the subject's height, and represents the dimension balance factor;
[0020] Perform velocity compensation correction on the dimensionless parameters to obtain the processed lower limb joint moments where v represents the average walking speed of the subject, and g = 9.81m / S 2 .
[0021] The present invention also relates to a lower limb biomechanical data estimation system based on data augmentation technology, including a data acquisition module, a data processing module, a data augmentation module, and a data estimation module.
[0022] The data acquisition module is used to synchronously acquire movement data during the movement process; the movement data includes inertial sensing data, ground reaction force, and electromyography signals;
[0023] The data processing module is used to filter the ground reaction force and inertial sensing data, and filter, rectify, and normalize the electromyography signals; it is also used to inversely deduce the lower limb joint moments during the movement process through inverse dynamics and normalize them according to the subject's body weight;
[0024] The data augmentation module is used to augment the processed ground reaction force, inertial sensing data, and lower limb joint moments through a pre-constructed TimeGAN model based on the TCN-spatiotemporal attention mechanism, fuse the augmented data with the original data to generate an augmented data set, and train a Bert model with this;
[0025] The data estimation module is used to obtain inertial sensing data in real time, extract features from it to generate input features, and input the input features into the trained Bert model to estimate the ground reaction force, electromyography signals, and lower limb joint moments.
[0026] The present invention also relates to a lower limb biomechanical data estimation device based on data augmentation technology, which includes an inertial sensing unit, a force platform, an electromyograph, a three-dimensional motion capture system, and a central processing unit.
[0027] An inertial sensing unit is installed on the lower limbs of a subject to collect inertial sensing data during the subject's movement; a force platform is used to obtain the ground reaction force of the subject; an electromyograph is installed on the lower limbs of the subject to collect electromyographic signals during the subject's movement; a three-dimensional motion capture system is used to spatially locate the subject's movement trajectory; a central processor is used to process the data collected by the inertial sensing unit, the force platform, the electromyograph and the three-dimensional motion capture system, and the processing method adopts the steps of the lower limb biomechanical data estimation method based on the data augmentation technology as described above.
[0028] Compared with the prior art, the beneficial effects of the present invention include:
[0029] The present invention fully considers the accuracy, diversity of the generated samples and the accuracy of GRF, EMG and joint torque estimation, thereby establishing an efficient and accurate motion evaluation method, which can facilitate the development of motion evaluation and guide motion training, and has good application prospects and economic benefits in the fields of medical rehabilitation, biomechanics, sports science and exoskeleton control. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] The disclosure of the present invention will be described with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In the drawings, the same reference numerals are used to refer to the same components. Among them:
[0031] Figure 1 is a flowchart of a lower limb biomechanical data estimation method based on data augmentation technology introduced in the present invention;
[0032] Figure 2 is based on Figure 1 flowchart of the original data augmentation;
[0033] Figure 3 is a flowchart for estimating EMG, IMU and joint torque. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0034] It is easy to understand that according to the technical solution of the present invention, without changing the essence of the present invention, those of ordinary skill in the art can propose various structural ways and implementation ways that can be mutually replaced. Therefore, the following detailed embodiments and drawings are only exemplary descriptions of the technical solution of the present invention, and should not be regarded as the whole of the present invention or regarded as a limitation or restriction on the technical solution of the present invention.
[0035] Embodiment 1
[0036] As Figure 1 shown, the present invention introduces a lower limb biomechanical data estimation method based on data augmentation technology, including the following steps:
[0037] Step 1: Synchronously collect the motion data of the subject during the motion process; the motion data includes inertial sensing data (IMU), ground reaction force (GRF), and electromyogram signal (EMG).
[0038] The collected data is used as the real data to provide a real basis for subsequent augmented data and estimated data.
[0039] Step 2: Filter the ground reaction force and inertial sensing data, and filter, rectify, and normalize the electromyogram signal.
[0040] Step 3: Use the processed data to recursively deduce the lower limb joint moments during the motion process through inverse dynamics, and normalize them according to the subject's weight. The specific steps are as follows:
[0041] Combine imaging and geometric modeling techniques to calculate the inertial tensor of each segment through the Yeadon algorithm, and use the spherical fitting algorithm to determine the joint center.
[0042] Recursively calculate the lower limb joint moments during the motion process from the foot to the proximal end:
[0043] τ j =I j ω˙ j +ω j ×(I j ω j )+∑F ext ×r j -∑m k (g+a com,k )×l j . Where, I j ∈R 3×3 represents the joint inertia tensor matrix, ω˙ j represents the joint angular acceleration, r j represents the distance vector from the external force application point to the joint center. I j ω˙ j +ω j ×(I j ω j ) is the inertial term, ∑F ext ×r j is the external force term, ∑m k (g+a com,k )×l j is the gravity term.
[0044] Then convert the lower limb joint moment τ j into a dimensionless parameter Where, BW represents the subject's weight, BH represents the subject's height, represents the dimension balance factor.
[0045] Perform velocity compensation correction on the dimensionless parameters to obtain the processed lower limb joint torque Among them, v represents the average walking speed of the subject, and g = 9.81m / S 2 .
[0046] Step 4: Perform data augmentation on the processed ground reaction force, inertial sensing data, and lower limb joint torque through a pre-constructed TimeGAN model based on the TCN-spatiotemporal attention mechanism. Fuse the augmented data with the original data to generate an augmented dataset, and train the Bert model with this dataset. The specific steps are as follows:
[0047] Segment the processed ground reaction force, inertial sensing data, and lower limb joint torque through a sliding window, and extract the frequency domain features of each window;
[0048] Input the frequency domain features into the TimeGAN model based on the TCN-spatiotemporal attention mechanism to output generated data, and calculate the difference between the generated data and the input real data in real time to obtain the mixing coefficient γ: Among them, k represents the scaling factor, and the value range is 2.5 - 4.0, D real represents the confidence of the real data, and the reference range is [-5, 5], D gen represents the confidence of the generated data, and the reference range is [-10, 3];
[0049] Dynamically adjust the ratio of the generated data to the real data through the mixing coefficient γ, and then obtain the augmented data. Fuse the augmented data with the original data to generate an augmented dataset, and train the Bert model with this dataset.
[0050] The rules for dynamically adjusting through the mixing coefficient γ are as follows:
[0051] When the quality of the generated data is significantly lower than that of the real data (i.e., D real - D gen > 2.0): That is, when K takes 2.5, γ is approximately equal to 0.993. At this time, the proportion of real data is automatically increased to 99%, suppressing the influence of low-quality generated data.
[0052] When the generated data is close to the real quality (i.e., |D real - D gen | < 0.5), Adopt an equal mixing strategy (real: generated = 1:1) to promote data diversity.
[0053] If the generated data has serious defects (such as D gen < -5.0): Forcefully switch to the pure real data mode (γ = 1) to prevent the training set from being contaminated by incorrect data.
[0054] The TimeGAN model based on the TCN-spatiotemporal attention mechanism is constructed by multi-scale spatiotemporal feature extraction, cross-modal correlation modeling and embedding biomechanical rules.
[0055] Step 5: Real-time obtain inertial sensing data, extract features from it to generate input features, and input the input features into the trained Bert model to estimate the ground reaction force, electromyogram signal and lower limb joint torque.
[0056] Extract features from the IMU data to generate input features: the x-axis accelerometer data a x , the y-axis accelerometer data a y , the z-axis accelerometer data a z , the combined acceleration a of the three-axis accelerometer data, the average acceleration a of the three-axis accelerometer data average , the x-axis gyroscope data b x , the y-axis gyroscope data b y , the z-axis gyroscope data b z , the combined value b of the three-axis gyroscope data, the average value b of the three-axis gyroscope data average . Where:
[0057]
[0058] a average =(a x +a y +a z ) / 3, b average =(ω x +ω y +ω z ) / 3.
[0059] This embodiment fully considers the accuracy and diversity of the generated samples and the accuracy of GRF, EMG and joint torque estimation, thus establishing an efficient and accurate motion evaluation method, which can facilitate the conduct of motion evaluation and the guidance of motion training, and has good application prospects and economic benefits in the fields of medical rehabilitation, biomechanics, sports science and exoskeleton control.
[0060] Embodiment 2
[0061] This embodiment introduces a lower limb biomechanical data estimation system based on data augmentation technology, including a data acquisition module, a data processing module, a data augmentation module and a data estimation module.
[0062] The data acquisition module is used to synchronously collect motion data during the motion; the motion data includes inertial sensing data, ground reaction force and electromyogram signal.
[0063] The data processing module is used to filter the ground reaction force and inertial sensing data, and filter, rectify and normalize the electromyogram signals. It is also used to recursively deduce the lower limb joint moments during the movement through inverse dynamics for the processed data, and normalize them according to the subject's body weight.
[0064] The data augmentation module is used to augment the processed ground reaction force, inertial sensing data and lower limb joint moments through a pre-constructed TimeGAN model based on the TCN-spatiotemporal attention mechanism, fuse the augmented data with the original data to generate an augmented data set, and train the Bert model with this.
[0065] The data estimation module is used to obtain inertial sensing data in real time, extract features from it to generate input features, and input the input features into the trained Bert model to estimate the ground reaction force, electromyogram signals and lower limb joint moments.
[0066] Embodiment 3
[0067] This embodiment also introduces a lower limb biomechanical data estimation device based on data augmentation technology, which includes an inertial sensing unit, a force platform, an electromyograph, a three-dimensional motion capture system and a central processor.
[0068] The inertial sensing unit is installed on the lower limbs of the subject and is used to collect inertial sensing data during the subject's movement; the force platform is used to obtain the ground reaction force of the subject; the electromyograph is installed on the lower limbs of the subject and is used to collect electromyogram signals during the subject's movement; the three-dimensional motion capture system is used to perform spatial positioning on the subject's movement trajectory; the central processor is used to process the data collected by the inertial sensing unit, the force platform, the electromyograph and the three-dimensional motion capture system, and the processing method adopts the steps of the lower limb biomechanical data estimation method based on data augmentation technology as described above.
[0069] In practical applications, 6 IMU nodes are deployed on both lower limbs, that is, they are installed on the thigh / calf / foot respectively, multi-node clock synchronization is achieved through the CAN bus, and a quaternion fusion algorithm is used to eliminate the coordinate deviations of each IMU coordinate system. The three-dimensional motion capture system has a camera array to achieve sub-millimeter-level spatial positioning of the movement trajectory of the human body or object, and constructs a six-degree-of-freedom (6DoF) kinematic model.
[0070] Embodiment 4
[0071] This embodiment provides a computer terminal, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps of the lower limb biomechanical data estimation method based on data augmentation technology in Embodiment 1.
[0072] When the lower limb biomechanical data estimation method based on the data augmentation technology of Embodiment 1 is applied, it can be applied in the form of software, such as designed as an independently running program and installed on a computer terminal, which can be a computer, a smart phone, etc. It can also be designed as an embedded running program and installed on a computer terminal, such as installed on a single-chip microcomputer.
[0073] Embodiment 5
[0074] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the lower limb biomechanical data estimation method based on the data augmentation technology of Embodiment 1 are implemented.
[0075] When the lower limb biomechanical data estimation method based on the data augmentation technology of Embodiment 1 is applied, it can be applied in the form of software, such as designed as an independently running program on a computer-readable storage medium. The computer-readable storage medium can be a USB flash drive, designed as a USB key, and designed as a program to start the whole method through external triggering via the USB flash drive.
[0076] The technical scope of the present invention is not limited to the content in the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.
Claims
1. A method for estimating lower limb biomechanical data based on data augmentation technology, characterized in that It includes the following steps: Synchronously collect the motion data of the subject during the motion; the motion data includes inertial sensing data, ground reaction force, and electromyogram signals; Filter the ground reaction force and inertial sensing data, and filter, rectify, and normalize the electromyogram signals; Deduce the lower limb joint moments during the motion through inverse dynamics for the processed data, and normalize them according to the subject's weight; Augment the processed ground reaction force, inertial sensing data, and lower limb joint moments through the pre-constructed TimeGAN model based on the TCN-spatiotemporal attention mechanism, fuse the augmented data with the original data to generate an augmented dataset, and train the Bert model with this; Obtain the inertial sensing data in real time, extract features from it to generate input features, and input the input features into the trained Bert model to estimate the ground reaction force, electromyogram signals, and lower limb joint moments.
2. The method for estimating lower limb biomechanical data based on the data augmentation technique according to claim 1, characterized in that, The input features generated by feature extraction of inertial sensing data include the x-axis accelerometer data a x , the y-axis accelerometer data a y , the z-axis accelerometer data a z , the combined acceleration a of the three-axis acceleration data, the average acceleration a of the three-axis acceleration data average , the x-axis gyroscope data b x , the y-axis gyroscope data b y , the z-axis gyroscope data b z , the combined value b of the three-axis gyroscope data, the average value b of the three-axis gyroscope data average .
3. The method for estimating lower limb biomechanical data based on data augmentation technology according to claim 1, characterized in that The specific steps for augmenting the processed ground reaction force, inertial sensing data, and lower limb joint moments through the pre-constructed TimeGAN based on the TCN-spatiotemporal attention mechanism are as follows: Segment the processed ground reaction force, inertial sensing data, and lower limb joint moments through a sliding window, and extract the frequency domain features of each window; Input the frequency-domain features into the TimeGAN model based on the TCN-spatiotemporal attention mechanism to output the generated data, and calculate the mixing coefficient γ in real time by calculating the difference between the generated data and the input real data: where k represents the scaling factor, D real represents the confidence of the real data, D gen represents the confidence of the generated data; Dynamically adjust the ratio of the generated data to the real data through the mixing coefficient γ to obtain the augmented data.
4. The method for estimating lower limb biomechanical data based on data augmentation technology according to claim 3, characterized in that According to D real The difference from D gen is greater than the preset upper limit value, the proportion of real data is adjusted to 99%; when D gen is less than the preset lower limit value, all real data is adopted.
5. The method for estimating lower limb biomechanical data based on the data augmentation technology according to claim 1, wherein The specific steps for normalizing the lower limb joint moments during the motion according to the subject's weight are as follows: Convert the lower limb joint torque during movement into dimensionless parameters Among them, τ j represents the lower limb joint torque during movement, BW represents the body weight of the subject, and BH represents the height of the subject represents the dimension balance factor Perform velocity compensation correction on the dimensionless parameter to obtain the processed lower limb joint moment Among them, v represents the average walking speed of the subject, and g = 9.81m / S 2 .
6. A lower limb biomechanical data estimation system based on data augmentation technology, characterized in that, It includes: A data acquisition module, which is used to synchronously collect the motion data during the motion; the motion data includes inertial sensing data, ground reaction force, and electromyogram signals; A data processing module, which is used to filter the ground reaction force and inertial sensing data, filter, rectify, and normalize the electromyogram signals; and is also used to deduce the lower limb joint moments during the motion through inverse dynamics for the processed data, and normalize them according to the subject's weight; A data augmentation module, which is used to augment the processed ground reaction force, inertial sensing data, and lower limb joint moments through the pre-constructed TimeGAN model based on the TCN-spatiotemporal attention mechanism, fuse the augmented data with the original data to generate an augmented dataset, and train the Bert model with this; A data estimation module, which is used to obtain the inertial sensing data in real time, extract features from it to generate input features, and input the input features into the trained Bert model to estimate the ground reaction force, electromyogram signals, and lower limb joint moments.
7. A lower limb biomechanical data estimation device based on data augmentation technology, characterized in that, It includes: An inertial sensing unit, which is installed on the lower limb of the subject and is used to collect the inertial sensing data during the subject's motion; A force platform, which is used to obtain the ground reaction force of the subject; An electromyograph, which is installed on the lower limb of the subject and is used to collect the electromyogram signals during the subject's motion; A three-dimensional motion capture system, which is used to perform spatial positioning on the subject's motion trajectory; A central processing unit, which is used to process the data collected by the inertial sensing unit, the force platform, the electromyograph and the three-dimensional motion capture system, and the processing method adopts the steps of the lower limb biomechanical data estimation method based on the data augmentation technology as described in any one of claims 1-5.
8. The lower limb biomechanical data estimation device based on the data augmentation technique according to claim 7, wherein The number of inertial sensing units is at least 6, which are respectively installed on the thigh, the calf and the foot.
9. A computer terminal, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it realizes the steps of the lower limb biomechanical data estimation method based on the data augmentation technology as described in any one of claims 1-5.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it realizes the steps of the lower limb biomechanical data estimation method based on the data augmentation technology as described in any one of claims 1-5.