Whole body attitude estimation method and system based on pressure and inertial sensor

Abstract

The invention discloses a whole body posture estimation method and system based on a pressure sensor and an inertial sensor, relates to a human body posture reconstruction technology, and provides the scheme for solving the problems of low technology fusion and the like in the prior art. The three inertial sensors are used for generating real IMU data; according to the human body posture reconstruction algorithm, real IMU data and virtual IMU data are fused, and whole body posture estimation is carried out through the following steps that S1, pressure-based biological estimation is carried out, wherein pressure distribution is obtained through a pressure sensor so as to estimate ground counter-acting force GRF and the pressure center CoP; and S2, estimating the body posture in combination with the ground reaction force GRF and the pressure center CoP. The method has the advantages that only three inertial sensors are combined with plantar pressure data, so that complex human motion can be accurately captured. By combining the virtual IMU data with the actually collected IMU data, a larger-scale data set is constructed, and the method can be applied to multiple human body research fields.

Description

Technical Field

[0001] The present invention relates to a human body posture reconstruction technology, and in particular to a whole body posture estimation method and system based on pressure and inertial sensors. Background Art

[0002] In the digital health and interactive entertainment fields, especially in the rapidly developing fields of rehabilitation, fitness applications and virtual reality (VR) games, the accuracy of human posture reconstruction in three-dimensional (3D) space has become the key to improve perception accuracy and user experience. By providing detailed and real-time feedback on the user's movements and postures, it not only significantly enhances the effectiveness of physical training and rehabilitation programs, but also makes the interaction in digital environments more natural and intuitive. At the same time, in VR games, accurate posture reconstruction can bring a more immersive and interactive experience, bridging the gap between virtual and real. With the popularity of these applications, the demand for advanced 3D human posture reconstruction technology is increasing. This field of research has broad prospects and has far-reaching impacts on the health, sports and entertainment industries. To meet this demand, it is urgent to develop innovative technologies that can effectively capture and analyze human motion, while minimizing the impact on user comfort while improving reconstruction accuracy. In recent years, important progress has been made in the field of human posture reconstruction, especially the emergence of deep learning-based machine vision solutions. However, these methods still face many challenges, such as occlusion problems, viewpoint limitations, and sensitivity to lighting conditions.

[0003] Inertial measurement units (IMUs), which typically include accelerometers, gyroscopes, and sometimes magnetometers, play a key role in capturing human motion and posture reconstruction. By providing acceleration and angular velocity data, IMU sensors are crucial in estimating joint positions and the overall orientation of the body. In order to offset sensor drift and optimize multi-sensor data fusion, Kalman filtering and sensor fusion algorithms are traditionally used, which significantly improve the accuracy of posture reconstruction. In recent years, the introduction of deep learning models has provided a powerful tool for interpreting IMU data and facilitated the processing of nonlinear relationships between sensor outputs and actual human motion. The development of deep learning enables effective learning from large amounts of IMU data, thereby achieving accurate reconstruction of complex human motion.

[0004] However, IMU-based posture reconstruction faces many challenges in practical applications, such as inaccurate sensor calibration and external signal interference, which may affect the stability and accuracy of the system. At the same time, how to find a balance between the number of sensors and user comfort when designing wearable devices remains a key issue that needs to be solved. Some current IMU-based wearable sports solutions use a small number of sensors (such as six IMUs) to estimate the whole body posture. Although effective, they usually need to be installed in parts such as the knees or thighs, which may not be convenient for athletes or rehabilitation patients. In addition, these solutions are generally not suitable for complex sports scenarios, so further exploration and verification are still needed in fields such as sports science.

[0005] On the other hand, pressure sensors, with their high sensitivity and accuracy, can capture subtle changes in pressure, providing higher data granularity for posture reconstruction. Integrating pressure sensors into wearable devices and environmental settings (such as mats and chairs) has proven to be very valuable for human posture reconstruction. These sensors detect the distribution of forces exerted by the body on different surfaces, identifying the distribution of weight and the contact points of various parts of the body. However, pressure-based posture reconstruction systems also have certain limitations, especially when detecting postures that are not directly in contact with the sensor surface (such as aerial movements). During running and walking, the tactile interaction between the foot and the ground can vividly present 2D pressure maps under different activities, providing unique insights into the dynamics of the ground, which is not possible with inertial units and visual sensors. By recording the pressure distribution when a person moves on a surface, such sensors not only reveal details of movement patterns, but also enhance the understanding of how humans interact with the environment.

[0006] Human posture reconstruction technology is currently divided into three categories, each of which has inherent defects: (1) Vision-based posture reconstruction: low recognition accuracy in occluded or low-light environments, may cause privacy issues, poor model interpretability, and strict requirements on the location of visual devices, which is not suitable for large-scale daily applications. (2) Pressure-based posture reconstruction: usually low recognition accuracy and more restrictions. (3) Wearable device-based posture reconstruction: high data collection cost and complex annotation, the device may affect the user experience (such as poor comfort).

[0007] In addition, most existing technologies fail to effectively integrate multi-sensor data, resulting in inefficient information utilization, and the potential of multimodal data fusion has not yet been fully realized. Some methods use complex machine learning or deep learning algorithms, which can improve the accuracy of posture reconstruction, but the computational cost is high and it is difficult to run in real time on mobile devices. These factors limit its widespread application in daily life. Summary of the invention

[0008] The object of the present invention is to provide a whole body posture estimation method and system based on pressure and inertial sensors to solve the problems existing in the above-mentioned prior art.

[0009] The whole body posture estimation system based on pressure and inertial sensors described in the present invention comprises: a pressure sensor, a posture reconstruction model, and three inertial sensors;

[0010] The pressure sensor is arranged on the insole to generate plantar pressure data; the three inertial sensors are used to generate real IMU data, one of which is arranged on the wrist, and the other two inertial sensors are arranged on both feet in a one-to-one correspondence;

[0011] The human body posture reconstruction algorithm fuses the real IMU data and the virtual IMU data through the following steps to perform full body posture estimation:

[0012] S1. Biological estimation based on pressure: The pressure distribution is obtained through the pressure sensor to estimate the ground reaction force GRF and the center of pressure CoP;

[0013] S2. Estimate body posture by combining ground reaction force GRF and center of pressure CoP.

[0014] The whole body posture estimation method based on pressure and inertial sensors described in the present invention utilizes the system to perform whole body posture estimation.

[0015] The whole-body posture estimation method and system based on pressure and inertial sensors described in the present invention have the advantage that only three inertial sensors combined with plantar pressure data are used to accurately capture complex human body movements. In view of the difficulty of collecting human body posture data in wearable devices, a virtual IMU data generation method is also designed. By combining virtual IMU data with the actually collected IMU data, a larger data set is constructed. This data set is used to reconstruct human body movement posture in conjunction with plantar pressure data. It can be applied to research in multiple fields such as sports science, gait analysis, virtual reality interaction, and ergonomics to further understand the human movement mechanism, and its effectiveness has been verified in multiple daily action scenes (such as running, walking, basketball, badminton, boxing, and dancing). BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the principle of the system described in the present invention.

[0017] Figure 2 It is a schematic diagram of the principle of human body posture reconstruction described in the present invention.

[0018] Figure 3 It is a bar graph of the verification results of the system described in the present invention among different individuals. DETAILED DESCRIPTION

[0019] like Figure 1 As shown, the whole body posture estimation system based on pressure and inertial sensors described in the present invention includes: a pressure sensor, a posture reconstruction model, and three inertial sensors. The pressure sensor is arranged on the insole to generate plantar pressure data. The three inertial sensors are used to generate real IMU data, one of which is arranged on the wrist, and the other two inertial sensors are arranged on both feet in a one-to-one correspondence.

[0020] The human body posture reconstruction algorithm fuses the real IMU data and the virtual IMU data through the following steps to perform full body posture estimation:

[0021] Pressure-based biological estimation: The pressure distribution is obtained through the plantar pressure sensor to estimate the ground reaction force GRF and center of pressure CoP.

[0022] First, the spatial distribution of foot pressure is collected and represented by a heat map H(x,y), where x and y represent the spatial coordinates of the insole surface area. Then, the GRF is estimated by integrating the pressure distribution of the entire insole area, formula (1):

[0023] GRF=∫∫H(x,y)d x d y #(1);

[0024] Then, the pressure center CoP is estimated using formula (2) and formula (3). The information of the pressure center reveals how the body adjusts posture and redistributes weight during dynamic processes to maintain balance and stability. These data are of great significance in improving the accuracy of dynamic motion posture estimation of the human body.

[0025]

[0026] The body posture is estimated by combining the ground reaction force GRF and the center of pressure CoP. The human body is regarded as a multi-body system composed of multiple joints, each of which corresponds to the movement of different parts of the body. The dynamics of human motion posture can be modeled through the Newton-Euler equation or Lagrangian mechanics method. Based on the data of GRF and CoP, the human motion posture reconstruction formula (4) is as follows:

[0027]

[0028] Where M(q) represents the human body mass matrix associated with the joint structure q, Represents the acceleration vector of the joint, which represents the inertial force on the system; is the Coriolis and centrifugal force matrix, which describes the rotational offset of the joint due to inertia, is the velocity vector of the joint, which describes the Coriolis force and centrifugal force generated by the relative motion of the various parts of the system; G(q) is the gravitational torque vector, J T (q) is the transposed vector of the Jacobian matrix of the system, which maps the forces in the task space back to the joint torques, and F represents the external force vector such as GRF, which represents the external force influence acting on the system.

[0029] Jacobian matrix J T (q) connects joint motion with CoP and GRF, thus showing how joint motion affects the change of CoP, as shown in formula (5):

[0030]

[0031] In activities such as walking, optimization algorithms are used to predict and enhance body movements, with a focus on body posture and gait stability. In order to accurately reconstruct posture from sensor data, physical constraints are integrated into the learning process of the neural network. The input of the model includes kinematic data of body joints, plantar GRF and CoP data. The joint position loss function improves accuracy by reducing the difference between the predicted joint position and the actual joint position. Its mean square error formula (6) is as follows, where p i and Represent the actual position and predicted position of the i-th joint, respectively, and N is the total number of joints;

[0032]

[0033] At the same time, the center of mass CoM of the body can be calculated according to the position and mass distribution of each joint of the body. The center of mass position loss formula (7) is added to ensure that the posture meets the physical constraints. h represents the estimated height of CoM from the support base, and g represents the gravitational acceleration:

[0034]

[0035] In addition, a dynamic consistency loss is added to verify that the dynamics of the estimated position conforms to Newton's laws, especially the effect of GRF on the body acceleration. Formula (8) uses the dynamic equation to calculate the loss component, where M is set to the mass matrix of the body, is the predicted acceleration of the joint:

[0036]

[0037] By integrating the above loss functions, the entire neural network loss function for estimating body posture is a combination of joint position loss, center of mass loss and dynamic consistency loss, formula (9):

[0038]

[0039] The model first uses a fully connected layer to process and integrate the input kinematic data, and then introduces the Transformer module to capture the complex temporal dependencies and interactions between different joints. The self-attention mechanism within the Transformer is particularly suitable for modeling the various spatial relationships of body joints and can effectively identify the dynamic associations between different joints. Finally, another set of fully connected layers is used to map the output of the Transformer to the predicted joint positions, thereby generating accurate posture reconstruction results.

[0040] In order to learn 3D human motion based on global feature-based dynamic chains, a scheme combining local and global modeling is designed. Due to the weak correlation between body parts, directly using the data of three IMU sensors as model input may lead to motion blur, where the data includes real IMU and virtual IMU data. To solve this problem, a local region modeling strategy is adopted to divide the human body into two main dynamic chains, such as Figure 2 As shown, to improve the accuracy of motion capture and analysis.

[0041] Lower limb and trunk chain: For the joint power chain of the lower limb and trunk, the insole pressure sensor is used to calculate the ground reaction force FRF and the center of pressure CoP, providing the necessary data for the distribution and management of the force in the lower body and core area. In addition, the inertial measurement unit IMU on the foot further enriches the data set, providing orientation and acceleration information, thus supporting the dynamic comprehensive analysis of the lower body and trunk.

[0042] Upper limb chain: The second dynamic chain consists of two arms, with wrist-worn IMUs as the primary sensors. Inertial information is critical for capturing the complex movements of the upper limbs, especially in tasks such as reaching, lifting, or throwing.

[0043] However, the main challenge faced by local region-based methods is how to maintain global consistency across body regions. To this end, global features are incorporated into the body pose network of the dynamic chain. The global features are denoted as G(IMU1, IMU2, IMU3), where G(·) represents the global feature aggregation function extracted by the fully connected layer, which roughly integrates the whole body motion information from the three IMUs. Subsequently, the global features G are combined with the input of the Physical-aware Body Pose Network, which is specifically modeled for two local dynamic chains. By fusing global and local dynamic information, the network is able to maintain coherence when reconstructing the whole body posture, making the local motion consistent with the overall body dynamics.

[0044] The obtained model is evaluated by three indicators: average joint error MPJPE, body jitter and translation. The performance of the proposed body posture reconstruction system is comprehensively evaluated:

[0045] MPJPE: Measures the spatial accuracy of pose reconstruction by calculating the average Euclidean distance between the predicted joint positions and the groundtruth positions, which is the main indicator for evaluating model accuracy.

[0046] Jitter: Measures the temporal variation between consecutive frames to evaluate the smoothness of motion prediction. A lower jitter value indicates that the motion estimation is more stable and consistent, which is an important indicator for dynamic posture analysis.

[0047] Translation: This is used to evaluate the accuracy of the model in predicting the overall body joint position in space. Even if the individual joint positions are accurately predicted, if the overall skeleton position deviates significantly from the true position, the practical applicability and accuracy of the model will be affected.

[0048] Finally, the body pose estimation network is trained end-to-end using PyTorch and PyTorch Lightning frameworks. The training uses the Adam optimizer with a batch size of 512 and a learning rate of 3e-4 for weight updates. The loss function is designed based on the comprehensive loss in step 3 to ensure accurate prediction of joint positions and consistency of the overall skeleton.

[0049] In some practical applications, the amount of data available for deep learning training may be insufficient. Therefore, the present invention also provides a method for expanding data, so that deep learning training can be performed even when there is only a small amount of real collected data, thereby better estimating the human body motion skeleton morphology.

[0050] Build a cross-modal transfer learning network to expand the training data set by integrating data from different sources, such as images, videos, and sensors. Not only does it enrich the data pool, but it also enhances the robustness of the model by simulating various real-life scenarios. In addition, the algorithm uses the large language model LLM to automatically generate motion descriptions, combined with motion synthesis technology, to improve the accuracy and realism of the posture reconstruction model. The specific implementation process is as follows:

[0051] (1) Text-to-Action Generation

[0052] The accuracy of IMU data generated from video may be limited by factors such as video resolution, lighting, occlusion, etc., especially for social media videos. To address these limitations, clear action descriptions such as "running" and "jumping" are provided through natural language text and directly used to generate virtual IMU data. This overcomes the limitations caused by changes in video quality and effectively replaces visual information.

[0053] (2) Text description generation for pose reconstruction

[0054] Apply LLMs in posture reconstruction to generate detailed descriptions of activities to better model the variability of human behavior. By generating context-specific text descriptions, such as "a strong athlete running smoothly on the track", the virtual IMU data is ensured to accurately reflect the real human movement and enhance the training effect of the model. These descriptions focus on describing the details of human movement and exclude environmental or emotional factors to support the accuracy of posture reconstruction tasks.

[0055] An advanced Large Language Model (LLM) is used to generate precise text descriptions to inform the synthesis of virtual IMU data. These descriptions, such as "a robust athlete runs smoothly on the track" or "a muscular hiker carefully adjusts their steps on uneven terrain", are used to describe body movements in detail while ignoring irrelevant environmental or emotional factors, ensuring that these descriptions are directly applicable to the pose reconstruction task. The prompt generation process is carefully designed to produce detailed and specific text guides that clarify the actions involved in various activities. For example, the descriptions include comprehensive details of upper and lower body movements, supplemented by descriptive adjectives such as "robust" and "smooth". By strictly focusing on the necessary action details and maintaining a specific length and expressiveness, these texts are tailored to support the system in accurately estimating poses without distracting irrelevant information.

[0056] (3) Motion Generation Model

[0057] 3D human motion is generated from text descriptions through a diffusion model, and a probabilistic mapping from text to motion is established using a denoising step, formula 10:

[0058]

[0059] x 0 Represents the initial skeleton joint data of human motion, q(x 0 ) represents the probability distribution of real motion skeleton joint data, the sequence x 0 ~x T Represents the intermediate potential features. The core process of the model is diffusion and back propagation. The diffusion process runs through the Markov chain mechanism, gradually adding Gaussian noise to the data until it approaches the potential Gaussian distribution N(0,1). This process is carefully controlled by a variance, expressed as formula (11) and formula (12):

[0060]

[0061] In contrast, the reverse process aims to reconstruct the original data by a single denoising step, which uses theT )=N(x T ; 0,I) to describe the inverse Markov chain, expressed as formula (13) and formula (14):

[0062]

[0063] The above operations enable the model to effectively generate high-fidelity output from the initial noisy data distribution. MotionGPT further converts text descriptions into specific motion sequences, such as "a person stands up from a sitting position and walks forward a few steps". The text is converted into motion symbols through a natural language model, and then reconstructed into a continuous motion sequence through a decoder, which truly reproduces the actions in the description.

[0064] (4) IMU data integration

[0065] In order to accurately estimate human motion, inverse kinematics (IK) is used to determine the rotational motion of each joint relative to its parent joint and the translational motion of the root joint, usually the pelvis. Inverse kinematics adjusts the rotation of the skeleton joints so that the end effectors such as hands and feet can accurately reach the preset target position in three-dimensional space. Specifically, the IK process first initializes the animation skeleton in a baseline pose, sets the target position for each joint, and then uses a robust inverse kinematics algorithm such as the Jacobi method or cyclic coordinate descent method to iteratively optimize the joint angles to achieve the specified target position. This process relies on predefined joint positions and bone hierarchies to ensure the alignment of joints with target trajectories, thereby improving the realism and accuracy of joint motion modeling.

[0066] Next, the IMUsim tool is used to calculate the joint motion acceleration and angular velocity based on the local joint rotation and root translation data obtained by inverse kinematics, thereby generating 22 strategically placed virtual IMU sensor data. In addition, IMUsim introduces noise to simulate the typical noise characteristics in real IMU data. By adding these noises, the virtual IMU data is closer to the real sensor readings, thereby improving the accuracy and authenticity of human motion dynamics modeling.

[0067] Finally, to ensure the consistency between the virtual IMU data and the real collected data, the generated virtual data is further calibrated and adjusted to match the distribution of IMU data collected by Xsens. The calibrated virtual data is combined with the real collected data to expand the initial training data set, thus significantly improving the training effect of the posture estimation system under the condition of limited data volume.

[0068] In order to verify the robustness and versatility of the whole body pose estimation system described in this invention in a variety of dynamic environments, experiments were conducted in a series of real-life activity scenarios. This evaluation aims to measure the accuracy of the system's reconstruction of human poses in different types of motion, which is critical for applications that require high-fidelity motion tracking under variable conditions. A total of 25 participants were recruited for the experiment, of which 10 completed all six specified activities, as shown in Table 1.

[0069] Table 1 Reconstruction accuracy of different activity postures

[0070] Activity running walk basketball badminton boxing dance MPJPE(cm) 4.91 5.22 7.29 11.54 10.81 7.68 <![CDATA[Jitter(10 2 m / s 3 )]]> 2.35 1.98 3.93 4.37 3.42 3.69 Translation(cm) 3.18 2.71 3.35 5.23 3.14 2.61

[0071] In each activity, participants performed a set of prescribed movements for 5 minutes in a 4m x 5m area, with the movements being as follows to reflect the typical dynamic patterns of each sport or activity:

[0072] (1) Basketball: including dribbling in place, step dribbling, three-step layup and shooting;

[0073] (2) Badminton: including forehand and backhand receiving in the front court and forehand and backhand hitting in the back court;

[0074] (3) Boxing: including straight punch, swing punch, hook punch and forward punch;

[0075] (4) Dance: including conventional dance movements.

[0076] In each activity, the accuracy of the pose reconstruction results generated by the system is evaluated by measuring the mean joint position error MPJPE, translation and jitter indicators. As shown in Table 1, the whole body pose estimation system shows high reconstruction accuracy and consistency.

[0077] According to the analysis of the results, the following conclusions were drawn: running and walking are cyclical sports with relatively stable movement patterns, while badminton, boxing, etc. are highly dynamic and highly variable sports with dramatic speed changes and acceleration and deceleration in a short period of time, and the transitions between movements are rapid and large. Therefore, in these highly variable sports, the accuracy of posture reconstruction is lower than that of running and walking.

[0078] Without using virtual IMU data, higher MPJPE values ​​were observed in sports such as basketball with an MPJPE of 10.5cm, badminton with an MPJPE of 14.3cm, and boxing with an MPJPE of 14.8cm. This indicates that the accuracy of posture estimation is significantly reduced in these dynamic and fast-paced sports. In contrast, more predictable and repetitive sports such as running with an MPJPE of 6.3cm and walking with an MPJPE of 6.4cm showed lower errors. This shows that the model performs well in regular and repetitive movement patterns, but performs poorly in situations with high unpredictability and fast movements such as basketball, badminton, and boxing. In order to further verify the performance variations of the system between different individuals during specific operations, the leave-one-out method was used for analysis, and the results are shown in the figure. Figure 3 shown.

[0079] Although the model shows good generalization across different users, it is also observed that deep learning-based models may face certain challenges when the tasks become more complex or involve untrained movements. As shown in Table 2, the performance of the model varies significantly across activities, reflecting the complexity and variability of human movement patterns. This finding emphasizes the need to fully consider movement diversity when designing and training pose estimation systems to improve the system's adaptability to unknown movement patterns.

[0080] To this end, it is necessary to verify the scheme after adding virtual IMU data, and the results are shown in Table 2. The introduction of virtual IMU data significantly enhances the diversity and quantity of training examples. It is worth noting that the MPJPE values ​​have decreased in all activities, especially in high-dynamic sports. For example, the MPJPE value in basketball decreased from 10.5cm to 9.2cm, while the MPJPE value in boxing decreased from 14.8cm to 12.5cm. This result shows that the introduction of virtual data can effectively improve the generalization ability of the model in a wider range of human activities, thereby enhancing its performance in different sports scenarios.

[0081] Table 2. Comparison of leave-one-out validation performance with and without virtual IMU data

[0082]

[0083]

[0084] Accurate reconstruction of human posture is essential in sports science, especially in outdoor environments and dynamic sports. However, current posture reconstruction methods have various problems. In contrast, the system is able to achieve detailed 3D posture reconstruction, which is essential for advanced sports training and performance evaluation. The reconstruction system can not only provide coaches with real-time posture and technique feedback of athletes in various sports activities to improve training effects, but also play a role in equipment design. A deeper understanding of the interaction between athletes and their equipment can promote innovation, improve sports performance, and reduce the risk of injury.

[0085] The method for estimating whole-body posture based on pressure and inertial sensors described in the present invention utilizes the system to perform whole-body posture estimation.

[0086] Compared with the prior art, the present invention has at least the following advantages:

[0087] (1) It can efficiently and accurately reconstruct complex body movements.

[0088] (2) By leveraging the text understanding and representation capabilities of a large language model and combining it with motion generation technology, the accuracy and authenticity of the generated virtual IMU are improved.

[0089] (3) We utilize a variety of human motion postures, focusing on daily movements (such as running, walking, basketball, boxing, etc.), and by reconstructing specific skeleton morphologies, we verify the effectiveness and robustness of posture estimation in accurately capturing and analyzing complex motions in various dynamic activities.

[0090] For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of the present invention.

Claims

1. A whole body posture estimation system based on pressure and inertial sensors, characterized in that: include: pressure sensor, posture reconstruction model, and three inertial sensors; The pressure sensor is arranged on the insole to generate plantar pressure data; The three inertial sensors are used to generate real IMU data, wherein one inertial sensor is arranged on the wrist, and the other two inertial sensors are arranged on the feet in a one-to-one correspondence; The human body posture reconstruction algorithm fuses the real IMU data and the virtual IMU data through the following steps to perform full body posture estimation: S1. Biological estimation based on pressure: The pressure distribution is obtained through the pressure sensor to estimate the ground reaction force GRF and the center of pressure CoP; S2. Estimate body posture by combining ground reaction force GRF and center of pressure CoP.

2. A whole body posture estimation system based on pressure and inertial sensors according to claim 1, characterized in that: The step S1 is specifically as follows: Collect the spatial distribution of foot pressure and represent it as a heat map H(x,y), where x and y represent the spatial coordinates within the insole surface area; By integrating the pressure distribution over the entire insole area, the ground reaction force GRF = ∫∫H(x,y)d x d y ; Estimate the center of pressure CoP based on the ground reaction force:

3. A whole body posture estimation system based on pressure and inertial sensors according to claim 2, characterized in that The step S2 is specifically as follows: Using the formula Reconstruct human body motion posture; Where M(q) represents the human body mass matrix associated with the joint structure q, represents the velocity of the joint, represents the acceleration of the joint, is the Coriolis force and centrifugal force matrix used to describe the joint rotation offset caused by inertia, G(q) is the gravitational torque vector, J T (q) is the Jacobian matrix of the body joints and CoP, and F represents the external force; Jacobian matrix J T (q) is to connect joint motion with CoP and GRF:

4. A whole body posture estimation system based on pressure and inertial sensors according to claim 3, characterized in that: In step S2, the joint position loss function improves the accuracy by reducing the difference between the predicted joint position and the actual joint position, and its mean square error formula is: Where p i represents the actual position of the i-th joint, They represent the predicted position of the i-th joint, and N is the total number of joints; The center of mass CoM of the body is calculated based on the position and mass distribution of each joint of the body, and the center of mass position loss L is added CoM Make sure the pose complies with the physical constraints; h is the estimated height of the CoM from the support base, and g is the acceleration due to gravity: A dynamic consistency loss is added to verify that the dynamics of the estimated position conform to Newton's laws, and the loss component is calculated using the dynamic equation where M is set to be the mass matrix of the body, is the predicted acceleration of the joint; Integrate each loss function to get the entire neural network loss function for estimating body posture: L total =λ1L pose +λ2L CoM +λ3L dyn ; The posture reconstruction model uses a fully connected layer to process and integrate the input kinematic data; then the Transformer is introduced to capture the temporal dependencies and interactions between different joints; finally, another set of fully connected layers is used to map the output of the Transformer to the predicted joint positions to generate the posture reconstruction results.

5. A whole body posture estimation system based on pressure and inertial sensors according to claim 4, characterized in that: The learning modeling of the posture reconstruction model is specifically as follows: Incorporating the global features into a body posture network of a dynamic chain; the dynamic chain includes a lower limb and trunk chain and an upper limb chain; The global feature is denoted as G(IMU1, IMU2, IMU3), where G(·) represents the global feature aggregation function extracted by the fully connected layer, which preliminarily integrates the whole body motion information from the three inertial sensors; Subsequently, the global feature G is combined with the input of the physics-aware body posture network to maintain coherence when reconstructing the whole body posture by fusing global and local dynamics information, making the local motion consistent with the overall body dynamics; Finally, the body pose estimation network is trained end-to-end using PyTorch and PyTorch Lightning frameworks; the training uses the Adam optimizer with a batch size of 512 and a learning rate of 3e-4 for weight updates.

6. A whole body posture estimation system based on pressure and inertial sensors according to claim 5, characterized in that: The virtual IMU data is generated based on the large language model, as follows: Text-to-action generation: Provide action descriptions through natural language text and directly generate virtual IMU data; Text description generation for pose reconstruction: Apply LLMs to generate detailed descriptions of activities in pose reconstruction; ensure that virtual IMU data accurately reflects real human motion by generating context-specific text descriptions, enhancing the training effect of the model; Motion generation model: Generate 3D human motion from text descriptions through a diffusion model, and use a denoising step to build a probabilistic mapping from text to motion: p θ (x0)∶=∫q(x0∶T)d x1 ∶T; x0 represents the initial skeleton joint data of human motion, q(x0) represents the probability distribution of real motion skeleton joint data; the sequence x0~x T Represents the intermediate latent features, and the model is completed through diffusion and back propagation; The diffusion process operates through a Markov chain mechanism, gradually adding Gaussian noise to the data until it approaches the underlying Gaussian distribution N(0,1). This process is controlled by the variance formula: In contrast, the reverse process reconstructs the original data by performing a denoising operation, which uses the T )=N(x T ; 0,I) to describe the inverse Markov chain, expressed as: p θ (x t-1 |x t )∶=N(x t-1 ;μ θ (x t ,t),Σ θ (x t ,t)); The above operations enable the model to generate high-fidelity output from the initial noisy data distribution; MotionGPT further converts the text description into a specific motion sequence, converts the text into motion symbols through the natural language model, and then reconstructs it into a continuous motion sequence through the decoder, truly reproducing the action in the description; IMU data synthesis: Use inverse kinematics to determine the rotational motion of each joint relative to its parent joint and the translational motion of the root joint; calculate the joint's motion acceleration and angular velocity based on the local joint rotation and root translation data obtained by inverse kinematics, thereby generating multiple strategically arranged virtual IMU sensor data; at the same time, introduce noise to simulate the typical noise characteristics in real IMU data; Calibrate and adjust the generated virtual data to match the collected IMU data distribution; The calibrated virtual data is combined with the real collected data to expand the initial training dataset.

7. The whole body posture estimation method based on pressure and inertial sensors according to claim 1, characterized in that: Full body posture estimation is performed using the system described in any one of claims 1-6.

Images