Exercise load intensity evaluation method, system and equipment based on micro-motion feature extraction and medium

Abstract

The invention relates to an exercise load intensity evaluation method, system and device based on micro-motion feature extraction and a medium. The method comprises the following steps: performing three-dimensional reconstruction processing on multi-view synchronous video data to obtain a dynamic three-dimensional human body grid sequence and a macroscopic motion posture parameter sequence; personalizing the human body physical simulation model based on the macroscopic motion attitude parameter sequence, generating a dynamic baseline three-dimensional grid sequence, and performing geometric difference calculation on the dynamic baseline three-dimensional grid sequence and the dynamic three-dimensional human body grid sequence to obtain a micro-motion residual field; performing multi-scale space-time geometric feature extraction on the micro-motion residual field, and separating a high-frequency geometric oscillation component through a space-time manifold decomposition method to obtain whole-body micro-motion feature representation; the whole body micro-motion features are represented, static physiological parameters are combined and input into the double-branch neural network mapping model, and predicted heart rate variability indexes and corresponding real-time motion load intensity are obtained. By adopting the method, the physiological micro-motion signal associated with the physiological load index can be extracted from the video.

Description

Technical Field

[0001] This invention belongs to the field of motion-assisted assessment technology, and in particular relates to a method, system, device and medium for assessing motion load intensity based on micro-motion feature extraction. Background Technology

[0002] With the rapid development of computer vision technology, video-based motion capture and analysis technology has emerged. It can perform macroscopic motion recognition and posture estimation by reconstructing three-dimensional human posture and surface with high precision, thereby realizing the characterization of human motion state.

[0003] Current exercise load assessments use traditional contact-based physiological sensors as the standard, collecting data through heart rate belts and electrocardiograms (ECG) and calculating heart rate variability (HRV), or directly obtaining physiological indicators through blood lactate tests. This assists video motion capture technology in recognizing and optimizing macroscopic movements, filtering out high-frequency, low-amplitude signals during exercise as noise.

[0004] However, the above methods, traditional contact sensors have problems such as device dependence, high invasiveness, and poor wearing comfort, making it difficult to apply in natural motion scenarios; existing video motion capture technology filters out micro-motion signals such as high-frequency, low-amplitude tremors or decreased stability caused by neuromuscular fatigue as noise, missing key assessment basis, making it difficult to meet the requirements of accurate, real-time, and non-contact motion load assessment. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, system, device, and medium for assessing the intensity of exercise load based on micro-motion feature extraction, which can quantitatively assess high-intensity exercise load by establishing visual micro-motion features, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for assessing exercise load intensity based on micro-motion feature extraction, including:

[0007] Acquire multi-view synchronous video data of the individual under test, and process the multi-view synchronous video data through 3D reconstruction to obtain the corresponding dynamic 3D human body mesh sequence and macroscopic motion posture parameter sequence.

[0008] Based on the macroscopic motion posture parameter sequence, the pre-constructed human physical simulation model is personalized to generate a dynamic baseline three-dimensional mesh sequence that simulates ideal smooth motion. The dynamic three-dimensional human mesh sequence and the dynamic baseline three-dimensional mesh sequence are then geometrically differentially calculated to obtain the micro-motion residual field.

[0009] Multi-scale spatiotemporal geometric features of the micro-motion residual field are extracted, and the high-frequency geometric oscillation components in the multi-scale spatiotemporal geometric features are separated by the spatiotemporal manifold decomposition method to obtain the whole-body micro-motion feature representation;

[0010] The micro-motion characteristics of the whole body are represented and combined with the static physiological parameters of the individual to be tested. The data are then input into a pre-trained two-branch neural network mapping model to obtain a predicted heart rate variability index. Based on the predicted heart rate variability index, the real-time exercise load intensity of the individual to be tested is evaluated.

[0011] In one embodiment, a pre-constructed human physical simulation model is personalized based on a macroscopic motion posture parameter sequence to generate a dynamic baseline 3D mesh sequence simulating ideal smooth motion. The dynamic 3D human mesh sequence and the dynamic baseline 3D mesh sequence are then geometrically differencing to obtain a micro-motion residual field, including:

[0012] Based on the mesh parameters of the first frame in the dynamic 3D human body mesh sequence, a personalized physical model of mass-spring-damper is constructed with mesh vertices as mass points and mesh edges and anatomical priors as springs. Based on the individual body shape parameters of the individual to be tested, the initial physical parameters of the personalized physical model of mass-spring-damper are obtained.

[0013] The macroscopic motion attitude parameter sequence is smoothed by low-frequency filtering to obtain the smoothed macroscopic motion driving sequence.

[0014] Using the smoothed macroscopic motion driving sequence as motion constraints, and combining it with the personalized physical model of mass-spring-damper based on initialized physical parameters, the dynamic equations are solved to obtain the dynamic baseline three-dimensional mesh sequence; the dynamic baseline three-dimensional mesh sequence is the motion trajectory of each mass point under ideal smooth motion.

[0015] The micro-motion residual field is obtained by subtracting the three-dimensional position coordinates of each mass point from the dynamic three-dimensional human body mesh sequence and the dynamic baseline three-dimensional mesh sequence frame by frame.

[0016] In one embodiment, the smoothed macroscopic motion-driven sequence is used as the motion constraint, and the dynamic equations are solved by combining a personalized physical model of mass-spring-damper based on initialized physical parameters to obtain a dynamic baseline three-dimensional mesh sequence, including:

[0017] Based on the mass matrix, damping matrix, and stiffness matrix in the personalized physical model of mass-spring-damper, and combined with the constraint force vector transformed from the corresponding smoothed macroscopic motion driving sequence, the dynamic differential equation of the particle system is established.

[0018] Based on the simulation time step synchronized with multi-view synchronous video data, the implicit numerical integration method is used to discretize and solve the dynamic differential equation to obtain the predicted position of each particle.

[0019] The predicted positions of all particles at each simulation time step are recombined into a triangular mesh structure, and the triangular mesh structures are arranged in chronological order to obtain a dynamic baseline three-dimensional mesh sequence.

[0020] In one embodiment, multi-scale spatiotemporal geometric features are extracted from the micro-motion residual field, and high-frequency geometric oscillation components in the multi-scale spatiotemporal geometric features are separated by spatiotemporal manifold decomposition to obtain a whole-body micro-motion feature representation, including:

[0021] Based on geodesic distance, local neighborhoods of multiple spatial scales are obtained with each particle in the micro-motion residual field as the center, and combined with short time windows intercepted along the time dimension, multi-scale spatiotemporal three-dimensional blocks are constructed.

[0022] Based on the residual distribution of the micro-motion residual field in each time slice within each spatiotemporal three-dimensional block, local surface fitting is performed, and the average curvature, Gaussian curvature and maximum shear strain based on the strain tensor of the fitted surface are calculated to obtain the local geometric feature sequence.

[0023] Based on the local geometric feature sequences of each mass point at each spatial scale on all time slices, a feature vector sequence is formed according to the time sequence. The feature vector at the current time point in the feature vector sequence is used as the processing object, and multiple temporally adjacent feature vectors are used to construct a local sample set.

[0024] Principal component analysis is performed on the local sample set to obtain the basis vectors of the local principal component space. The feature vector at the current time point is then projected onto the local principal component space to obtain the projection coefficients of the feature vector at the current time point onto the first K principal component spaces.

[0025] Based on the projection coefficients and the basis vectors of the first K principal component spaces, the low-frequency background feature vector at the current time point is reconstructed, and the difference between the feature vector at the current time point and the low-frequency background feature vector is calculated to obtain the high-frequency geometric oscillation component at the current time point.

[0026] Based on a predefined fatigue-prone region of the human body, all high-frequency geometric oscillation components of all particles within the fatigue-prone region at various spatial scales are aggregated, and the statistics and spatial coherence of the high-frequency geometric oscillation components are calculated to obtain a representation of the whole-body micro-motion characteristics.

[0027] In one embodiment, the whole-body micromotor features are represented and combined with the static physiological parameters of the individual being tested, then input into a pre-trained two-branch neural network mapping model to obtain a predicted heart rate variability index, including:

[0028] The whole-body micro-motion features are represented in the time dimension as a spatiotemporal graph with fatigue-prone areas of the human body as graph nodes, anatomical connections as spatial edges, and temporal continuity as temporal edges. The spatiotemporal graph data is then input into the spatiotemporal graph convolutional network of the visual feature branch to obtain the visual encoding vector.

[0029] The individual's static physiological parameters are input into the fully connected network of the physiological prior branch to obtain the physiological context encoding vector;

[0030] The visual encoding vector and the physiological context encoding vector are concatenated and mapped to a predictive heart rate variability index through a fully connected regression layer.

[0031] Secondly, this application also provides a motion load intensity assessment system based on micro-motion feature extraction, comprising:

[0032] The real-time monitoring module is used to acquire multi-view synchronous video data of the individual under test, and to obtain the corresponding dynamic three-dimensional human body mesh sequence and macroscopic motion posture parameter sequence by processing the multi-view synchronous video data through three-dimensional reconstruction.

[0033] The micro-motion module is used to personalize the pre-built human physical simulation model based on the macro-motion posture parameter sequence, generate a dynamic baseline 3D mesh sequence that simulates ideal smooth motion, and perform geometric difference calculation between the dynamic 3D human mesh sequence and the dynamic baseline 3D mesh sequence to obtain the micro-motion residual field.

[0034] The feature module is used to extract multi-scale spatiotemporal geometric features from the micro-motion residual field, and to separate the high-frequency geometric oscillation components in the multi-scale spatiotemporal geometric features by spatiotemporal manifold decomposition to obtain the whole-body micro-motion feature representation.

[0035] The load intensity assessment module is used to represent the micro-motion features of the whole body, combine them with the static physiological parameters of the individual to be tested, input them into a pre-trained two-branch neural network mapping model, obtain the predicted heart rate variability index, and evaluate the real-time exercise load intensity of the individual to be tested based on the predicted heart rate variability index.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above-described motion load intensity assessment methods based on micro-motion feature extraction.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described motion load intensity assessment methods based on micro-motion feature extraction.

[0038] The aforementioned method, system, equipment, and medium for assessing exercise load intensity based on micro-motion feature extraction acquire multi-view videos of an individual and reconstruct them into a dynamic three-dimensional mesh sequence and macroscopic posture parameters. The macroscopic posture parameters drive a personalized physical simulation model to generate a dynamic baseline mesh sequence simulating ideal smooth motion. Geometric difference is performed between the actual observed mesh and the baseline mesh to remove macroscopic motion components, resulting in a residual field dominated by micro-motion. Multi-scale spatiotemporal geometric features are extracted from this residual field, and residual low-frequency deformation interference is filtered out using spatiotemporal manifold decomposition, separating pure high-frequency geometric oscillation components to construct a robust representation of whole-body micro-motion features against non-rigid deformation. This whole-body micro-motion feature representation, along with individual static physiological parameters, is input into a pre-trained bi-branch neural network mapping model to establish a cross-modal quantitative mapping from visual micro-motion to heart rate variability indicators. This enables real-time, non-contact quantitative assessment of exercise load intensity, stably extracting physiological micro-motion signals with extremely low signal-to-noise ratios from conventional videos and successfully correlating them with core physiological load indicators. This achieves accurate and quantitative assessment of high-intensity exercise load based on visual input. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating the motion load intensity assessment method based on micro-motion feature extraction of the present invention.

[0041] Figure 2 This is a flowchart illustrating the steps of step S102.

[0042] Figure 3 This is a flowchart illustrating the steps of step S203.

[0043] Figure 4 This is a structural diagram of the motion load intensity assessment system based on micro-motion feature extraction of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] In one embodiment, such as Figure 1As shown, a method for assessing motion load intensity based on micro-motion feature extraction is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0046] S101. Acquire multi-view synchronous video data of the individual to be tested, and process the multi-view synchronous video data through three-dimensional reconstruction to obtain the corresponding dynamic three-dimensional human body mesh sequence and macroscopic motion posture parameter sequence.

[0047] This illustration demonstrates how a multi-view camera system surrounding a moving individual acquires synchronized video data from multiple perspectives. The number of perspectives meets the parallax requirements for 3D reconstruction, ensuring coverage of the entire range of motion. The camera frame rate must meet the sampling integrity requirements of micro-motion signals, meaning the frame rate must be at least twice the highest frequency of micro-motion to avoid signal sampling distortion. Video data synchronization can be achieved through hardware or software synchronization. Hardware synchronization relies on a synchronization trigger module to ensure simultaneous acquisition by all cameras, while software synchronization uses timestamp calibration technology to achieve time alignment of video frames from different perspectives.

[0048] Furthermore, various 3D reconstruction techniques can be used to process the synchronously acquired multi-view video data, including point cloud-based dynamic fusion technology, parametric human model-driven technology, or deep learning end-to-end reconstruction technology, to output a temporally coherent and vertex-indexed dynamic 3D human mesh sequence. This sequence can accurately represent the dynamic geometric shape of the human body surface. At the same time, a macroscopic motion posture parameter sequence is output. This parameter sequence can be represented by rotation matrices, Euler angles, or posture coefficients of the parametric human model, and is used to describe the macroscopic motion state of the human joints.

[0049] S102. Based on the macroscopic motion posture parameter sequence, the pre-constructed human body physical simulation model is personalized to generate a dynamic baseline three-dimensional mesh sequence that simulates ideal smooth motion. The dynamic three-dimensional human body mesh sequence and the dynamic baseline three-dimensional mesh sequence are geometrically differentially calculated to obtain the micro motion residual field.

[0050] Optionally, the pre-built human physical simulation model can be a mass-spring-damper system, a finite element model, or a biomechanical model based on human anatomy. The model personalization process is based on human body shape parameters and musculoskeletal anatomy features, adjusting the model's structural parameters and physical properties to match the physical characteristics of the individual being simulated, ensuring the accuracy of the motion simulation. The dynamic baseline 3D mesh sequence for ideal smooth motion is obtained by filtering out high-frequency micro-motion components from the macro-motion posture parameter sequence, retaining only the smooth trend of macro-motion changes. High-frequency component filtering can be achieved through low-pass filtering, time-domain smoothing, or kinematic interpolation. The processed macro-motion driving sequence is used to control the personalized human physical simulation model, simulating an ideal smooth motion state without neuromuscular fatigue micro-motion interference, ultimately outputting the dynamic baseline 3D mesh sequence.

[0051] The geometric difference calculation between the dynamic 3D human body mesh sequence and the dynamic baseline 3D mesh sequence is used to obtain the geometric morphological differences between the two at the same time step. The difference calculation can be achieved through methods such as vertex-by-vertex 3D coordinate difference, surface normal vector difference analysis, or local region deformation error calculation. The calculated results constitute a micro-motion residual field. This residual field contains high-frequency, low-amplitude signals from the motion process of the individual being measured, mainly covering physiological micro-motions, model fitting errors, and noise introduced by 3D reconstruction.

[0052] S103. Multi-scale spatiotemporal geometric features are extracted from the micro-motion residual field, and the high-frequency geometric oscillation components in the multi-scale spatiotemporal geometric features are separated by the spatiotemporal manifold decomposition method to obtain the whole-body micro-motion feature representation.

[0053] Furthermore, the extraction of multi-scale spatiotemporal geometric features combines information from both spatial and temporal dimensions. Spatially, multiple local neighborhoods of varying ranges are defined centered on each particle in the micro-motion residual field, based on methods such as Euclidean distance, geodesic distance, or human functional zoning, thus achieving spatial scale division. Temporally, a fixed-length short-time window is extracted centered on the current time frame. The length of the time window can be adaptively adjusted according to the periodic characteristics of the micro-motion to ensure complete capture of the temporal changes of the micro-motion. By combining spatial neighborhoods and time windows, multi-scale spatiotemporal data blocks are constructed, and geometric features are extracted from each spatiotemporal data block. The extracted geometric features may include curvature features, strain features, displacement rate of change, or frequency domain features, etc.

[0054] For example, spatiotemporal manifold decomposition is used to process the extracted multi-scale spatiotemporal geometric features to distinguish between high-frequency oscillation components and low-frequency background components in the features. Spatiotemporal manifold decomposition can be achieved through algorithms such as moving principal component analysis, wavelet transform, Fourier frequency domain decomposition, or high-low frequency separation of time series signals. These algorithms remove the low-frequency background components from the multi-scale spatiotemporal geometric features, retaining the high-frequency geometric oscillation components that can reflect physiological micro-movements.

[0055] The whole-body micro-motion feature representation is achieved through region aggregation. That is, based on the principles of human movement physiology, human body regions sensitive to exercise fatigue are divided, and the corresponding micro-motion features can effectively reflect the exercise load state. The high-frequency geometric oscillation components of all particles in each sensitive region are aggregated. The aggregation method can include statistical feature calculation, spatial coherence analysis, or feature weighted fusion, etc., and finally form a fixed-dimensional whole-body micro-motion feature representation. This feature representation can comprehensively and accurately depict the micro-motion state of the whole body.

[0056] S104. Represent the whole-body micro-movement features, combine them with the static physiological parameters of the individual to be tested, input them into a pre-trained dual-branch neural network mapping model, obtain the predicted heart rate variability index, and evaluate the real-time exercise load intensity of the individual to be tested based on the predicted heart rate variability index.

[0057] Specifically, the pre-trained dual-branch neural network mapping model includes a visual feature branch and a physiological prior branch. This model can utilize architectures such as Transformer fusion networks, convolutional neural network-long short-term memory hybrid models, or fully connected networks with enhanced attention mechanisms. The visual feature branch processes the spatiotemporal sequence data corresponding to the whole-body micro-motion feature representations, uncovering the correlation between micro-motion features and heart rate variability indicators. The physiological prior branch receives the static physiological parameters of the individual being tested, including age, sex, body type, and physiological indicators at rest. These parameters are encoded through a fully connected network, providing individual-specific information for predicting heart rate variability indicators.

[0058] For example, during model training, the timestamps of the whole-body micro-motion feature sequences and physiological signal sequences are precisely aligned, and training samples are generated using a sliding window method. The prediction of heart rate variability (HRV) indices is based on the fusion output of a dual-branch network. The predicted HRV indices can be time-domain, frequency-domain, or nonlinear indices. The time-domain indices are... ,in, Indicates adjacent The interval is the root mean square of the time difference between two consecutive heartbeats. Indicates the sliding window The total number of intervals Indicates the first indivual The duration of the interval Indicates the first indivual The duration of the interval; frequency domain indicators are ,in, This represents the ratio of low-frequency power to high-frequency power. The power spectral density function of the RR interval signal is represented. Indicates the starting frequency of the low-frequency band. Indicates the termination frequency of the low-frequency band. Indicates the starting frequency of the high-frequency band. This indicates the termination frequency of the high-frequency band. Through fusion calculations using a two-branch network, a predicted heart rate variability index is output.

[0059] Optionally, the assessment of exercise load intensity is based on the predicted heart rate variability index, combined with the baseline heart rate variability value of the individual under test at rest, and is achieved by calculating the degree of deviation between the predicted index and the baseline value. The assessment method can adopt a graded classification or a continuous quantitative scoring method. The graded classification divides the exercise load intensity into different levels according to the degree of deviation, while the continuous quantitative scoring method uses a specific algorithm to convert the degree of deviation into a quantitative load index, and finally outputs the real-time exercise load intensity assessment result of the individual under test.

[0060] In the aforementioned method for assessing exercise load intensity based on micro-motion feature extraction, multi-view synchronous video data of the individual under test is collected and three-dimensionally reconstructed to obtain a dynamic three-dimensional human body mesh sequence and a macroscopic motion posture parameter sequence. Based on the macroscopic motion posture parameter sequence, a pre-constructed human physical simulation model is personalized to generate a dynamic baseline three-dimensional mesh sequence simulating ideal smooth motion. By geometrically differencing the dynamic three-dimensional human body mesh sequence with the dynamic baseline three-dimensional mesh sequence, a micro-motion residual field is obtained. Multi-scale spatiotemporal geometric features are extracted from the micro-motion residual field, and high-frequency geometric oscillation components are separated using the spatiotemporal manifold decomposition method to form a whole-body micro-motion feature representation. This whole-body micro-motion feature representation, along with the static physiological parameters of the individual under test, is input into a pre-trained bi-branch neural network mapping model to obtain a predicted heart rate variability index, which is then used to assess real-time exercise load intensity. This achieves non-contact physiological load assessment, successfully solving the core problem of stably separating low signal-to-noise ratio physiological micro-motion signals from a dynamic, non-rigid three-dimensional surface. It achieves personalized and physiologically accurate predictions, while providing whole-body load-related insights, thus improving the applicability, reliability, and practical application value of the assessment method.

[0061] In one embodiment, a pre-constructed human physical simulation model is personalized based on a macroscopic motion posture parameter sequence to generate a dynamic baseline 3D mesh sequence simulating ideal smooth motion. The dynamic 3D human mesh sequence and the dynamic baseline 3D mesh sequence are then geometrically differencing to obtain a micro-motion residual field, including:

[0062] S201. Based on the mesh parameters of the first frame in the dynamic three-dimensional human body mesh sequence, construct a personalized physical model of mass-spring-damper with mesh vertices as mass points and mesh edges and anatomical priors as springs. Based on the individual body shape parameters of the individual to be tested, obtain the initial physical parameters of the personalized physical model of mass-spring-damper.

[0063] Indicatively, the first frame of a dynamic 3D human body mesh sequence contains the static geometric structure information of the individual being tested. Based on this, when constructing the mass-spring-damper physical model, each vertex of the first frame mesh is directly mapped to a mass point in the model, ensuring the consistency between the model's geometry and the actual surface morphology of the human body. The setting of spring connections requires a dual approach: firstly, based on the edge connections of the mesh's own triangular facets, ensuring the structural integrity and topological consistency of the model; secondly, referencing prior knowledge of human anatomy, optimizing the spring distribution based on the connection paths of muscles, fascia, and bones, so that the spring connections can simulate the actual mechanical transmission relationships of human tissues.

[0064] The core of model personalization lies in the adjustment of initial physical parameters, including the mass of each particle, the spring constant, and the damping coefficient of the damper. Individual body shape parameters provide the basis for personalization of physical parameters. The corresponding density is determined based on the tissue type of the human body part to which the particle belongs, such as bone, muscle, or fat, and the mass of the particle is calculated in combination with the volume of that part. Based on the differences in the elastic properties of tissues in the human anatomical structure, corresponding spring constants are assigned to different types of springs, such as hard springs simulating bone connections and soft springs simulating soft tissue connections. The damping coefficient is set based on the viscous properties of human tissue to ensure that the model can realistically reflect the mechanical response of human movement, ultimately forming a personalized physical model that accurately matches the body characteristics of the individual being tested.

[0065] S202. Perform low-frequency filtering and smoothing on the macroscopic motion attitude parameter sequence to obtain the smoothed macroscopic motion driving sequence.

[0066] For example, the macroscopic motion posture parameter sequence contains the basic trend of macroscopic motion and superimposed high-frequency micro-motion components. Low-frequency filtering and smoothing preserves the smooth changes in macroscopic motion while filtering out high-frequency micro-motion interference. Filtering can employ methods such as low-pass filtering, Gaussian smoothing, or time-domain moving average. This involves setting a frequency threshold or smoothing window to allow macroscopic motion frequency components below the threshold to pass through, while suppressing high-frequency micro-motion components above the threshold. During processing, it is crucial to ensure the integrity of the macroscopic motion is not compromised. The smoothed macroscopic motion driving sequence must accurately reflect the overall motion trajectory and posture change trend of the human joints, avoiding distortion of macroscopic motion information due to over-smoothing. This sequence will serve as input for subsequent dynamic solutions in the physical model, providing motion constraints for simulating ideal smooth motion.

[0067] S203. Using the smoothed macroscopic motion driving sequence as motion constraints, and combining it with the personalized physical model of mass-spring-damper based on the initialized physical parameters, the dynamic equations are solved to obtain the dynamic baseline three-dimensional mesh sequence; the dynamic baseline three-dimensional mesh sequence is the motion trajectory of each mass point under ideal smooth motion.

[0068] Furthermore, the smoothed macroscopic motion drive sequence is transformed into motion constraints of a mass-spring-damper personalized physical model. These constraints are applied to the model in the form of constraint force vectors to simulate the macroscopic motion drive of human joints. The establishment of the dynamic equations is based on the mechanical equilibrium principle of the mass system.

[0069] Optionally, the dynamic equations are solved using implicit numerical integration, which effectively ensures the stability of the motion simulation of complex mechanical systems and avoids numerical oscillations. During the solution process, the simulation time steps are strictly aligned with the time frames of the multi-view synchronized video data to ensure that the solution results at each time step match the corresponding video frame time. Through iterative solution step by step, the position, velocity, and acceleration information of each particle under ideal smooth motion is obtained, i.e., the ideal motion trajectory of each particle. The ideal positions of all particles at each simulation time step are reorganized according to the vertex index and triangular patch topological relationship of the dynamic 3D human body mesh sequence to form a complete triangular patch mesh structure. The mesh structures of each time step are arranged in chronological order to finally obtain the dynamic baseline 3D mesh sequence, which completely simulates the ideal smooth motion state without high-frequency micro-motion interference.

[0070] S204. Subtract the three-dimensional position coordinates of each mass point from the dynamic three-dimensional human body mesh sequence and the dynamic baseline three-dimensional mesh sequence frame by frame to obtain the micro-motion residual field.

[0071] For example, the geometric difference calculation between the dynamic 3D human body mesh sequence and the dynamic baseline 3D mesh sequence must ensure temporal and spatial consistency. Temporal consistency requires calculations to be performed on both sets of meshes within the same time frame, while spatial consistency requires calculations to be performed on vertices (particles) with identical indices in both sets of meshes, ensuring that the difference results accurately reflect the geometric differences of the same human body part at the same time. Specifically, for each vertex in each time frame, the 3D position coordinates of that vertex in the dynamic 3D human body mesh sequence and the corresponding vertex in the dynamic baseline 3D mesh sequence are extracted, and the residual vector of that vertex in the current time frame is obtained through vector subtraction. The residual vectors of all vertices in all time frames are integrated to form a spatiotemporally continuous micro-motion residual field. This residual field concentrates high-frequency, low-amplitude signals in the dynamic 3D human body mesh sequence that are not represented by the dynamic baseline 3D mesh sequence, mainly including the physiological micro-motions of the individual being tested, while also covering model fitting errors and noise components introduced during 3D reconstruction.

[0072] In one embodiment, the smoothed macroscopic motion-driven sequence is used as the motion constraint, and the dynamic equations are solved by combining a personalized physical model of mass-spring-damper based on initialized physical parameters to obtain a dynamic baseline three-dimensional mesh sequence, including:

[0073] S301. Based on the mass matrix, damping matrix, and stiffness matrix in the personalized physical model of mass-spring-damper, and combined with the constraint force vector transformed from the corresponding smoothed macroscopic motion driving sequence, the dynamic differential equation of the particle system is established.

[0074] Indicatively, the mechanical parameter matrices of a personalized physical model of a mass-spring-damper system include a mass matrix, a damping matrix, and a stiffness matrix, which together determine the model's dynamic response characteristics. The mass matrix is ​​a diagonal matrix, with its diagonal elements corresponding to the mass of each mass point in the model. The matrix dimension is consistent with the total number of mass points, ensuring that the inertial characteristics of each mass point are independently represented. The damping matrix describes the energy dissipation during the system's motion and can be constructed using a proportional damping model. This is achieved by combining the mass matrix and stiffness matrix in a specific ratio, simplifying calculations and accurately reflecting the viscous damping characteristics of human tissue. The stiffness matrix is ​​constructed in a one-to-one correspondence with the spring distribution. The elements in the matrix are assigned values ​​according to the spring constants. For a spring connecting two mass points, its spring constants are filled into the corresponding positions in the stiffness matrix at the indices of the two mass points, forming a symmetrical stiffness matrix that accurately represents the elastic restoring force characteristics of the system.

[0075] Specifically, the smoothed macroscopic motion drive sequence needs to be transformed into a constraint force vector acting on the particle system. This transformation process is based on the principles of human kinematics, converting the joint motions corresponding to the macroscopic motion posture parameters into forced motion constraints on the corresponding skeletal segment particles in the model, thus forming a constraint force vector. The dimension of the constraint force vector is consistent with the total number of particles, and each element corresponds to the magnitude and direction of the constraint force on a single particle, ensuring that the macroscopic motion drive can be accurately transmitted to the entire particle system. Based on the mechanical parameter matrix and the constraint force vector, the dynamic differential equation of the particle system is established, i.e. ,in, This is the mass matrix, representing the inertial properties of each mass point; Let be the acceleration vector of each particle, describing the rate of change of the particle's velocity; The damping matrix characterizes the energy dissipation capability of the system. Let be the velocity vector of each particle, describing the speed and direction of the particle's motion; is the stiffness matrix, which characterizes the elastic recovery capability of the system; This is the current position vector of each particle, describing the real-time coordinates of the particle in three-dimensional space; Let be the resting position vector of each particle, describing the initial equilibrium position of the particle when it is not subjected to external forces; The constraint force vector represents the forced constraint force driving the transformation of macroscopic motion.

[0076] S302. Based on the simulation time step synchronized with the multi-view synchronous video data, the implicit numerical integration method is used to discretize and solve the dynamic differential equation to obtain the predicted position of each mass point.

[0077] Furthermore, the simulation time step is set to maintain consistency with the temporal resolution of the multi-view synchronized video data, ensuring strict synchronization between the time progress of the dynamics solution and the video acquisition time progress. Each simulation time step corresponds to one frame of video data, achieving time alignment between the motion simulation and the actual motion process. Time step synchronization can be achieved through hardware synchronization signals or timestamp calibration technology, ensuring that the error between the simulation time and the video acquisition time is controlled within an acceptable range, avoiding motion trajectory deviations caused by time asynchrony.

[0078] The solution to the dynamic differential equations employs an implicit numerical integration method. By considering the relationship between the state variables of the current time step and the next time step, the solution equations are constructed, exhibiting good numerical stability and effectively handling rigid dynamic systems, avoiding numerical oscillations caused by significant differences in particle mass and spring stiffness coefficients. For example, in the discretization process, discretized dynamic equations are constructed based on the initial particle position, velocity, and other state variables. The particle acceleration at the next time step is calculated iteratively, thereby deriving the particle velocity and position at the next time step. This process is repeated iteratively step by step to obtain the predicted positions of all particles at each simulation time step. Optionally, the calculation results at each time step are validated for rationality to ensure that the particle positions conform to the physiological and geometric constraints of human motion, avoiding unrealistic situations such as particle penetration or excessive deformation. If unreasonable results occur, they are corrected by adjusting the constraint force vector or mechanical parameter matrix to ensure the accuracy and rationality of the solution results, ultimately outputting the predicted positions of each particle at each simulation time step.

[0079] S303. Recombine the predicted positions of all mass points at each simulation time step into a triangular mesh structure, and arrange the triangular mesh structures in time order to obtain a dynamic baseline three-dimensional mesh sequence.

[0080] Specifically, the predicted positions of all particles at each simulation time step are merely discrete sets of three-dimensional coordinates. These can be reassembled based on the topology of the original dynamic three-dimensional human body mesh to form a complete triangular mesh structure. During the reassembly process, the vertex index relationship and triangular mesh connection rules of the original mesh are followed. Each particle at a predicted position corresponds to a vertex with the same index in the original mesh. Following the triangular mesh construction method of the original mesh, the predicted positions of adjacent particles are connected to form triangular meshes, ensuring the topological consistency between the reassembled mesh structure and the original mesh, and guaranteeing that the geometry of the mesh can accurately simulate the human body surface. After completing the mesh reassembly for a single simulation time step, the baseline three-dimensional mesh corresponding to that time step is obtained. The baseline three-dimensional meshes corresponding to all simulation time steps are arranged sequentially in chronological order to form a dynamic baseline three-dimensional mesh sequence. Each mesh in this sequence corresponds to one time step, completely recording the dynamic changes in the geometry of the human body surface under ideal smooth motion, providing an accurate baseline reference for the subsequent extraction of micro-motion residual fields.

[0081] In one embodiment, multi-scale spatiotemporal geometric features are extracted from the micro-motion residual field, and high-frequency geometric oscillation components in the multi-scale spatiotemporal geometric features are separated by spatiotemporal manifold decomposition to obtain a whole-body micro-motion feature representation, including:

[0082] S11. Based on geodesic distance, local neighborhoods of multiple spatial scales are obtained with each mass point in the micro-motion residual field as the center, and combined with short-time windows intercepted along the time dimension, multi-scale spatiotemporal three-dimensional blocks are constructed.

[0083] In a schematic manner, the spatial feature extraction of the micro-motion residual field conforms to the surface characteristics of the human body. Specifically, geodesic distance is used as the basis for spatial scale division. Geodesic distance accurately represents the actual path length between particles on the human body's curved surface, avoiding the measurement bias caused by Euclidean distance in curved space. Centered on each particle in the micro-motion residual field, multiple local neighborhoods of varying sizes are delineated by setting different geodesic distance thresholds, forming multi-scale spatial coverage. This ensures the capture of different levels of features, from localized subtle tremors to regional coordinated micro-motions.

[0084] Feature extraction along the time dimension needs to focus on the temporal dynamic changes of micro-motions. Short-time windows are extracted along the time axis, with the window length determined based on the periodic characteristics of the micro-motions. This ensures that the temporal information of at least one micro-motion cycle is fully contained, avoiding feature loss due to excessively short windows or the introduction of redundant information due to excessively long windows. Local neighborhoods at each spatial scale are combined with short-time windows to construct multi-scale spatiotemporal 3D blocks. Each spatiotemporal 3D block contains both residual distribution information within a specific spatial range and the dynamic changes of that spatial range over continuous time, providing complete spatiotemporal data support for subsequent geometric feature extraction.

[0085] S12. Based on the residual distribution of the micro-motion residual field in each time slice within each spatiotemporal three-dimensional block, perform local surface fitting, and calculate the average curvature, Gaussian curvature and maximum shear strain based on the strain tensor of the fitted surface to obtain the local geometric feature sequence.

[0086] Furthermore, each spatiotemporal 3D block contains multiple consecutive time slices, each corresponding to the spatial residual distribution at a specific moment. For each time slice, a surface fitting algorithm is used to fit the residual distribution of the local neighborhood within that slice. The fitting process is based on the least squares principle, approximating the actual geometric shape of the residual distribution by constructing a quadratic surface model to ensure that the fitted surface accurately reflects the spatial variation trend of the local residuals. Based on the fitted quadratic surface, the mean curvature, Gaussian curvature, and maximum shear strain are calculated. The mean curvature is... , The average curvature represents the average degree of curvature of the surface at that point. and The two principal curvatures of the fitted surface correspond to the maximum and minimum curvatures of the surface in mutually perpendicular directions, respectively. The Gaussian curvature is... , The Gaussian curvature represents the type and intensity of bending of the surface at that point; its positive and negative values ​​correspond to different bending morphologies. The maximum shear strain is calculated based on the strain tensor, which is: , The strain tensor characterizes the degree of deformation in a local region; Let be the gradient tensor of the residual displacement field, which describes the rate of change of the residual displacement in space; This is the transpose of the gradient tensor. The maximum shear strain is a derivation of the eigenvalues ​​of the strain tensor, i.e. , The maximum shear strain represents the maximum shear deformation degree of a local region during the deformation process. and The two principal eigenvalues ​​of the strain tensor correspond to the maximum and minimum principal strains, respectively. The above geometric feature calculations are performed sequentially on all time slices within each spatiotemporal three-dimensional block to obtain a sequence of local geometric features for each particle at that spatial scale. This sequence fully records the dynamic changes of the geometric features over time.

[0087] S13. Based on the local geometric feature sequences of each mass point at each spatial scale on all time slices, a feature vector sequence is formed according to the time sequence. The feature vector at the current time point in the feature vector sequence is used as the processing object, and multiple temporally adjacent feature vectors are used to construct a local sample set.

[0088] Optionally, at each spatial scale, the local geometric feature sequence of each particle includes the time-varying values ​​of three features: mean curvature, Gaussian curvature, and maximum shear strain. The values ​​of these three features at the same time point are combined to form the feature vector for that time point. Then, the feature vectors from all time points are arranged in chronological order to construct the feature vector sequence of that particle at that spatial scale. Specifically, the feature vector at the current time point in the feature vector sequence is used as the processing object, and multiple temporally adjacent feature vectors are selected to jointly construct a local sample set. The selection range of adjacent feature vectors extends forward and backward from the current time point. The extension length must ensure that the sample set can cover the local temporal change trend of the feature vector sequence, while avoiding the introduction of irrelevant features from too far back in time, ensuring that the sample set can effectively reflect the local temporal context information of the features at the current time point.

[0089] S14. Perform principal component analysis on the local sample set to obtain the basis vectors of the local principal component space, and project the feature vector of the current time point onto the local principal component space to obtain the projection coefficients of the feature vector of the current time point on the first K principal component spaces.

[0090] Optionally, principal component analysis (PCA) can reduce the dimensionality of the local sample set and separate the main trend and secondary fluctuation components in the features. PCA is performed on the constructed local sample set by calculating the covariance matrix of the sample set and solving for the eigenvalues ​​and eigenvectors of the covariance matrix. The eigenvectors are the basis vectors of the local principal component space. The magnitude of the eigenvalues ​​corresponds to the variance contribution of each principal component; the higher the variance contribution, the richer the sample information contained in that principal component. The feature vector at the current time point is used as the projection vector and projected onto the obtained local principal component space. The projection process essentially decomposes the high-dimensional feature vector into the dimensions corresponding to the basis vectors of each principal component. The top K principal components whose cumulative variance contribution reaches a preset proportion are selected, and the projection coefficients of the feature vector at the current time point onto these K principal component spaces are recorded. The projection coefficients fully characterize the component distribution of the current feature vector in the direction of the main trend.

[0091] S15. Based on the projection coefficients and the basis vectors of the first K principal component spaces, reconstruct the low-frequency background feature vector at the current time point, and calculate the difference between the feature vector at the current time point and the low-frequency background feature vector to obtain the high-frequency geometric oscillation component at the current time point.

[0092] Specifically, the reconstruction of low-frequency background feature vectors is based on the previous... The information of each principal component is used to reconstruct the information by linearly combining the projection coefficients of the current time point with the corresponding principal component basis vectors. The reconstruction formula is as follows: ,in, The reconstructed low-frequency background feature vector represents the main trend of change in the features; The feature vector at the current time point is at the th Projection coefficients on each principal component; For the first basis vectors of the principal component space; The number of principal components selected. The eigenvector at the current time point contains low-frequency background components and high-frequency oscillation components. By calculating the difference between this eigenvector and the reconstructed low-frequency background eigenvector, the low-frequency background components are removed, and the high-frequency geometric oscillation components are obtained, i.e. ,in, It is a high-frequency geometric oscillation component, corresponding to the physiological micro-motion signal in the micro-motion residual field; This is the original feature vector at the current time point.

[0093] S16. Based on the predefined human fatigue-prone region, aggregate all high-frequency geometric oscillation components of all particles within the human fatigue-prone region at various spatial scales, and calculate the statistics and spatial coherence of the high-frequency geometric oscillation components to obtain the whole-body micro-motion characteristic representation.

[0094] For example, the predefined fatigue-prone regions of the human body are based on the principles of sports anatomy, selecting muscle groups that bear a large load and exhibit significant neuromuscular fatigue during exercise to sensitively reflect changes in exercise load intensity. According to predefined region division rules, the set of particles contained in each fatigue-prone region is determined, ensuring that the region boundaries are consistent with the human anatomical structure. For each fatigue-prone region, the high-frequency geometric oscillation components of all particles within that region are collected at various spatial scales, and multi-dimensional feature aggregation is performed. The aggregation process calculates the statistics of the high-frequency geometric oscillation components, including the mean, variance, and quantiles of the component amplitudes, reflecting the overall intensity and fluctuation characteristics of micro-movements within the region. Furthermore, spatial coherence is calculated by analyzing the temporal correlation of the high-frequency geometric oscillation components of different particles within the region to obtain a spatial coherence coefficient. This coefficient can distinguish between global synchronous jitter and local specific tremors, avoiding interference from non-target signals. The statistics and spatial coherence coefficients of all fatigue-prone regions are integrated to form a fixed-dimensional vector. This vector represents the whole-body micro-movement features, comprehensively and accurately depicting the key characteristics of whole-body micro-movements, providing core feature input for subsequent exercise load intensity assessment.

[0095] In one embodiment, the whole-body micromotor features are represented and combined with the static physiological parameters of the individual being tested, then input into a pre-trained two-branch neural network mapping model to obtain a predicted heart rate variability index, including:

[0096] S21. The whole-body micro-motion features are represented in the time dimension as a spatiotemporal graph with fatigue-prone areas of the human body as graph nodes, anatomical connections as spatial edges, and temporal continuity as temporal edges. The spatiotemporal graph data is then input into the spatiotemporal graph convolutional network of the visual feature branch to obtain the visual encoding vector.

[0097] Schematic representation: Whole-body micro-motion features include the aggregation of high-frequency geometric oscillation components of various fatigue-prone regions of the human body at multiple spatial scales. When organizing these features into a spatiotemporal graph, the core components of the graph structure are clearly defined. Graph nodes directly correspond to predefined fatigue-prone regions of the human body, and the feature of each node is the whole-body micro-motion feature vector corresponding to that region, ensuring that node features accurately represent the micro-motion state of the corresponding region. The construction of spatial edges is based on anatomical connections in the human body. The connection state and weight of edges are defined according to the degree of physiological correlation between fatigue-prone regions. For example, strong connection weights are set between functionally synergistic muscle groups, while weak connections or no connections are set between anatomically distant and functionally independent regions, enabling spatial edges to simulate the spatial transmission path of micro-motion features during human movement. The construction of temporal edges is based on the principle of temporal continuity, connecting nodes of the same fatigue-prone region at adjacent time steps to form temporal associations, ensuring that the spatiotemporal graph can capture the dynamic evolution of micro-motion features over time.

[0098] Furthermore, a spatiotemporal graph convolutional network is used to simultaneously mine the spatial correlation and temporal dynamic information of micro-motion features. Specifically, the spatial graph convolutional layer processes the spatial graph of each time step, adjusts the node features using the anatomical connection weights between regions, aggregates the micro-motion information of adjacent regions, and strengthens the feature correlation in the spatial dimension. The temporal convolutional layer processes the feature sequence of consecutive time steps to capture the changing trends and dependencies of micro-motion features in the temporal dimension. After multiple rounds of spatiotemporal convolution and feature transformation, the spatiotemporal features are compressed and integrated through global pooling operations, outputting a visual encoding vector with fixed dimensions and condensed information. This vector fully represents the spatiotemporal distribution and dynamic change patterns of whole-body micro-motion features.

[0099] S22. Input the individual's static physiological parameters into the fully connected network of the physiological prior branch to obtain the physiological context encoding vector.

[0100] Specifically, an individual's static physiological parameters encompass core indicators reflecting the body's physiological basis and exercise potential, characterizing individual physiological specificity from different dimensions and providing important basis for personalized prediction of heart rate variability indicators. A fully connected network transforms static physiological parameters of varying dimensions and types into a unified physiological context encoding vector that can be fused with the visual encoding vector. During network processing, the input static physiological parameters are standardized to eliminate dimensional differences between parameters, ensuring that each parameter has an equal weight contribution in the feature space. Furthermore, multiple fully connected layers perform nonlinear mapping and feature extraction on the standardized parameters. Each layer introduces nonlinear transformation capabilities through activation functions, gradually uncovering the potential correlations and interactions between physiological parameters, transforming discrete physiological parameters into continuous feature representations. The output layer outputs a fixed-dimensional physiological context encoding vector, which accurately characterizes the individual's basic physiological features, providing a suitable feature form for subsequent fusion with the visual encoding vector.

[0101] S23. The visual encoding vector and the physiological context encoding vector are concatenated and mapped to a predictive heart rate variability index through a fully connected regression layer.

[0102] Optionally, the visual encoding vector and the physiological context encoding vector are concatenated directly, with the two vectors joined end-to-end in sequence to form a fused feature vector whose dimension is the sum of the dimensions of the two vectors. This fused vector simultaneously includes micro-motor spatiotemporal features reflecting the motion state and physiological prior features reflecting individual differences, achieving a comprehensive integration of motion state information and individual physiological information, and providing sufficient feature support for the accurate prediction of heart rate variability indicators.

[0103] The high-dimensional fused feature vector is mapped to specific heart rate variability index (HRV) predicted values ​​through a fully connected regression layer. Specifically, a multi-layer fully connected network progressively reduces the dimensionality of the fused feature vector and performs feature transformation. Each layer continuously optimizes the expressive power of the features through a combination of linear transformations and activation functions, mapping the fused features to the value space of the HRV index. The HRV index prediction mapping process can be represented as follows: ,in, This is a predicted value for the heart rate variability index; This is the fused feature vector obtained by concatenating the visual encoding vector and the physiological context encoding vector; Here are the weight matrices for each fully connected layer, used to implement linear transformation of the features; These are the bias vectors for each fully connected layer, used to adjust the offset of the features; The activation function is used to introduce nonlinear transformations and enhance the feature representation ability of the network. This represents the number of layers in the fully connected regression layer.

[0104] Through the above mapping process, the fully connected regression layer outputs the final predicted heart rate variability index value. This index value can accurately reflect the current physiological load status of the individual being tested, providing core data support for the subsequent assessment of exercise load intensity.

[0105] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0106] Based on the same inventive concept, this application also provides a motion load intensity assessment system based on micro-motion feature extraction for implementing the motion load intensity assessment method based on micro-motion feature extraction described above. The solution provided by this system is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more embodiments of the motion load intensity assessment system based on micro-motion feature extraction provided below can be found in the limitations of the motion load intensity assessment method based on micro-motion feature extraction described above, and will not be repeated here.

[0107] In one exemplary embodiment, such as Figure 4 As shown, a motion load intensity assessment system based on micro-motion feature extraction is provided, including:

[0108] The real-time monitoring module 401 is used to acquire multi-view synchronous video data of the individual under test, and to obtain the corresponding dynamic three-dimensional human body mesh sequence and macroscopic motion posture parameter sequence by processing the multi-view synchronous video data through three-dimensional reconstruction.

[0109] The micro-motion module 402 is used to personalize the pre-constructed human physical simulation model based on the macro-motion posture parameter sequence, generate a dynamic baseline three-dimensional mesh sequence that simulates ideal smooth motion, and perform geometric difference calculation between the dynamic three-dimensional human mesh sequence and the dynamic baseline three-dimensional mesh sequence to obtain the micro-motion residual field.

[0110] Feature module 403 is used to extract multi-scale spatiotemporal geometric features from the micro-motion residual field, and separate the high-frequency geometric oscillation components in the multi-scale spatiotemporal geometric features by spatiotemporal manifold decomposition method to obtain the whole-body micro-motion feature representation;

[0111] The load intensity assessment module 404 is used to represent the whole-body micro-movement features, combine them with the static physiological parameters of the individual to be tested, input them into a pre-trained dual-branch neural network mapping model, obtain the predicted heart rate variability index, and evaluate the real-time exercise load intensity of the individual to be tested based on the predicted heart rate variability index.

[0112] In one embodiment, the micro-motion module 402 is further configured to:

[0113] Based on the mesh parameters of the first frame in the dynamic 3D human body mesh sequence, a personalized physical model of mass-spring-damper is constructed with mesh vertices as mass points and mesh edges and anatomical priors as springs. Based on the individual body shape parameters of the individual to be tested, the initial physical parameters of the personalized physical model of mass-spring-damper are obtained.

[0114] The macroscopic motion attitude parameter sequence is smoothed by low-frequency filtering to obtain the smoothed macroscopic motion driving sequence.

[0115] Using the smoothed macroscopic motion driving sequence as motion constraints, and combining it with the personalized physical model of mass-spring-damper based on initialized physical parameters, the dynamic equations are solved to obtain the dynamic baseline three-dimensional mesh sequence; the dynamic baseline three-dimensional mesh sequence is the motion trajectory of each mass point under ideal smooth motion.

[0116] The micro-motion residual field is obtained by subtracting the three-dimensional position coordinates of each mass point from the dynamic three-dimensional human body mesh sequence and the dynamic baseline three-dimensional mesh sequence frame by frame.

[0117] In one embodiment, a solving module is also included, for:

[0118] Based on the mass matrix, damping matrix, and stiffness matrix in the personalized physical model of mass-spring-damper, and combined with the constraint force vector transformed from the corresponding smoothed macroscopic motion driving sequence, the dynamic differential equation of the particle system is established.

[0119] Based on the simulation time step synchronized with multi-view synchronous video data, the implicit numerical integration method is used to discretize and solve the dynamic differential equation to obtain the predicted position of each particle.

[0120] The predicted positions of all particles at each simulation time step are recombined into a triangular mesh structure, and the triangular mesh structures are arranged in chronological order to obtain a dynamic baseline three-dimensional mesh sequence.

[0121] In one embodiment, the feature module 403 is further configured to:

[0122] Based on geodesic distance, local neighborhoods of multiple spatial scales are obtained with each particle in the micro-motion residual field as the center, and combined with short time windows intercepted along the time dimension, multi-scale spatiotemporal three-dimensional blocks are constructed.

[0123] Based on the residual distribution of the micro-motion residual field in each time slice within each spatiotemporal three-dimensional block, local surface fitting is performed, and the average curvature, Gaussian curvature and maximum shear strain based on the strain tensor of the fitted surface are calculated to obtain the local geometric feature sequence.

[0124] Based on the local geometric feature sequences of each mass point at each spatial scale on all time slices, a feature vector sequence is formed according to the time sequence. The feature vector at the current time point in the feature vector sequence is used as the processing object, and multiple temporally adjacent feature vectors are used to construct a local sample set.

[0125] Principal component analysis is performed on the local sample set to obtain the basis vectors of the local principal component space. The feature vector at the current time point is then projected onto the local principal component space to obtain the projection coefficients of the feature vector at the current time point onto the first K principal component spaces.

[0126] Based on the projection coefficients and the basis vectors of the first K principal component spaces, the low-frequency background feature vector at the current time point is reconstructed, and the difference between the feature vector at the current time point and the low-frequency background feature vector is calculated to obtain the high-frequency geometric oscillation component at the current time point.

[0127] Based on a predefined fatigue-prone region of the human body, all high-frequency geometric oscillation components of all particles within the fatigue-prone region at various spatial scales are aggregated, and the statistics and spatial coherence of the high-frequency geometric oscillation components are calculated to obtain a representation of the whole-body micro-motion characteristics.

[0128] In one embodiment, the load strength assessment module 404 is further configured to:

[0129] The whole-body micro-motion features are represented in the time dimension as a spatiotemporal graph with fatigue-prone areas of the human body as graph nodes, anatomical connections as spatial edges, and temporal continuity as temporal edges. The spatiotemporal graph data is then input into the spatiotemporal graph convolutional network of the visual feature branch to obtain the visual encoding vector.

[0130] The individual's static physiological parameters are input into the fully connected network of the physiological prior branch to obtain the physiological context encoding vector;

[0131] The visual encoding vector and the physiological context encoding vector are concatenated and mapped to a predictive heart rate variability index through a fully connected regression layer.

[0132] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps in the above method embodiments.

[0133] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0134] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0135] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for assessing motion load intensity based on micro-motion feature extraction, characterized in that, The method includes: Acquire multi-view synchronous video data of the individual to be tested, and process the multi-view synchronous video data through three-dimensional reconstruction to obtain the corresponding dynamic three-dimensional human body mesh sequence and macroscopic motion posture parameter sequence. Based on the macroscopic motion posture parameter sequence, the pre-constructed human body physical simulation model is personalized to generate a dynamic baseline three-dimensional mesh sequence that simulates ideal smooth motion. The dynamic three-dimensional human body mesh sequence and the dynamic baseline three-dimensional mesh sequence are then geometrically differentially calculated to obtain the micro-motion residual field. Multi-scale spatiotemporal geometric features are extracted from the micro-motion residual field, and the high-frequency geometric oscillation components in the multi-scale spatiotemporal geometric features are separated by the spatiotemporal manifold decomposition method to obtain the whole-body micro-motion feature representation; The whole-body micro-movement features are represented and combined with the static physiological parameters of the individual under test. The data are then input into a pre-trained two-branch neural network mapping model to obtain a predicted heart rate variability index. Based on the predicted heart rate variability index, the real-time exercise load intensity of the individual under test is evaluated.

2. The method according to claim 1, characterized in that, The process involves personalizing the pre-constructed human physical simulation model based on the macroscopic motion posture parameter sequence to generate a dynamic baseline 3D mesh sequence simulating ideal smooth motion. Geometric difference calculations are then performed between the dynamic 3D human mesh sequence and the dynamic baseline 3D mesh sequence to obtain a micro-motion residual field, including: Based on the mesh parameters of the first frame in the dynamic three-dimensional human body mesh sequence, a personalized physical model of mass-spring-damper is constructed with mesh vertices as mass points and mesh edges and anatomical priors as springs. Based on the individual body shape parameters of the individual to be tested, the initial physical parameters of the personalized physical model of mass-spring-damper are obtained. The macroscopic motion attitude parameter sequence is subjected to low-frequency filtering and smoothing to obtain a smoothed macroscopic motion driving sequence; Using the smoothed macroscopic motion driving sequence as motion constraints, and combining it with the mass-spring-damper personalized physical model based on the initialized physical parameters, the dynamic equations are solved to obtain the dynamic baseline three-dimensional mesh sequence; the dynamic baseline three-dimensional mesh sequence is the motion trajectory of each mass point under ideal smooth motion; The micro-motion residual field is obtained by subtracting the three-dimensional position coordinates of each mass point from the dynamic three-dimensional human body mesh sequence frame by frame and the dynamic baseline three-dimensional mesh sequence.

3. The method according to claim 2, characterized in that, The smoothed macroscopic motion-driven sequence is used as the motion constraint, and the dynamic equations are solved using the personalized mass-spring-damper physical model based on the initialized physical parameters to obtain the dynamic baseline three-dimensional mesh sequence, including: Based on the mass matrix, damping matrix, and stiffness matrix in the mass-spring-damper personalized physical model, and combined with the constraint force vector transformed from the smoothed macroscopic motion driving sequence, the dynamic differential equation of the particle system is established. Based on the simulation time step synchronized with the multi-view synchronous video data, the dynamic differential equation is discretized and solved using the implicit numerical integration method to obtain the predicted position of each particle. The predicted positions of all mass points at each simulation time step are recombined into triangular mesh structures, and the triangular mesh structures are arranged in chronological order to obtain the dynamic baseline three-dimensional mesh sequence.

4. The method according to claim 1, characterized in that, The process involves extracting multi-scale spatiotemporal geometric features from the micro-motion residual field and separating the high-frequency geometric oscillation components from the multi-scale spatiotemporal geometric features using spatiotemporal manifold decomposition to obtain a representation of whole-body micro-motion features, including: Based on geodesic distance, local neighborhoods of multiple spatial scales are obtained with each particle in the micro-motion residual field as the center, and combined with short time windows intercepted along the time dimension, multi-scale spatiotemporal three-dimensional blocks are constructed. Based on each of the aforementioned three-dimensional spatiotemporal blocks, the residual distribution of the micro-motion residual field on each time slice is fitted with a local surface, and the average curvature, Gaussian curvature and maximum shear strain based on the strain tensor of the fitted surface are calculated to obtain a local geometric feature sequence. Based on the local geometric feature sequences of each mass point at each spatial scale on all time slices, a feature vector sequence is formed according to the time sequence, and the feature vector at the current time point in the feature vector sequence is used as the processing object. Multiple temporally adjacent feature vectors are used to construct a local sample set. Principal component analysis is performed on the local sample set to obtain the basis vectors of the local principal component space. The feature vector at the current time point is then projected onto the local principal component space to obtain the projection coefficients of the feature vector at the current time point onto the first K principal component spaces. Based on the projection coefficients and the basis vectors of the first K principal component spaces, the low-frequency background feature vector at the current time point is reconstructed, and the difference between the feature vector at the current time point and the low-frequency background feature vector is calculated to obtain the high-frequency geometric oscillation component at the current time point. Based on a predefined human fatigue-prone region, all high-frequency geometric oscillation components of all particles within the human fatigue-prone region at various spatial scales are aggregated, and the statistics and spatial coherence of the high-frequency geometric oscillation components are calculated to obtain the whole-body micro-motion feature representation.

5. The method according to claim 4, characterized in that, The process of representing the whole-body micro-motion features, combining them with the static physiological parameters of the individual under test, and inputting them into a pre-trained two-branch neural network mapping model to obtain a predicted heart rate variability index includes: The whole-body micro-movement features are represented in the time dimension as a spatiotemporal graph with the fatigue-prone areas of the human body as graph nodes, anatomical connections as spatial edges, and temporal continuity as temporal edges. The spatiotemporal graph data is then input into a spatiotemporal graph convolutional network of the visual feature branch to obtain a visual encoding vector. The static physiological parameters of the individual are input into the fully connected network of the physiological prior branch to obtain the physiological context encoding vector; The visual encoding vector and the physiological context encoding vector are concatenated and mapped to the predicted heart rate variability index through a fully connected regression layer.

6. A motion load intensity assessment system based on micro-motion feature extraction, characterized in that, The system includes: The real-time monitoring module is used to acquire multi-view synchronous video data of the individual under test and to obtain the corresponding dynamic three-dimensional human body mesh sequence and macro-motion posture parameter sequence through three-dimensional reconstruction. The micro-motion module is used to personalize the pre-constructed human physical simulation model based on the macro-motion posture parameter sequence, generate a dynamic baseline three-dimensional mesh sequence that simulates ideal smooth motion, and perform geometric difference calculation between the dynamic three-dimensional human mesh sequence and the dynamic baseline three-dimensional mesh sequence to obtain the micro-motion residual field. The feature module is used to extract multi-scale spatiotemporal geometric features from the micro-motion residual field, and to separate the high-frequency geometric oscillation components in the multi-scale spatiotemporal geometric features by spatiotemporal manifold decomposition to obtain a whole-body micro-motion feature representation. The load intensity assessment module is used to input the whole-body micro-movement feature representation, combined with the static physiological parameters of the exercise individual to be tested, into a pre-trained dual-branch neural network mapping model to obtain a predicted heart rate variability index, and to assess the real-time exercise load intensity of the exercise individual to be tested based on the predicted heart rate variability index.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images