Gait analysis method and system based on imu and convolutional neural network with kinematic constraints

Abstract

The application discloses a gait analysis method and system based on an IMU and a kinematic constraint convolutional neural network, and relates to the technical field of gait biomechanics analysis. The method is characterized in that IMU sensors are arranged at the positions of human feet, shanks, thighs and sacrum of a trunk, a convolutional neural network is constructed, distances from the sensors to joint axes and limb segment centroid positioning are adaptively estimated in combination with static human body parameters of a subject, then gait energy estimation is performed to realize gait analysis, and gait modes such as normal walking, obstacle crossing, walking on a sandy ground, going upstairs and going downstairs can be distinguished through coupling relationships among different energies. The gait analysis method and system based on the IMU and the kinematic constraint convolutional neural network can reduce system errors caused by position deviation of sensor wearing, is suitable for rehabilitation evaluation, motion function monitoring, auxiliary identification of abnormal gait and continuous gait energy analysis in a daily environment, and does not need to depend on an optical motion capture system and a force platform.

Description

Technical Field

[0001] This invention relates to the field of gait biomechanical analysis technology, and in particular to a gait analysis method and system based on IMU and kinematically constrained convolutional neural networks. Background Technology

[0002] Gait analysis is a well-established research area in the medical and health fields. For a long time, research has primarily focused on spatiotemporal parameters (such as walking speed and cadence), kinematic parameters (such as lower limb segmental and joint range of motion), and dynamic parameters (such as joint torque). Quantitative and qualitative analyses of these indicators have been widely applied to gait anomaly identification and comprehensive assessment, and can provide support for clinical decision-making and rehabilitation treatment optimization.

[0003] Current gait analysis typically uses optical motion capture systems and force tables as standard measurement methods to acquire parameters such as human joint movement, ground reaction force, and gait energy. However, these devices are expensive, complex to install, and can only be used in laboratory environments, making them unsuitable for home, community, hospital ward, or long-term continuous monitoring scenarios.

[0004] Wearable inertial measurement units (IMUs) have advantages such as small size, low cost, ease of wear, and suitability for long-term monitoring, and have been used to estimate gait parameters such as gait speed, cadence, stride length, and joint angles. However, existing IMU-based gait analysis still has the following problems: First, most methods primarily estimate joint torques, gait spatiotemporal parameters, or local kinematic parameters.

[0005] Second, traditional model-driven methods typically require the inertial measurement unit (IMU) to be worn in an accurate position. If the sensor's position relative to the joint axis or the limb's center of mass deviates, it will lead to significant systematic errors in the lower limb energy calculation.

[0006] Third, existing methods typically lack a unified energy index that simultaneously reflects "whole-body balance control" and "local mechanical work of the lower limbs," making it difficult to effectively distinguish gait in complex scenarios. Summary of the Invention

[0007] The technical problem to be solved by this invention is to provide a gait analysis method and system based on IMU and kinematically constrained convolutional neural network, construct a kinematically constrained convolutional neural network model that can adaptively estimate limb segment parameters, and establish an integrated calculation framework for the mechanical energy of the body's center of mass oscillation and the mechanical energy of the lower limbs, so as to realize the quantitative characterization of energy coupling characteristics under multiple velocities, and realize gait analysis based on highly sensitive energy indicators.

[0008] In a first aspect, the present invention provides a gait analysis method based on an IMU and a kinematically constrained convolutional neural network, comprising: Neural network construction process: Convolutional neural network with embedded kinematic constraints is constructed and trained. The convolutional neural network with embedded kinematic constraints is used to fuse dynamic time series data of IMU sensor and static human body parameters, and output the distance from IMU sensor to joint axis. Data acquisition and preprocessing process: Raw time-series data of triaxial acceleration and triaxial angular velocity are synchronously acquired through IMU sensors. The IMU sensors are respectively set at the feet, calves, thighs and sacrum of the trunk to acquire static human body parameters, including height, weight and gender; the raw time-series data are preprocessed to obtain dynamic time-series data; The limb centroid adaptive prediction process is as follows: dynamic time series data and static human body parameters are input into a convolutional neural network with kinematic constraints to obtain the predicted distance from the sensor to the joint axis; the lower limb centroid is located by the predicted distance from the sensor to the joint axis and the static human body parameters, and then the distance from the IMU sensor to the limb centroid is calculated to obtain the coordinate transformation matrix from the IMU sensor coordinate system to the body limb centroid coordinate system. Gait energy estimation process: The oscillation energy in the forward, backward, left-right, and vertical directions is calculated based on the dynamic time-series data from the IMU sensor in the sacral region of the trunk. These three values ​​are then summed to obtain the whole-body center-of-mass oscillation energy OE. The forward kinetic energy KE of the human body during the gait cycle is calculated based on the body mass and average forward velocity. Each lower limb segment is treated as a rigid body, and for each segment, the translational energy caused by external forces, the rotational energy caused by external torques, and the change in gravitational potential energy caused by vertical height changes are calculated separately. These are then summed to obtain the lower limb mechanical energy LE.

[0009] Furthermore, it also includes a gait energy coupling analysis process: the total mechanical energy (TME) is obtained based on the whole-body centroid oscillation energy (OE), forward kinetic energy (KE), and lower limb mechanical energy (LE), and the energy proportion characteristics are analyzed. Complex gaits, including normal walking, crossing obstacles, walking on gravel, going upstairs, and going downstairs, are distinguished by the energy proportion characteristics.

[0010] Secondly, the present invention provides a gait analysis system based on an IMU and a kinematically constrained convolutional neural network, comprising: A neural network construction module is used to construct and train a convolutional neural network with embedded kinematic constraints. The convolutional neural network with embedded kinematic constraints is used to fuse dynamic time-series data from IMU sensors and static human body parameters to output the distance from the IMU sensor to the joint axis. The data acquisition and preprocessing module is used to synchronously acquire raw time-series data of triaxial acceleration and triaxial angular velocity through IMU sensors. The IMU sensors are respectively installed on the feet, calves, thighs and sacrum of the trunk to acquire static human body parameters, including height, weight and gender; and preprocess the raw time-series data to obtain dynamic time-series data. The limb centroid adaptive prediction module is used to input dynamic time series data and static human body parameters into a convolutional neural network with kinematic constraints to obtain the predicted distance from the sensor to the joint axis; the lower limb centroid is located by using the predicted distance from the sensor to the joint axis and the static human body parameters, and then the distance from the IMU sensor to the limb centroid is calculated to obtain the coordinate transformation matrix from the IMU sensor coordinate system to the body limb centroid coordinate system. The gait energy estimation module calculates the oscillatory energy in the forward, left-right, and vertical directions based on the dynamic time-series data from the IMU sensor in the sacral region of the trunk. The sum of these three values ​​yields the whole-body center-of-mass oscillation energy OE. Based on the body mass and average forward velocity, the module calculates the forward kinetic energy KE of the body during the gait cycle. Treating each lower limb segment as a rigid body, the module calculates the translational energy caused by external forces, the rotational energy caused by external torques, and the change in gravitational potential energy caused by vertical height changes for each lower limb segment. The sum of these values ​​yields the lower limb mechanical energy LE.

[0011] The technical solutions provided in the embodiments of the present invention have at least the following technical effects: 1. It has high estimation accuracy and small systematic error, effectively overcoming the shortcomings of existing unconstrained network fitting, which is easily affected by noise and placement deviation and has weak model generalization ability. This advantage is achieved by embedding the kinematic physical constraint of the consistency of instantaneous acceleration at the joint center of adjacent limb segments into the convolutional neural network. The network parameter output is constrained by biomechanical laws, abandoning the limitations of pure data fitting. It eliminates the calculation error caused by non-standard sensor layout from the algorithm level, ensuring the stability and accuracy of parameter estimation results.

[0012] 2. Convolutional neural networks with embedded kinematic constraints can effectively learn the nonlinear mapping relationship between the distance from the sensor to the joint axis and human parameters and IMU dynamic data. Compared with the traditional least squares method, it reduces computational complexity and improves noise resistance while ensuring the accuracy of the distance calculation from the sensor to the joint axis, providing high-precision parameter support for subsequent limb centroid position calculation and OE and LE energy modeling.

[0013] 3. Gait assessment is highly sensitive and comprehensive in its characterization dimensions, solving the problem that existing technologies can only calculate a single dynamic parameter and cannot quantify gait energy regulation characteristics. This advantage is achieved by relying on an integrated unified solution framework for whole-body oscillatory mechanical energy and lower limb mechanical energy, simultaneously completing the quantitative calculation and standardization of the two types of core energy indicators. Combined with the analysis of multi-velocity energy coupling characteristics, a highly sensitive energy system with both whole-body balance and lower limb function assessment capabilities is constructed, significantly improving the technical effectiveness of gait function assessment and enabling early identification of sports injuries.

[0014] 4. A gait discrimination method based on energy coupling features is proposed to achieve accurate identification of complex gaits by using energy proportion features, thereby improving the clinical effectiveness of gait assessment.

[0015] 5. It has the technical advantages of lightweight system and wide adaptability to various scenarios. It breaks through the limitations of existing gold standard technologies that rely on large laboratory equipment and cannot perform long-term outdoor monitoring. This advantage is realized based on the closed-loop architecture of pure wearable inertial measurement. The overall algorithm has low computational load and low hardware cost. It can be stably applied to various non-laboratory scenarios such as clinical rehabilitation and home monitoring. While ensuring the accuracy of assessment, it realizes the portability and large-scale implementation of gait energy analysis technology.

[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0018] Figure 1 This is a schematic diagram of the hardware architecture of an embodiment of the present invention; Figure 2 This is a flowchart illustrating the overall process of the method in Embodiment 1 of the present invention. Figure 3 This is a schematic diagram of the convolutional neural network architecture with embedded kinematic constraints in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram illustrating the differentiated IMU installation location settings in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the training convergence curve of Fold2 when training the network using five-fold cross-validation in Embodiment 1 of the present invention; Figure 6 This is a schematic diagram showing the height change of the centroid of a limb along the direction of gravity in Embodiment 1 of the present invention; Figure 7 The analysis results based on OE, KE, and LE energy efficiency parameters in Embodiment 1 of the present invention; Figure 8 The energy allocation ratio and its statistical results under different time-state modes in Embodiment 1 of the present invention; Figure 9 This is a schematic diagram of the system structure in Embodiment 2 of the present invention. Detailed Implementation

[0019] This invention provides a gait analysis method and system based on IMU and kinematically constrained convolutional neural networks. It constructs a kinematically constrained convolutional neural network model that can adaptively estimate limb segment parameters, and establishes an integrated calculation framework for the mechanical energy of the body's center of mass oscillation and the mechanical energy of the lower limbs. This enables quantitative characterization of energy coupling characteristics under multiple velocities and achieves gait analysis based on highly sensitive energy indicators.

[0020] The overall concept of the technical solutions in the embodiments of the present invention is as follows: Human gait is a near-energy-optimal movement mechanism: muscles convert metabolic energy into mechanical energy to achieve stable and efficient movement with minimal cost. Individuals actively adjust their walking speed, frequency, and stride length to reduce metabolic costs during walking; simultaneously, the severity of gait impairment is closely related to the energy cost of walking. During gait, the oscillatory energy (OE) of the body's center of mass and the lower-limb energy (LE) can quantitatively reflect energy consumption and control efficiency, serving as important indicators for assessing motor function and rehabilitation progress.

[0021] This invention proposes a gait energy estimation method that relies solely on six-axis IMU acceleration and angular velocity signals. In the walking task, a local coordinate system is defined, and the positions of the centroids of each segment of the lower limb are estimated by combining human body proportion parameters and a kinematically constrained recurrent neural network (RNN). A gait analysis framework is established, including methods such as IMU calibration, coordinate system definition, coordinate transformation, oscillation velocity decomposition, and mechanical energy decomposition of each segment of the lower limb.

[0022] This invention also proposes a convolutional neural network architecture with embedded kinematic physical constraints. The consistency of instantaneous acceleration at the joint centers of adjacent segments is incorporated into the network as a biomechanical constraint, replacing the existing unconstrained data-driven model. This fundamentally eliminates the systematic errors caused by sensor placement and improves the accuracy of parameter estimation and the generalization of the model.

[0023] This invention overcomes the technical bottlenecks of low accuracy and poor generalization in models without physical constraints, eliminates systematic errors caused by sensor placement and calibration dependencies, constructs a kinematically constrained convolutional neural network model that can adaptively estimate limb segment parameters, and establishes an integrated calculation framework for the mechanical energy of the body's center of mass oscillation and the mechanical energy of the lower limbs. This enables quantitative characterization of energy coupling characteristics under multiple velocities, and relies on highly sensitive energy indicators to accurately identify complex gait patterns. Ultimately, this results in a lightweight gait energy estimation and analysis technology solution that is independent of laboratory requirements, highly accurate, and highly adaptable, meeting the practical application needs of clinical rehabilitation, motor function monitoring, and other scenarios. Before introducing specific embodiments, the hardware devices required for the methods of this application will be introduced first, such as... Figure 1 As shown, the wearable inertial measurement data acquisition module uses a multi-node six-axis inertial measurement sensor, which is fixed to the dorsum of the foot, calf, thigh, and sacrum of the torso using elastic bands. The sensor simultaneously acquires raw time-series signals of three-axis acceleration and three-axis angular velocity, and also records three types of anthropometric static parameters of the subject: height, weight, and gender. All acquired data is transmitted to the terminal in real time via a wireless communication link, completing the unified aggregation and caching of raw data. This module adopts industry-standard sampling configuration and fixing methods, requiring no additional customized design, ensuring the system's lightweight design and compatibility. Example

[0024] This embodiment provides a gait analysis method based on IMU and kinematically constrained convolutional neural networks, such as... Figure 2 As shown, it includes: S1. Neural Network Construction Process: Construct and train a convolutional neural network with embedded kinematic constraints. The convolutional neural network with embedded kinematic constraints is used to fuse dynamic time series data from the IMU sensor and static human body parameters, and output the distance from the IMU sensor to the joint axis.

[0025] To solve the problem of calculating the distance from the IMU to the joint axis using the traditional least squares method The existing problems include high computational complexity, weak noise resistance, and insufficient real-time performance. This embodiment constructs a convolutional neural network with embedded kinematic constraints. The solution is then performed. In one specific embodiment, the convolutional neural network architecture with embedded kinematic constraints is as follows: Figure 3As shown, it is a one-dimensional convolutional neural network architecture. The main body of the network consists of three convolutional layers and two fully connected perceptron layers. The convolutional layers are used to extract local motion features from inertial time-series data, and the perceptrons are used to fuse individual features from human static parameters. The neural network receives preprocessed kinematic time-series data and anthropometry parameters, and outputs accurate limb segment biomechanical parameters in an end-to-end manner. Its core improvement lies in embedding human kinematic physical constraints into the network training and inference process. Based on the principle of rigid body motion acceleration decoupling, the acceleration measured by the sensor is separated into the translational acceleration at the joint center and the additional acceleration of the sensor around the joint rotation. A customized loss function is constructed with the consistency of the instantaneous acceleration at the joint center of adjacent limb segments as a constraint condition, forcing the network to output parameter results that conform to the physical laws of human motion, thus avoiding the unconstrained fitting defects of pure data-driven models.

[0026] Based on the principle of kinematic consistency between adjacent limb segments, the joint center, as the connecting node, must maintain uniform instantaneous acceleration in adjacent local coordinate systems. The kinematic constraint equations are constructed as follows: ; in, It is the (n-1)th IMU at time... Total acceleration, Is the nth IMU at time... Total acceleration, This represents the acceleration generated by the IMU rotating around the joint center. It is the position vector of the IMU from the joint axis in the sensor coordinate system. Indicates at time angular velocity, Indicates at time The angular acceleration, where n represents the IMU serial number.

[0027] Convolutional neural networks with embedded kinematic constraints can effectively learn the nonlinear mapping relationship between the distance from the sensor to the joint axis and human parameters and IMU dynamic data, eliminating systematic errors caused by sensor installation position and adaptively solving for the distance from the sensor to the joint axis and the centroid position of the limb segment. Compared with the traditional least squares method, this method reduces computational complexity and improves noise resistance while ensuring the accuracy of the distance calculation from the sensor to the joint axis, providing high-precision parameter support for subsequent limb centroid position calculation and OE and LE energy modeling.

[0028] In practice, the network uses a sliding window to process time-series data in frames. The input feature dimensions include twelve-dimensional dynamic motion data and three-dimensional static human body parameters. The network is trained through five-fold cross-validation. The optimizer uses the AdamW algorithm combined with gradient clipping and learning rate annealing strategies to ensure training stability. During the inference phase, no laboratory calibration equipment is required. The network can adaptively output key parameters such as the distance from the sensor to the joint axis and the centroid offset of each segment. The output results are directly input into the energy calculation module to achieve fully automatic elimination of placement errors.

[0029] To cover individual differences and variations in motion constraints at IMU installation locations, such as Figure 4 As shown, multiple combinations of R1 and R2 were designed through experiments. R1 and R2 represent the distance from the IMU to the joint axis. Eight valid data points were collected for each subject for each configuration, resulting in a dataset with a total sample size of 1080. The original IMU data consisted of time-series sequences of three-dimensional acceleration and three-dimensional angular velocity (dimensions: 1080×30×12, where 30 is the time series length and 12 is the feature dimension). Simultaneously, subject height, weight, and gender data (dimensions: 1080×3) and real n labels (dimensions: 1080×2) were collected, providing sufficient multimodal input and gold standard annotations for model training.

[0030] Figure 5 The training convergence curves of the optimal fold (Fold2) during five-fold cross-validation training of the network are shown, including mean squared error (MSE), root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R²). During training, the training loss and validation loss gradually decrease with increasing epochs and tend to stabilize, without significant overfitting or underfitting, indicating that the model has good ability to fuse temporal motion data and static human body parameters. The overall RMSE and MAE, as well as the R1 and R2 components, continuously decrease, with the R1 error index decreasing at a faster rate and eventually stabilizing at a low level. The R² curve gradually approaches 1, reflecting the model's ability to adapt to different data types. n The degree of fit between the predicted and actual values ​​is constantly improving, especially the prediction fit for R1 is better.

[0031] Experiments have verified that the method in this embodiment can effectively learn r. n The nonlinear mapping relationship with human body parameters and IMU dynamic data provides high-precision parameter support for subsequent calculation of lower limb segment centroid positions and OE and LE energy modeling.

[0032] S2. Data Acquisition and Preprocessing: Raw time-series data of triaxial acceleration and triaxial angular velocity are synchronously acquired through IMU sensors. The IMU sensors are respectively installed on the feet, calves, thighs and sacrum of the trunk to acquire static human body parameters, including height, weight and gender; the raw time-series data are preprocessed to obtain dynamic time-series data.

[0033] In one specific embodiment, the preprocessing steps are as follows: First, the acceleration measured by the sensor is transformed into coordinates to subtract the gravitational acceleration component; then, the acceleration and angular velocity signals are filtered to reduce sensor noise; for the velocity portion obtained by integrating acceleration, a zero-velocity correction method is used to reduce integral drift; then, the data from each sensor are aligned according to the time synchronization relationship, and the angular acceleration is calculated using the fourth-order Runge-Kutta method. The coordinate system linkage transformation and accurate angular acceleration calculation are customized improvements to adapt to subsequent energy calculations, ensuring a high degree of matching between kinematic parameters and the human rigid body mechanical model.

[0034] Before acquiring the dynamic time-series data of the subject's movements, a dual-posture static calibration process is performed. First, the subject stands naturally for several seconds to determine the vertical direction using the direction of gravity in a static state. Then, the subject raises one lower limb and holds it for several seconds to establish the local coordinate orientation of each lower limb segment based on the changes in limb posture. This yields the transformation relationship from the sensor coordinate system to the limb coordinate system and from the limb coordinate system to the world coordinate system. By identifying the static gravity vector through the natural standing posture and establishing the local coordinate system of each limb segment through the hip flexion, knee flexion, and leg raising posture, the initial posture calibration of the sensor is completed.

[0035] S3. Limb centroid adaptive prediction process: Dynamic time series data and static human body parameters are input into a convolutional neural network with kinematic constraints to obtain the predicted distance from the sensor to the joint axis; the lower limb centroid is located by the predicted distance from the sensor to the joint axis and the static human body parameters, and then the distance from the IMU sensor to the limb centroid is calculated to obtain the coordinate transformation matrix from the IMU sensor coordinate system to the body limb centroid coordinate system.

[0036] Distance from IMU sensor to limb center of mass The calculation formula is: ; in, It is the anatomical distance from the center of mass of the limb segment to the joint proximal to the heart (calculated from the subject's height and standardized human proportion factor). It is the position vector of the IMU to the joint axis in the sensor coordinate system (obtained by collecting acceleration, angular velocity data and kinematic constraints).

[0037] Coordinate transformation matrix from IMU sensor coordinate system to body limb centroid coordinate system The core parameters are The acceleration and angular velocity motion data, after removing gravitational acceleration, are transformed into the body's center of mass coordinate system through a pre-calculated coordinate transformation matrix after four Butterworth filters.

[0038] S4. Gait Energy Estimation Process: Calculate the oscillation energy in the forward, backward, left-right, and vertical directions based on the dynamic time series data of the IMU sensor in the sacral region of the trunk. Then sum the three to obtain the whole-body center of mass oscillation energy OE. Calculate the forward kinetic energy KE of the human body during the gait cycle based on the human body mass and average forward velocity. Treat each lower limb segment as a rigid body, calculate the translational energy caused by external forces, the rotational energy caused by external torques, and the change in gravitational potential energy caused by vertical height changes for each lower limb segment. Summate these to obtain the lower limb mechanical energy LE.

[0039] In one specific embodiment, the detailed solution formula is as follows: 1. Whole-body center of mass oscillation energy OE The three-dimensional motion velocity of the human body's center of mass is decoupled and decomposed, separating the forward and backward motion velocity into a constant forward component and a zero-mean oscillation component. The pure oscillation components of the inward, outward, and vertical motions are directly extracted. Through numerical integration of a single gait cycle, the three types of oscillation energy components in the forward, inward, and vertical directions are solved respectively. The total oscillation mechanical energy of the whole body is obtained by superimposing and summing, which is used to quantify the energy consumption of human gait balance regulation.

[0040] The formula for calculating the whole-body center-of-mass oscillation energy OE is: ; in, It is the oscillation velocity component in the forward and backward directions. It is the oscillation velocity component in the left and right directions. It represents the vertical oscillation velocity component, where m is the subject's weight, T is the gait cycle time, and t is time.

[0041] 2. Forward kinetic energy KE of the human body during the gait cycle Based on the human body mass and average speed in the forward direction, the forward kinetic energy KE of the human body during the gait cycle is calculated. This parameter reflects the main effective energy output of the human body during forward movement and can serve as a benchmark for comparing propulsion efficiency at different speeds and in different gait modes.

[0042] The formula for calculating the forward kinetic energy KE of the human body during the gait cycle is: ; Where m is the subject's weight and T is the gait cycle time. Let t be the average speed of the human body in the direction of forward movement, and t be the time.

[0043] 3. Lower limb mechanical energy (LE) Each lower limb segment is treated as a rigid body. For each segment, three types of energy contributions are calculated: The first type is translational energy caused by external forces, which reflects the translational work done by the center of mass of the limb segment as the body moves. The second type is rotational energy caused by external torque, which reflects the rotational work done when a limb rotates around its own center of mass or a joint. The third category is the change in gravitational potential energy caused by changes in vertical height, reflecting the work done against gravity when the lower limbs are raised, lowered, or supporting the body.

[0044] The mechanical energy of the lower limb (LE) is obtained by calculating and summing the three types of energy separately for the foot, lower leg, and thigh.

[0045] The formula for calculating the mechanical energy (LE) of the lower limbs is: ; ; Where n represents the IMU serial number, An external force acting on a limb. The velocity of the limb's center of mass is m, and the subject's weight is m. It is the acceleration due to gravity. The change in height of the limb's center of mass along the direction of gravity (e.g.) Figure 6 (As shown), t is time. Angular acceleration, This is the torque acting on the limb about its center of mass.

[0046] Preferably, it also includes: S5, gait energy coupling analysis process: the total mechanical energy TME is obtained based on the whole body center of mass oscillation energy OE, forward kinetic energy KE and lower limb mechanical energy LE, and the energy proportion characteristics are analyzed. Complex gaits are distinguished by the energy proportion characteristics, including normal walking, crossing obstacles, walking on gravel, going upstairs and going downstairs.

[0047] Different time-state scenarios have different energy distribution characteristics: Normal walking on flat ground is usually characterized by a high proportion of forward kinetic energy (KE) and a low proportion of whole-body center of mass oscillation energy (OE), indicating that the human body has high propulsion efficiency and relatively stable body swing control. When crossing obstacles, the proportion of whole-body center of mass oscillation energy (OE) increases significantly because the human body needs to increase leg lifting, obstacle avoidance, and body posture adjustment movements. When walking on gravel or uneven ground, the proportion of forward kinetic energy (KE) decreases while the proportion of lower limb mechanical energy (LE) increases, indicating that the body uses more energy to maintain joint stability and balance control rather than to propel itself forward. When going upstairs, the proportion of lower limb mechanical energy (LE) increases, mainly because the human body needs to do work to overcome gravity. When going downstairs, both the mechanical energy (LE) and oscillatory energy of the lower limbs may increase, mainly because the muscles of the lower limbs need to cushion and control the descent.

[0048] The energy proportion features of the gait to be identified are compared with the normal gait reference pattern, and the gait pattern recognition results are output, including categories such as normal walking, crossing obstacles, walking on gravel, going upstairs, and going downstairs; at the same time, the corresponding energy interpretation results are output, such as "decreased propulsion efficiency", "increased balance control burden", and "increased antigravity work of the lower limbs".

[0049] This study involved 15 participants who completed walking tests at various speeds on different terrains. The OE (Output Energy Efficiency) and LE (Energy Flow Rate) were calculated and analyzed under different speed conditions, along with their relationship. Simultaneously, several joint energy efficiency parameters based on OE and LE were constructed to evaluate the specificity of each parameter in different walking environments, such as uphill / downhill and obstacle-crossing scenarios, thus verifying the reliability of the raw IMU data and the energy index estimation results. Figure 7 and Figure 8 The analysis results based on OE, KE, and LE energy efficiency parameters demonstrate the energy distribution ratio and its statistical comparison under different gait modes. Tests show that the combined energy efficiency parameters exhibit good sensitivity and stability in identifying complex walking environments such as inclines, declines, and obstacle crossings, and can improve the clinical effectiveness of gait assessment.

[0050] Based on the same inventive concept, this application also provides a system corresponding to the method in Embodiment 1, as detailed in Embodiment 2.

[0051] Example 2 This embodiment provides a gait analysis system based on IMU and kinematically constrained convolutional neural networks, such as Figure 9 As shown, it includes: A neural network construction module is used to construct and train a convolutional neural network with embedded kinematic constraints. The convolutional neural network with embedded kinematic constraints is used to fuse dynamic time-series data from IMU sensors and static human body parameters to output the distance from the IMU sensor to the joint axis. The data acquisition and preprocessing module is used to synchronously acquire raw time-series data of triaxial acceleration and triaxial angular velocity through IMU sensors. The IMU sensors are respectively installed on the feet, calves, thighs and sacrum of the trunk to acquire static human body parameters, including height, weight and gender; and preprocess the raw time-series data to obtain dynamic time-series data. The limb centroid adaptive prediction module is used to input dynamic time series data and static human body parameters into a convolutional neural network with kinematic constraints to obtain the predicted distance from the sensor to the joint axis; the lower limb centroid is located by using the predicted distance from the sensor to the joint axis and the static human body parameters, and then the distance from the IMU sensor to the limb centroid is calculated to obtain the coordinate transformation matrix from the IMU sensor coordinate system to the body limb centroid coordinate system. The gait energy estimation module calculates the oscillatory energy in the forward, left-right, and vertical directions based on the dynamic time-series data from the IMU sensor in the sacral region of the trunk. The sum of these three values ​​yields the whole-body center-of-mass oscillation energy OE. Based on the body mass and average forward velocity, the module calculates the forward kinetic energy KE of the body during the gait cycle. Treating each lower limb segment as a rigid body, the module calculates the translational energy caused by external forces, the rotational energy caused by external torques, and the change in gravitational potential energy caused by vertical height changes for each lower limb segment. The sum of these values ​​yields the lower limb mechanical energy LE.

[0052] Preferably, it also includes: a gait energy coupling analysis module, used to obtain the total mechanical energy (TME) based on the whole-body center of mass oscillation energy (OE), forward kinetic energy (KE), and lower limb mechanical energy (LE), and analyze the energy proportion characteristics, and distinguish complex gaits, including normal walking, crossing obstacles, walking on gravel, going upstairs, and going downstairs, based on the energy proportion characteristics.

[0053] Since the system described in Embodiment 2 of this invention is a system used to implement the method of Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this system based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All systems used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.

[0054] The technical solutions provided in the embodiments of the present invention have at least the following technical effects: 1. It has high estimation accuracy and small systematic error, effectively overcoming the shortcomings of existing unconstrained network fitting, which is easily affected by noise and placement deviation and has weak model generalization ability. This advantage is achieved by embedding the kinematic physical constraint of the consistency of instantaneous acceleration at the joint center of adjacent limb segments into the convolutional neural network. The network parameter output is constrained by biomechanical laws, abandoning the limitations of pure data fitting. It eliminates the calculation error caused by non-standard sensor layout from the algorithm level, ensuring the stability and accuracy of parameter estimation results.

[0055] 2. Convolutional neural networks with embedded kinematic constraints can effectively learn the nonlinear mapping relationship between the distance from the sensor to the joint axis and human parameters and IMU dynamic data. Compared with the traditional least squares method, it reduces computational complexity and improves noise resistance while ensuring the accuracy of the distance calculation from the sensor to the joint axis, providing high-precision parameter support for subsequent limb centroid position calculation and OE and LE energy modeling.

[0056] 3. Gait assessment is highly sensitive and comprehensive in its characterization dimensions, solving the problem that existing technologies can only calculate a single dynamic parameter and cannot quantify gait energy regulation characteristics. This advantage is achieved by relying on an integrated unified solution framework for whole-body oscillatory mechanical energy and lower limb mechanical energy, simultaneously completing the quantitative calculation and standardization of the two types of core energy indicators. Combined with the analysis of multi-velocity energy coupling characteristics, a highly sensitive energy system with both whole-body balance and lower limb function assessment capabilities is constructed, significantly improving the technical effectiveness of gait function assessment and enabling early identification of sports injuries.

[0057] 4. A gait discrimination method based on energy coupling features is proposed to achieve accurate identification of complex gaits by using energy proportion features, thereby improving the clinical effectiveness of gait assessment.

[0058] 5. It has the technical advantages of lightweight system and wide adaptability to various scenarios. It breaks through the limitations of existing gold standard technologies that rely on large laboratory equipment and cannot perform long-term outdoor monitoring. This advantage is realized based on the closed-loop architecture of pure wearable inertial measurement. The overall algorithm has low computational load and low hardware cost. It can be stably applied to various non-laboratory scenarios such as clinical rehabilitation and home monitoring. While ensuring the accuracy of assessment, it realizes the portability and large-scale implementation of gait energy analysis technology.

[0059] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0060] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0063] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A gait analysis method based on IMU and kinematically constrained convolutional neural networks, characterized in that, include: Neural network construction process: Convolutional neural network with embedded kinematic constraints is constructed and trained. The convolutional neural network with embedded kinematic constraints is used to fuse dynamic time series data of IMU sensor and static human body parameters, and output the distance from IMU sensor to joint axis. Data acquisition and preprocessing process: Raw time-series data of triaxial acceleration and triaxial angular velocity are synchronously acquired through IMU sensors. The IMU sensors are respectively installed on the feet, calves, thighs and the sacrum of the torso to acquire static human body parameters, including height, weight and gender; the raw time-series data are preprocessed to obtain dynamic time-series data. The limb centroid adaptive prediction process involves: inputting dynamic time-series data and static human body parameters into a convolutional neural network with embedded kinematic constraints to obtain the predicted distance from the sensor to the joint axis; locating the lower limb centroid using the predicted distance from the sensor to the joint axis and the static human body parameters; then calculating the distance from the IMU sensor to the limb centroid, thereby obtaining the coordinate transformation matrix from the IMU sensor coordinate system to the body limb centroid coordinate system. Gait energy estimation process: The oscillating energy in the forward, backward, left-right, and vertical directions is calculated based on the dynamic time-series data from the IMU sensor at the sacral region of the trunk. These three values ​​are then summed to obtain the whole-body center-of-mass oscillation energy OE. The forward kinetic energy KE of the human body during the gait cycle is calculated based on the body mass and average forward velocity. Each lower limb segment is treated as a rigid body, and for each segment, the translational energy caused by external forces, the rotational energy caused by external torques, and the change in gravitational potential energy caused by vertical height changes are calculated separately. These are then summed to obtain the lower limb mechanical energy LE.

2. The method according to claim 1, characterized in that, It also includes a gait energy coupling analysis process: the total mechanical energy (TME) is obtained based on the whole body center of mass oscillation energy (OE), forward kinetic energy (KE), and lower limb mechanical energy (LE), and the energy proportion characteristics are analyzed. Complex gaits are distinguished by the energy proportion characteristics, including normal walking, crossing obstacles, walking on gravel, going upstairs, and going downstairs.

3. The method according to claim 1, characterized in that: The preprocessing of the original time series data specifically includes: first, performing coordinate transformation on the acceleration measured by the sensor and deducting the gravitational acceleration component; then filtering the acceleration and angular velocity signals to reduce sensor noise; for the part of velocity obtained by integrating acceleration, using a zero-velocity correction method to reduce integration drift; and then aligning the data from each sensor according to the time synchronization relationship and calculating the angular acceleration.

4. The method according to claim 1, characterized in that: The kinematic constraint equations of the convolutional neural network embedded with kinematic constraints are as follows: ; in, It is the (n-1)th IMU at time... Total acceleration, Is the nth IMU at time... Total acceleration, This represents the acceleration generated by the IMU rotating around the joint center. It is the position vector of the IMU from the joint axis in the sensor coordinate system. Indicates at time angular velocity, Indicates at time The angular acceleration, where n represents the IMU serial number.

5. The method according to claim 1, characterized in that, include: Distance from IMU sensor to limb center of mass The calculation formula is: ; in, It is the anatomical distance from the center of mass of the limb segment to the joint proximal to the heart. It is the position vector of the IMU to the joint axis in the sensor coordinate system.

6. The method according to claim 1, characterized in that, The formula for calculating the whole-body center-of-mass oscillation energy OE is: ; in, It is the oscillation velocity component in the forward and backward directions. It is the oscillation velocity component in the left and right directions. It represents the vertical oscillation velocity component, where m is the subject's weight, T is the gait cycle time, and t is time.

7. The method according to claim 1, characterized in that: The formula for calculating the forward kinetic energy KE of the human body during the gait cycle is: ; Where m is the subject's weight and T is the gait cycle time. Let t be the average speed of the human body in the direction of forward movement, and t be the time.

8. The method according to claim 1, characterized in that: The formula for calculating the lower limb mechanical energy LE is: ; Where n represents the IMU serial number, An external force acting on a limb. The velocity of the limb's center of mass is m, and the subject's weight is m. It is the acceleration due to gravity. The change in height of the limb's center of mass along the direction of gravity is represented by t, where t is time. Angular acceleration, This is the torque acting on the limb about its center of mass.

9. The method according to claim 1, characterized in that: Before the data acquisition and preprocessing process, there is also a dual-posture static calibration process: first, the subject is asked to stand naturally for several seconds, and the vertical direction is determined by the direction of gravity in a static state; then, the subject is asked to raise one lower limb and hold it for several seconds, and the local coordinate direction of each lower limb segment is established according to the changes in limb posture, thereby obtaining the transformation relationship from the sensor coordinate system to the limb coordinate system and from the limb coordinate system to the world coordinate system.

10. A gait analysis system based on IMU and kinematically constrained convolutional neural networks, characterized in that, include: A neural network construction module is used to construct and train a convolutional neural network with embedded kinematic constraints. The convolutional neural network with embedded kinematic constraints is used to fuse dynamic time-series data from IMU sensors and static human body parameters to output the distance from the IMU sensor to the joint axis. The data acquisition and preprocessing module is used to synchronously acquire raw time-series data of triaxial acceleration and triaxial angular velocity through IMU sensors. The IMU sensors are respectively set at the feet, calves, thighs, and sacrum or trunk to acquire static human body parameters, including height, weight and gender; and preprocess the raw time-series data to obtain dynamic time-series data. The limb centroid adaptive prediction module is used to input dynamic time series data and static human body parameters into a convolutional neural network with kinematic constraints to obtain the predicted distance from the sensor to the joint axis; the lower limb centroid is located by using the predicted distance from the sensor to the joint axis and the static human body parameters, and then the distance from the IMU sensor to the limb centroid is calculated to obtain the coordinate transformation matrix from the IMU sensor coordinate system to the body limb centroid coordinate system. The gait energy estimation module is used to calculate the oscillation energy in the anterior-posterior, lateral, and vertical directions based on the dynamic time series data of the IMU sensor in the sacrum or trunk, and then sum the three to obtain the whole-body center of mass oscillation energy OE. Based on the human body mass and average speed in the forward direction, calculate the forward kinetic energy KE of the human body during the gait cycle; treat each lower limb segment as a rigid body, and calculate the translational energy caused by external force, the rotational energy caused by external torque, and the change in gravitational potential energy caused by the change in vertical height for each lower limb segment. Summing them up gives the mechanical energy LE of the lower limb.

Images