Fitness action recognition and evaluation method based on sparse inertial sensor

Abstract

The invention discloses a body-building action recognition and evaluation method based on a sparse inertial sensor, and belongs to the technical field of artificial intelligence and exercise health. The body-building action recognition and evaluation method comprises the following steps: acquiring 6-node motion information of a human body through the inertial sensor, acquiring 24-node motion information of the human body through a human body motion reconstruction network, and inputting the information into a trained body-building action classification model; and finally, according to a body-building action recognition result, performing quantitative evaluation on the body-building actions by utilizing dynamic time warping (DTW), and quickly comparing the real-time body-building actions with standard actions. According to the body-building action recognition and evaluation method based on the sparse inertial sensor, accurate recognition and quantitative evaluation of the body-building action can be effectively achieved, and meanwhile the body-building action recognition and evaluation method is low in price and higher in practical applicability.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and sports and health technology, and in particular to a method for fitness movement recognition and evaluation based on sparse inertial sensors. Background Technology

[0002] With rapid economic and social development and the continuous improvement of residents' living standards, health awareness has become increasingly ingrained in people's minds, and the public's attention to physical exercise and scientific fitness has significantly increased. Against this backdrop, how to accurately identify, standardize, and evaluate the effectiveness of fitness movements has become a crucial technical requirement in the fields of fitness guidance, sports rehabilitation, and personal health management. Currently, most mainstream motion recognition and analysis systems rely on optical sensors, using computer vision technology to capture and analyze human postures. While this method can achieve high recognition accuracy, its deployment is usually limited by specific environments, the equipment cost is high, and it is easily affected by external factors such as light and occlusion, making it difficult to widely apply in daily life.

[0003] To overcome the aforementioned limitations, motion recognition solutions based on inertial sensors have gradually gained attention. Inertial sensors are characterized by their small size, low power consumption, and low cost, and can be easily integrated into wearable devices to achieve real-time and continuous acquisition of motion data. However, existing inertial sensor-based solutions still have shortcomings: some solutions require dense deployment of sensors at multiple nodes on the body, resulting in cumbersome wearing and low user acceptance; some solutions lack efficient motion data completion mechanisms, making it difficult to reconstruct the whole-body motion state from sparse node data; at the same time, most evaluation methods can only achieve qualitative judgments and cannot accurately and quantitatively assess the standardization of movements, failing to meet users' needs for scientific fitness. Summary of the Invention

[0004] The purpose of this invention is to provide a fitness movement recognition and evaluation method based on sparse inertial sensors, which retains the advantages of low cost and strong environmental adaptability of inertial sensors, while solving the problems of cumbersome wearing, low data completion accuracy, and inaccurate evaluation in existing solutions, thus providing technical support for popular scientific fitness.

[0005] To achieve the above objectives, this invention provides a method for fitness motion recognition and evaluation based on sparse inertial sensors, comprising the following steps: S1. Based on multiple inertial sensors sparsely arranged at specific nodes of the human body, acquire the raw motion measurement data of the nodes; S2. Input the original motion measurement data into a pre-trained human motion reconstruction network to reconstruct motion information of multiple joint nodes throughout the human body; S3. Input the motion information of the reconstructed human body joint nodes into the pre-trained fitness movement classification model to identify the classification result of the current fitness movement. S4. Based on the classification results, match the corresponding standard template from the pre-stored standard action template library, and use the dynamic time warping algorithm to compare the motion information of the current action with the standard template, and output a quantitative evaluation result.

[0006] Preferably, in step S1, there are a total of 6 inertial sensors, which are respectively arranged at the head, abdomen, left wrist, right wrist, left calf and right calf nodes of the human body; the raw motion measurement data includes at least angular velocity information and acceleration information measured by the gyroscope and accelerometer in the inertial sensors.

[0007] Preferably, step S1 further includes: filtering the angular velocity information and solving the attitude angle information at each sensor node using quaternion differential equations; and combining the acceleration information and the attitude angle information to form the motion information of each node.

[0008] Preferably, the human motion reconstruction network in step S2 is a neural network based on a bidirectional gated recurrent unit (GRU). The motion reconstruction steps specifically include: The original motion measurement data is calibrated and normalized before being converted to a unified SMPL human body parameter model coordinate system. Using inertial measurement data and the fixed position of the main blade joint in the SMPL human body parameter model, the relative positions of the other 5 blade joints relative to the main blade joint are inferred. By combining the estimated position data of the six blade joints with inertial measurement information, the relative positions of all non-blade joints with respect to the main blade joint are deduced. Using a data-trained inverse kinematics solver and combining the attitude information of the six leaf joints, the attitude information of all joints in the human body is solved, and finally the attitude angles and motion acceleration information of 24 joint nodes in the whole body are reconstructed.

[0009] Preferably, the calibration process is implemented based on the following mathematical relationship: ; in, This represents the attitude rotation matrix between the coordinate systems of each inertial sensor and the corresponding skeletal coordinate system at each joint. Indicates the first The pose rotation matrix represents the rotational relationship between the skeletal coordinate system and the SMPL human body parameter model coordinate system of each joint. Indicates the first The attitude rotation matrix of each inertial sensor itself. This represents the transition matrix from the global inertial coordinate system to the SMPL coordinate system.

[0010] Preferably, the normalization preprocessing includes normalization operations performed on the main blade joint measurements and the blade joint measurements, respectively: The normalized calculation from the blade joint measurements is as follows: ; in, , This represents the normalized value measured from the blade joint. , These represent the attitude rotation matrices between the articular skeleton coordinate system and the SMPL coordinate system of the primary and secondary blades, respectively. , These represent the skeletal motion accelerations of the primary blade and the secondary blade in the SMPL coordinate system, respectively. The normalized calculation of the main blade joint measurements is as follows: ; in, , This represents the normalized joint measurement value of the main blade.

[0011] Preferably, the fitness movement classification model in step S3 is an LSTM-RF hybrid model composed of a Long Short-Term Memory (LSTM) network and a Random Forest (RF) network. The recognizable classification results include three types: squat, bench press, and lat pulldown. The specific training process includes: S31. Construct the original feature dataset from the motion information of the reconstructed joint nodes of the whole human body. S32. Input the original feature dataset into the LSTM model to extract the long-term dependencies and dynamic change patterns in the data, and obtain the output. ; S33, the output The new feature dataset is constructed by merging it with the original feature dataset. S34. Input the new feature dataset into the RF model for final classification to obtain the classification result of the fitness movement.

[0012] Preferably, step S4 specifically includes: S41. Based on the classification results, select the predetermined key joints; S42. Retrieve the standard action template corresponding to the classification result from the standard template library. The standard action is a standard sequence. ,in, Indicates the standard sequence length; S43. Extract the timing data of the key joints in the current action during the movement process to form a test sequence. ,in, Indicates the length of the test sequence; S44. Calculate the dynamic time-warped distance between the test sequence and the standard sequence, and obtain the quantitative evaluation result based on the distance, specifically including: Build a A matrix, each element of the matrix Point Time The distance is calculated using the following formula: The regular path To obtain it, three conditions must be met: Boundary condition: The starting point of the path must be... The destination is ; Continuity condition: if Then the next dynamic regularization point of the path Must meet and That is, the two points must be adjacent and continuous, and there cannot be any gaps between them; Monotonicity condition: if Then the next dynamic regularization point on the path It must meet and ; Among the points that meet these three conditions, select the point with the smallest distance as the point of the regular path, and connect the selected points to obtain the entire regular path.

[0013] Preferably, the quantitative evaluation result in step S44 is expressed in the form of a score, which is calculated based on the cumulative warping distance obtained by the dynamic time warping algorithm. And obtained by mapping using the following formula: ; in, This indicates the final score. This indicates a perfect score. Indicates the base value. Indicates the impact factor.

[0014] Therefore, this invention employs a fitness movement recognition and evaluation method based on sparse inertial sensors, achieving accurate recognition and standardized measurement of fitness movements through a 6-node sparse inertial sensor + motion reconstruction network + LSTM-RF classification + DTW quantitative evaluation. This avoids the limitations of optical solutions, such as environmental constraints and high costs, and solves the problems of cumbersome deployment and poor data completion in existing inertial sensors. It combines low cost, strong environmental adaptability, and high evaluation accuracy, providing efficient and convenient technical support for popular scientific fitness.

[0015] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the implementation of an embodiment of the present invention; Figure 2 This is a flowchart of the LSTM-RF hybrid model prediction process according to an embodiment of the present invention; Figure 3 This is a selection diagram of the regularized matching path according to an embodiment of the present invention; Figure 4 This is a preset T-pose human posture diagram according to an embodiment of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0018] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0019] Example 1 This embodiment uses squats as an example, such as Figure 1 As shown, a method for fitness motion recognition and evaluation based on sparse inertial sensors includes the following steps: S1. Based on six inertial sensors sparsely arranged at specific nodes on the human body, each sensor first enters a self-calibration state. After self-calibration is completed, one of the inertial sensors (IMU) is fixed to the left wrist of the human body with a flexible strap, and the left arm is raised horizontally and held for 3 seconds to align the IMU coordinate system. The axis is aligned with the left side of the human body. The axis is aligned with the top of the human body. Align the axis with the frontal direction of the human body; maintain the posture, fix the remaining 5 sensors to the preset human body nodes, and then hold the preset T-pose posture with known posture data for 3 seconds, as shown in the attached figure. Figure 4As shown, the measurement data of each IMU is read, and its three-axis average acceleration and attitude rotation matrix are calculated respectively.

[0020] S2. Input the original motion measurement data into the pre-trained human motion reconstruction network to reconstruct the motion information of multiple joint nodes throughout the human body.

[0021] The human motion reconstruction network is a neural network based on a bidirectional gated recurrent unit (GRU). The specific steps of motor reconstruction include: The raw motion measurement data is calibrated and normalized before being converted to a unified SMPL human body parameter model coordinate system; Using inertial measurement data and the fixed position of the main blade joint in the SMPL human body parameter model, the relative positions of the other 5 blade joints relative to the main blade joint are inferred. By combining the estimated position data of the six blade joints with inertial measurement information, the relative positions of all non-blade joints with respect to the main blade joint are deduced. Using a data-trained inverse kinematics solver and combining the attitude information of the six leaf joints, the attitude information of all joints in the human body is solved, and finally the attitude angles and motion acceleration information of 24 joint nodes in the whole body are reconstructed.

[0022] The calibration process is based on the following mathematical relationship: ; in, This represents the attitude rotation matrix between the coordinate systems of each inertial sensor and the corresponding skeletal coordinate system at each joint. Indicates the first The pose rotation matrix represents the rotational relationship between the skeletal coordinate system and the SMPL human body parameter model coordinate system of each joint. Indicates the first The attitude rotation matrix of each inertial sensor itself. This represents the transition matrix from the global inertial coordinate system to the SMPL coordinate system. After starting motion capture, according to the above formula and This allows us to inversely calculate the value of each frame under any pose. And abbreviated as ; For acceleration, the triaxial acceleration measured by the inertial sensor is calculated according to the following formula. The acceleration of the corresponding joint bones in the SMPL coordinate system is converted. And abbreviated as : ; For the results obtained after calibration and The system performs normalization operations on the blade measurement values ​​and main blade measurement values ​​for each frame, with the belly node as the reference reference during the processing. Normalization from the blade joint: For the remaining 5 IMU nodes, calculate their relative acceleration and relative rotation relative to the main blade joint: ; in, , This represents the normalized value measured from the blade joint. , These represent the attitude rotation matrices between the articular skeleton coordinate system and the SMPL coordinate system of the primary and secondary blades, respectively. , These represent the skeletal motion accelerations of the primary blade and the secondary blade in the SMPL coordinate system, respectively. The normalized calculation of the measurements of the main blade joint (ventral node) is as follows: ; in, , This represents the normalized measurements of the main blade joints. Finally, all the normalized data are combined and scaled to form a standardized input vector suitable for subsequent human motion reconstruction networks.

[0023] S3. Input the motion information of the reconstructed human body joint nodes into the pre-trained fitness movement classification model to identify the classification result of the current fitness movement.

[0024] The fitness exercise classification model is an LSTM-RF hybrid model composed of a Long Short-Term Memory (LSTM) network and a Random Forest (RF) network. It can classify exercises into three types: squat, bench press, and lat pulldown. The specific training process is as follows: Figure 2 As shown, it includes: S31. Construct the original feature dataset from the motion information of the reconstructed joint nodes of the whole human body. S32. Input the original feature dataset into the LSTM model to extract the long-term dependencies and dynamic change patterns in the data, and obtain the output. ; S33, the output The new feature dataset is constructed by merging it with the original feature dataset. S34. Input the new feature dataset into the RF model for final classification to obtain the classification result of the fitness movement.

[0025] S4. Based on the classification results, match the corresponding standard template from the pre-stored standard action template library, and use the dynamic time warping algorithm to compare the motion information of the current action with the standard template, and output the quantitative evaluation result.

[0026] like Figure 3 As shown, step S4 specifically includes: S41. Based on the classification results, select the predetermined key joints; S42, Standard Template The data is obtained by performing a squat, selecting the hip and knee joints. A professional wearing equipment performs a set of squat movements, and the calculated hip and knee joint data for each frame is saved and plotted as a curve to create a standard template. This standard movement is a standard sequence. ,in, Indicates the standard sequence length; S43, Test Template The data was acquired by performing a squat. After the tester correctly wore the sensors and performed the same movement, the hip and knee joint data for each frame were calculated and saved, and plotted as curves to form the test template. ,in, Indicates the length of the test sequence; S44. Calculate the dynamic time-warped distance between the test sequence and the standard sequence, and obtain the quantitative evaluation result based on the distance, specifically including: Build a A matrix, each element of the matrix Point Time The distance is calculated using the following formula: The regular path To obtain it, three conditions must be met: Boundary condition: The starting point of the path must be... The destination is ; Continuity condition: if Then the next dynamic regularization point of the path Must meet and That is, the two points must be adjacent and continuous, and there cannot be any gaps between them; Monotonicity condition: if Then the next dynamic regularization point on the path It must meet and ; Among the points that meet these three conditions, select the point with the smallest distance as the point of the regular path, and connect the selected points to obtain the entire regular path.

[0027] The quantitative evaluation results are expressed in the form of scores, which are calculated based on the cumulative warped distance obtained by the dynamic time warping algorithm. And obtained by mapping using the following formula: ; in, This indicates the final score. This indicates a perfect score. Indicates the base value. The influencing factor is indicated, and the quality assessment result is determined based on the final score.

[0028] Therefore, this invention employs a fitness movement recognition and evaluation method based on sparse inertial sensors, achieving accurate recognition and standardized measurement of fitness movements through a 6-node sparse inertial sensor + motion reconstruction network + LSTM-RF classification + DTW quantitative evaluation. This avoids the limitations of optical solutions, such as environmental constraints and high costs, and solves the problems of cumbersome deployment and poor data completion in existing inertial sensors. It combines low cost, strong environmental adaptability, and high evaluation accuracy, providing efficient and convenient technical support for popular scientific fitness.

[0029] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for fitness motion recognition and evaluation based on sparse inertial sensors, characterized in that, Includes the following steps: S1. Based on multiple inertial sensors sparsely arranged at specific nodes of the human body, acquire the raw motion measurement data of the nodes; S2. Input the original motion measurement data into a pre-trained human motion reconstruction network to reconstruct motion information of multiple joint nodes throughout the human body; S3. Input the motion information of the reconstructed human body joint nodes into the pre-trained fitness movement classification model to identify the classification result of the current fitness movement. S4. Based on the classification results, match the corresponding standard template from the pre-stored standard action template library, and use the dynamic time warping algorithm to compare the motion information of the current action with the standard template, and output a quantitative evaluation result.

2. The fitness movement recognition and evaluation method based on sparse inertial sensors according to claim 1, characterized in that, In step S1, there are a total of 6 inertial sensors, which are respectively arranged at the head, abdomen, left wrist, right wrist, left calf and right calf nodes of the human body; the raw motion measurement data includes at least the angular velocity information and acceleration information measured by the gyroscope and accelerometer in the inertial sensors.

3. The fitness movement recognition and evaluation method based on sparse inertial sensors according to claim 2, characterized in that, Step S1 further includes: filtering the angular velocity information and solving the attitude angle information at each sensor node by quaternion differential equations; and combining the acceleration information and the attitude angle information to form the motion information of each node.

4. The fitness movement recognition and evaluation method based on sparse inertial sensors according to claim 1, characterized in that, The human motion reconstruction network in step S2 is a neural network based on a bidirectional gated recurrent unit (GRU). The motion reconstruction steps specifically include: The raw motion measurement data is calibrated and normalized before being converted to a unified SMPL human body parameter model coordinate system. The preprocessed data is input into the human motion reconstruction network to predict the posture rotation matrix and motion acceleration of multiple joint nodes of the human body in the coordinate system of the human body parameter model.

5. The fitness movement recognition and evaluation method based on sparse inertial sensors according to claim 4, characterized in that, The calibration process is based on the following mathematical relationship: ； in, This represents the attitude rotation matrix between the coordinate systems of each inertial sensor and the corresponding skeletal coordinate system at each joint. Indicates the first The pose rotation matrix represents the rotational relationship between the skeletal coordinate system and the SMPL human body parameter model coordinate system of each joint. Indicates the first The attitude rotation matrix of each inertial sensor itself. This represents the transition matrix from the global inertial coordinate system to the SMPL coordinate system.

6. The fitness movement recognition and evaluation method based on sparse inertial sensors according to claim 5, characterized in that, The normalization preprocessing includes normalization operations performed on the main blade joint measurements and the blade joint measurements, respectively: The normalized calculation from the blade joint measurements is as follows: ； in, , This represents the normalized value measured from the blade joint. , These represent the attitude rotation matrices between the articular skeleton coordinate system and the SMPL coordinate system of the primary and secondary blades, respectively. , These represent the skeletal motion accelerations of the primary blade and the secondary blade in the SMPL coordinate system, respectively. The normalized calculation of the main blade joint measurements is as follows: ； in, , This represents the normalized joint measurement value of the main blade.

7. The method for fitness motion recognition and evaluation based on sparse inertial sensors according to claim 1, characterized in that, The fitness movement classification model in step S3 is an LSTM-RF hybrid model composed of a Long Short-Term Memory (LSTM) network and a Random Forest (RF) network. The specific training process includes: S31. Construct the original feature dataset from the motion information of the reconstructed joint nodes of the whole human body. S32. Input the original feature dataset into the LSTM model to extract the long-term dependencies and dynamic change patterns in the data, and obtain the output. ; S33, the output The new feature dataset is constructed by merging it with the original feature dataset. S34. Input the new feature dataset into the RF model for final classification to obtain the classification result of the fitness movement.

8. The method for fitness motion recognition and evaluation based on sparse inertial sensors according to claim 1, characterized in that, Step S4 specifically includes: S41. Based on the classification results, select the predetermined key joints; S42. Retrieve the standard action template corresponding to the classification result from the standard template library, wherein the standard action is a standard sequence; S43. Extract the timing data of the key joints in the current action during the movement process to form a test sequence; S44. Calculate the dynamic time-warped distance between the test sequence and the standard sequence, and obtain the quantitative evaluation result based on the distance.

9. A method for fitness motion recognition and evaluation based on sparse inertial sensors according to claim 8, characterized in that, The quantitative evaluation results in step S44 are expressed in the form of scores, which are calculated based on the cumulative warping distance obtained by the dynamic time warping algorithm. And obtained by mapping using the following formula: ； in, This indicates the final score. This indicates a perfect score. Indicates the base value. Indicates the impact factor.

Images