IMU (Inertial Measurement Unit) and structured light characteristic multi-modal motion posture dynamic evaluation method

Abstract

The invention discloses a multi-modal motion posture dynamic evaluation method based on IMU and structured light features, and belongs to the technical field of motion posture recognition. The method comprises the following steps: judging whether to carry out camera initialization operation and human body key point data dynamic correction or not according to external illumination parameters; judging whether high-frequency data correction is carried out or not according to the time domain evaluation parameters; judging whether data correction is performed or not, if yes, processing according to the corrected high-frequency data and the corrected human body key point data to obtain a detection action time sequence, and if not, directly processing according to the high-frequency data and the human body key point data to obtain the detection action time sequence; and optimizing the obtained detection action time sequence to obtain an optimized detection action time sequence, and evaluating the eligibility of the multi-modal motion attitude based on the core motion parameters obtained by the optimized detection action time sequence, thereby improving the real-time precision and robustness of the dynamic evaluation of the motion attitude.

Description

Technical Field

[0001] The present invention relates to the technical field of motion posture recognition, and in particular to a multi-modal motion posture dynamic evaluation method based on IMU and structured light features. Background Art

[0002] Multimodal motion posture dynamic assessment is achieved by fusing multi-source data such as vision, inertial measurement units, pressure sensors, and combining computer vision, biomechanical modeling, and signal processing technology to capture and analyze motion posture data in real time to achieve high-precision and robust motion state quantification. It aims to improve the intelligence level of human-computer interaction and health monitoring through dynamic, multi-dimensional posture assessment.

[0003] Existing multimodal motion posture dynamic assessment systems usually collect data in real time through multi-sensor fusion, combine computer vision, inertial measurement and biomechanical modeling, and use filtering algorithms and deep learning for data alignment, noise suppression and kinematic optimization, ultimately outputting high-precision motion posture data, which is applied to medical rehabilitation, motion analysis, virtual reality, human-computer interaction and other fields.

[0004] For example, the invention patent publication number CN118470806B discloses a method for athlete motion recognition based on posture assessment, including posture data acquisition, feature extraction, motion capture model construction, motion recognition, and practical application. This method selects optimal segmentation variables and points, iteratively partitions the posture data space, generates a decision tree, and extracts key posture data features. The method also constructs a motion capture model, creates a skeleton model, weight-binds virtual bones to the skeleton model surface, performs debugging and optimization, and performs vertex mapping transformations. Furthermore, the method acquires human motion data and introduces the human joint structure, calculates a loss function with local posture constraints, and performs hyperparameter optimization.

[0005] For example, the human posture recognition model construction method, motion posture evaluation method and system disclosed in the patent application with publication number CN119625838A include: obtaining a public human posture dataset and performing preprocessing; based on the YOLOv8-pose network framework, introducing an improved spatial pyramid pooling module SPPF and deformable convolution DCNv4, combining target box and human key point prediction to realize the detection and posture estimation of the human body in the image, and obtaining an improved human posture recognition network framework; repeatedly training the improved human posture recognition network framework according to the human posture dataset until the recognition rate reaches more than 90%, and obtaining the final human posture recognition model.

[0006] However, in the process of implementing the technical solutions of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:

[0007] In the existing technology, when evaluating the changes in an athlete's posture during exercise, sudden changes in illumination occur when the external light moves from shadow to strong light. At the same time, high-frequency swinging of the limbs may trigger high-frequency vibration fluctuations in the IMU (Inertial Measurement Unit) sensor, generating false signals. This results in low accuracy in the key point data of the human body obtained by the IMU sensor and the key point detection of the human body obtained by camera image analysis, and cumulative deviations in multi-modal motion posture parameters, resulting in low accuracy in real-time dynamic evaluation of athletic performance. Summary of the Invention

[0008] The present invention provides a multimodal motion posture dynamic evaluation method using IMU and structured light features, which solves the problem in the prior art that, when evaluating changes in an athlete's posture during exercise, sudden changes in illumination occur when external light moves from shadow to strong light, and high-frequency swinging of limbs may trigger high-frequency vibration fluctuations in the IMU sensor, generating false signals. This results in low accuracy in the detection of key human points obtained by the IMU sensor and camera image analysis, and cumulative deviations in multimodal motion posture parameters, leading to low accuracy in real-time dynamic evaluation of athletic performance. This improves the real-time accuracy and robustness of dynamic evaluation of motion posture.

[0009] The present invention provides a multi-modal motion posture dynamic evaluation method based on IMU and structured light features, comprising the following steps: performing a quantitative judgment on the external light illumination influence according to the external light illumination parameters monitored in real time before each motion cycle, obtaining a judgment result on the external light illumination influence, judging whether to perform a camera initialization operation and a dynamic correction of the key point data of the human body based on the judgment result on the external light illumination influence, wherein the dynamic correction of the key point data of the human body indicates that the human posture image acquired by the camera is optimized according to the external light illumination parameters to reduce the influence of the external light on the key point data of the human body; performing a quantitative judgment on the high-frequency vibration influence according to the time domain evaluation parameters monitored in real time during each motion cycle, obtaining a judgment result on the high-frequency vibration influence, judging whether to perform a high-frequency data correction based on the high-frequency vibration influence judgment result, wherein the high-frequency data correction indicates that the high-frequency data acquired by the IMU sensor is corrected according to the time domain evaluation parameters during each motion cycle. Optimize to reduce the impact of high-frequency vibration on data acquisition; determine whether the high-frequency data and the human body key point data have been corrected. If both have been corrected, the detection action time series is obtained based on the corrected high-frequency data and the corrected human body key point data. If the high-frequency data has been corrected but the human body key point data has not been corrected, the detection action time series is obtained based on the corrected high-frequency data and the human body key point data. If the high-frequency data has not been corrected but the human body key point data has been corrected, the detection action time series is obtained based on the high-frequency data and the corrected human body key point data. Otherwise, the detection action time series is directly obtained based on the high-frequency data and the human body key point data. Optimize the obtained detection action time series to obtain an optimized detection action time series, and evaluate the eligibility of the multimodal motion posture based on the core motion parameters obtained from the optimized detection action time series.

[0010] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0011] 1. The present invention provides a multimodal motion posture dynamic assessment method based on IMU and structured light features, which can monitor and quantify the impact of external lighting and high-frequency vibration on motion posture data in real time, thereby realizing dynamic correction of key point data and high-frequency data of the human body, optimizing the data acquisition process, and ultimately improving the accuracy and reliability of motion posture assessment.

[0012] 2. The present invention realizes the evaluation of the impact of external lighting on the accuracy of key point data of the human body through real-time monitoring and quantitative determination of external lighting parameters in each motion cycle, and then realizes the dynamic correction of key point data of the human body or adjustment of the camera filter device, thereby ensuring the accuracy and effectiveness of the key point data obtained by the camera under different lighting conditions.

[0013] 3. The present invention quantitatively determines the time domain evaluation parameters within each motion cycle, combines the high-frequency vibration impact evaluation results, and uses a dynamic correction method to process the IMU sensor data, thereby effectively reducing the data errors caused by high-frequency vibration, thereby achieving improved accuracy and reliability of high-frequency data.

[0014] 4. The present invention accurately evaluates multimodal motion postures based on the core motion parameters obtained from the optimized detection action time series, obtains the motion posture change, effectively distinguishes qualified and unqualified learning effects, thereby ensuring the effectiveness of the learning effect evaluation. By comparing the posture position difference with the standard key point position, the qualification judgment of the motion posture is further optimized, thereby realizing a comprehensive evaluation and precise marking of the learning effect, and improving the accuracy and scientificity of the motion posture evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of a multi-modal motion posture dynamic evaluation method based on IMU and structured light features provided in an embodiment of the present application.

[0016] Figure 2 A schematic diagram of the process of correcting the external light influence provided in an embodiment of the present application.

[0017] Figure 3 A schematic diagram of the process of correcting high-frequency vibration effects provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The embodiment of the present application provides a multimodal motion posture dynamic evaluation method using IMU and structured light features, which solves the problem in the prior art that when evaluating the changes in the athlete's posture during movement, the illumination mutation will occur when the external light moves from shadow to strong light, and the high-frequency swing of the limbs may cause high-frequency vibration fluctuations of the IMU sensor, generating false signals, resulting in low accuracy of the human body key point data acquired by the IMU sensor and the human body key point detection obtained by camera image analysis, and the multimodal motion posture parameters will produce cumulative deviations, resulting in low accuracy of real-time dynamic evaluation of sports performance. The method quantitatively determines the influence of external light according to the external light parameters monitored in real time before each movement cycle to obtain the external light influence determination result, and determines whether to perform camera initialization operation and dynamic correction of human body key point data based on the external light influence determination result. The dynamic correction of human body key point data means optimizing the human posture image acquired by the camera according to the external light parameters to reduce the influence of external light on the human body key point data; quantitatively determines the influence of high-frequency vibration according to the time domain evaluation parameters monitored in real time during each movement cycle to obtain the high-frequency vibration influence determination result. The method is used to determine whether to perform high-frequency data correction based on the high-frequency vibration influence judgment result. High-frequency data correction means optimizing the high-frequency data obtained by the IMU sensor according to the time domain evaluation parameters in each motion cycle to reduce the influence of high-frequency vibration on data acquisition. The method is used to determine whether the high-frequency data and the human body key point data have been corrected. If both have been corrected, the detection action time series is obtained based on the corrected high-frequency data and the corrected human body key point data. If the high-frequency data has been corrected but the human body key point data has not been corrected, the detection action time series is obtained based on the corrected high-frequency data and the human body key point data. If the high-frequency data has not been corrected but the human body key point data has been corrected, the detection action time series is obtained based on the high-frequency data and the corrected human body key point data. Otherwise, the detection action time series is directly obtained based on the high-frequency data and the human body key point data. The obtained detection action time series is optimized to obtain an optimized detection action time series. The eligibility of the multimodal motion posture is evaluated based on the core motion parameters obtained based on the optimized detection action time series, thereby improving the real-time accuracy and robustness of the dynamic evaluation of the motion posture.

[0019] The technical solution in the embodiments of the present application is to solve the above-mentioned problem of evaluating the changes in posture of athletes during movement. When the external light moves from shadow to strong light, a sudden change in illumination will occur. At the same time, the high-frequency swinging of the limbs may cause high-frequency vibration fluctuations in the IMU sensor, generating false signals. As a result, the key point data of the human body acquired by the IMU sensor and the key point detection accuracy of the human body obtained by camera image analysis are low, and the multi-modal motion posture parameters will produce cumulative deviations, resulting in low accuracy of real-time dynamic evaluation of sports performance. The overall concept is as follows:

[0020] By real-time monitoring of external lighting parameters and making quantitative judgments, the camera initialization and human key point data correction are dynamically adjusted to reduce the impact of sudden lighting changes on visual data. At the same time, the IMU high-frequency vibration is quantitatively judged based on time domain evaluation parameters, and high-frequency data correction is selectively triggered to suppress false signals. Subsequently, based on the data correction status, multimodal data is fused and processed to generate an optimized detection action time series, and finally high-precision core motion parameters are extracted, achieving improved real-time accuracy and robustness of dynamic motion posture evaluation.

[0021] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0022] like Figure 1As shown, it is a flow chart of a multi-modal motion posture dynamic evaluation method based on IMU and structured light features provided by an embodiment of the present application. The method includes the following steps: quantitatively determining the influence of external light illumination according to the external light parameters monitored in real time before each motion cycle, obtaining the external light illumination influence determination result, determining whether to perform camera initialization operation and dynamic correction of human key point data based on the external light illumination influence determination result, and the dynamic correction of human key point data means optimizing the human posture image acquired by the camera according to the external light parameters to reduce the influence of external light on the human key point data, wherein the human key point data is the human body posture image acquired by the camera according to the external light parameters to reduce the influence of external light on the human body key point data. light, SL) captures human posture images and uses AlphaPose (open source real-time multi-person human posture estimation system) to analyze the obtained human key point data, including the three-dimensional time series coordinates of each human key point, and each human key point includes shoulder, elbow, wrist, hip, knee, ankle joint, etc.; according to the time domain evaluation parameters monitored in real time during each motion cycle, the influence of high-frequency vibration is quantitatively determined to obtain the high-frequency vibration influence determination result, and based on the high-frequency vibration influence determination result, it is determined whether to perform high-frequency data correction. High-frequency data correction means optimizing the high-frequency data obtained by the IMU sensor according to the time domain evaluation parameters within each motion cycle to reduce the influence of high-frequency vibration on data acquisition. The high-frequency data includes acceleration data of each human key point, angular velocity data of each human key point, angle data of each human key point, torso posture angle data, velocity data of each human key point and Magnetic field strength data, etc.; determine whether the high-frequency data and the human body key point data have been corrected. If they have been corrected, the detection action time series is obtained based on the corrected high-frequency data and the corrected human body key point data. If the high-frequency data has been corrected but the human body key point data has not been corrected, the detection action time series is obtained based on the corrected high-frequency data and the human body key point data. If the high-frequency data has not been corrected but the human body key point data has been corrected, the detection action time series is obtained based on the high-frequency data and the corrected human body key point data. Otherwise, the detection action time series is directly obtained based on the high-frequency data and the human body key point data. The obtained detection action time series is optimized to obtain an optimized detection action time series. Based on the core motion parameters obtained from the optimized detection action time series, dynamic time warping (DTW) is used to evaluate the eligibility of the multimodal motion posture.

[0023] In this embodiment, the present invention significantly improves the accuracy and reliability of motion posture assessment in complex environments through dynamic illumination compensation and vibration suppression technology: based on real-time monitored illumination parameters, camera initialization is triggered, and structured light and AlphaPose are used to dynamically correct the data of each key point, effectively resisting the interference of strong light mutation on visual capture; the impact of high-frequency vibration on the IMU sensor is quantified through time domain evaluation parameter analysis, and high-frequency data is targeted to optimize to eliminate signal distortion caused by limb swing; the corrected multimodal data is selectively fused and compared with standardized motion parameters to achieve accurate posture qualification assessment in illumination mutation and high-frequency vibration scenarios. The present invention realizes adaptive calibration of multimodal data through a closed-loop feedback mechanism, and at the same time, accurately evaluates by selecting key motion indicators, which can comprehensively capture the details and dynamic trends of posture changes, improve the accuracy of posture assessment, and solve the challenges of dynamic posture changes, spatial feature extraction and multimodal data fusion during movement, thereby providing an efficient posture detection and evaluation tool for sports events, and ultimately achieving accurate motion analysis and optimization.

[0024] In addition, the motion posture database is used to store relevant data of the multimodal motion posture dynamic evaluation method based on IMU and structured light features, including: critical external light intensity, reference external light reflectivity, allowable deviation external light reflectivity, reference flicker frequency, first external light threshold and second external light threshold, etc. The data in the motion posture database can be directly queried through public databases for human posture estimation and motion capture such as the MPII (MPII Human Pose Dataset) dataset, and can be obtained through cooperation with corporate departments in the fields of sports competition, robotic automation or medical rehabilitation.

[0025] like Figure 2 The figure shows a flow chart of the correction of external light effects provided by an embodiment of the present application. The present invention monitors external light parameters in real time and performs external light index determination. When the external light index is less than or equal to the first external light threshold, the corresponding external light impact determination result is determined as the first light impact, and the original data processing of each key point of the human body is maintained; when the external light index is greater than the first external light threshold and less than the second external light threshold, the corresponding external light impact determination result is determined as the second light impact, and the data of each key point of the human body is corrected according to the difference between the second external light threshold and the external light index; when the external light index is greater than the second external light threshold, the corresponding external light impact determination result is determined as the third light impact, and camera filtering processing is performed, and finally the processed data of each key point of the human body is output.

[0026] Specifically, the influence of external light is quantitatively determined based on the external light parameters monitored in real time before each motion cycle, and the steps for obtaining the judgment result of the influence of external light include: first, obtaining reference data of external light parameters from a preset motion posture database, specifically including: critical external light intensity, reference external light reflectivity, allowable deviation external light reflectivity and reference flicker frequency; performing a proportion approach calculation on the external light intensity obtained before each motion cycle and the critical external light intensity, the proportion approach calculation is a ratio calculation, and obtaining the external light intensity influence parameter; performing a relative deviation conformity calculation on the external light reflectivity of each key point of the human body obtained before each motion cycle and the reference external light reflectivity and the allowable deviation external light reflectivity, the relative deviation conformity calculation is a dynamic normalization process, and then performing a relative deviation conformity calculation on the relative deviation conformity. The calculation results are averaged to obtain the external light reflectivity influencing parameters; the reference flicker frequency and the flicker frequency obtained before each motion cycle are compared with the reference flicker frequency for relative deviation conformity calculation to obtain the flicker frequency influencing parameters; the external light intensity influencing parameters, the external light reflectivity influencing parameters and the flicker frequency influencing parameters are weighted using the external light parameter weight ratio, and the weighted processing results are coupled to obtain the external light index of each motion cycle. The external light parameter weight ratio includes the external light intensity weight ratio, the external light reflectivity weight ratio and the flicker frequency weight ratio. The external light index represents the quantitative data of the degree of influence of the external light parameters on the accuracy of the human body key point data. The external light parameters include the external light intensity, the external light reflectivity and the flicker frequency of each human body key point.

[0027] The external light index of each motion cycle is obtained as follows:

[0028]

[0029] Where, EL j represents the external light index of the jth motion cycle, α represents the weight ratio of the external light intensity, β represents the weight ratio of the external light reflectivity, γ represents the weight ratio of the flicker frequency, and II j It represents the external light intensity of the j-th motion cycle, which can be directly measured by using a light sensor such as a photometer or illuminance meter on the camera. II0 represents the critical external light intensity, LR ij It represents the external light reflectance of the i-th key point of the human body in the j-th motion cycle. It can be measured by using a spectral reflectometer at each key point of the human body. LR0 represents the reference external light reflectance, LR2 represents the allowable deviation external light reflectance, and FF jrepresents the flicker frequency of the jth motion cycle, which can be measured by using a scintillation meter on the camera. FF0 represents the reference flicker frequency, that is, the camera frequency. Here, i is the number of each key point on the human body, i = 1, 2, 3, ..., N, N is the total number of key points on the human body, j is the number of each motion cycle, j = 1, 2, 3, ..., M, M is the total number of motion cycles.

[0030] α, β, and γ are respectively the weight ratios of the external light intensity, the external light reflectivity of the key points of the human body, and the flicker frequency preset in the motion posture database. These weight ratios are numerical indicators that measure the influence of the above-mentioned external light parameters on the external light index. Specifically, there is a mapping relationship table for the external light intensity, the external light reflectivity of each key point of the human body, and the flicker frequency. The table records each possible external light parameter value and its weight ratio. These mapping relationships can be one-to-one or many-to-one. For example, in actual applications, when it is necessary to evaluate the external light index of a certain motion cycle, the measured external light intensity, the external light reflectivity of each key point of the human body, and the flicker frequency can be input into their respective corresponding mapping relationship tables, and the weight ratios of these values ​​can be quickly found. The weight ratio ranges from 0 to 1.

[0031] The three factors are interrelated. For example, ambient light intensity and the reflectivity of key points on the human body together determine the effect of light reflection. Stronger ambient light intensity and higher human body reflectivity result in higher reflected light intensity, affecting the user's visual perception and posture accuracy. Furthermore, ambient light intensity directly influences the intensity of reflected light. Excessive ambient light intensity can lead to saturation of reflected light, making changes in reflectivity less noticeable, thus affecting accuracy assessment. Flicker frequency is closely related to the type of light source. Flicker frequency variations can affect light fluctuations during motion detection. Stronger ambient light intensity can result in higher flicker frequencies, while weaker ambient light intensity has a smaller impact on flicker frequency, thus affecting camera measurement stability. The ambient light index, derived from comprehensive analysis, reflects the stability and intensity of ambient light. This helps optimize illumination compensation in motion detection and posture estimation algorithms based on the ambient light index, reducing the impact of ambient light variations on detection results and improving the robustness of motion analysis. This can especially be achieved in complex lighting conditions, avoiding misjudgments or data errors caused by light fluctuations.

[0032] Then, the first external light threshold and the second external light threshold are obtained from the preset motion posture database; the external light index of each motion cycle is compared with the first external light threshold and the second external light threshold respectively; if the external light index of a certain motion cycle is less than or equal to the first external light threshold, the corresponding external light illumination impact judgment result is recorded as the first light impact; if the external light index of a certain motion cycle is greater than the first external light threshold and less than or equal to the second external light threshold, the corresponding external light illumination impact judgment result is recorded as the second light impact; if the external light index of a certain motion cycle is greater than the second external light threshold, the corresponding external light illumination impact judgment result is recorded as the third light impact.

[0033] Finally, based on the result of the external light illumination influence judgment, it is determined whether to perform the camera initialization operation and the dynamic correction of the human body key point data: if the external light illumination influence judgment result of the camera is the first illumination influence, no additional processing is performed; if the external light illumination influence judgment result of the camera is the second illumination influence, the difference between the second external light threshold and the external light index is marked as the external light deviation index, and the external light deviation index is matched with the human body key point data correction value corresponding to each external light deviation index preset in the motion posture database to obtain the human body key point data correction value. In the motion posture database, each external light deviation index and the human body key point data correction value correspond one to one to form a mapping relationship table, which records each external light deviation index and its corresponding human body key point data correction value. These relationships can be one-to-one or many-to-one. When correcting the human key point data, it is only necessary to input the external lighting deviation index into the mapping relationship table, and the motion posture database can quickly locate and return the human key point data correction value corresponding to the external lighting deviation index. The smaller the external lighting deviation index, the larger the human key point data correction value. The human key point data correction value is summed with the human key point data to obtain the corrected human key point data; if the camera's external light illumination impact judgment result is the third illumination impact, the corresponding human key point data is marked as invalid data, and it is determined whether the camera filter device, including optical filters, light hoods, etc., has been turned on. If so, the camera filter device is turned on for an extended period of time until the next motion cycle, otherwise the camera filter device is turned on, and dynamic lighting compensation is performed. The ambient light sensor is used to monitor the light intensity in real time, and the brightness of the structured light is automatically adjusted according to the light intensity.

[0034] In this embodiment, the present invention significantly improves the accuracy of motion posture analysis in complex lighting environments through an intelligent hierarchical lighting compensation mechanism: the external lighting index of the motion cycle is compared with the dual thresholds, and dynamically divided into three levels: the first lighting impact, the second lighting impact, and the third lighting impact, to achieve quantitative hierarchical management of lighting interference; for the second lighting impact, the visual data compensation amount is quickly matched through a pre-built deviation index-correction value mapping table to avoid the over-correction problem of the traditional fixed threshold method; when the third lighting impact occurs, the filtering device is automatically triggered and dynamic lighting compensation is started, and the brightness of the structured light is adjusted in conjunction with the ambient light sensor to ensure the validity of the data under extreme lighting.

[0035] like Figure 3 The figure shows a flow chart of the high-frequency vibration impact correction provided by an embodiment of the present application. The present invention monitors the IMU time domain parameters in real time and makes a judgment based on the time domain evaluation index obtained from the IMU time domain parameters. When the time domain evaluation index is less than or equal to the first time domain evaluation threshold, the corresponding high-frequency vibration impact judgment result is judged as the first vibration impact, and the original IMU data, that is, the high-frequency data directly monitored by the IMU sensor, is retained; when the time domain evaluation index is greater than the first time domain evaluation threshold and less than or equal to the second time domain evaluation threshold, the corresponding high-frequency vibration impact judgment result is judged as the second vibration impact, and the IMU data is corrected according to the difference between the second time domain evaluation threshold and the time domain evaluation index; when the time domain evaluation index is greater than the second time domain evaluation threshold, the corresponding high-frequency vibration impact judgment result is judged as the third vibration impact, and the IMU sensor signal is filtered and processed, and the optimized IMU data, that is, the optimized high-frequency data, is finally output.

[0036] Specifically, the quantitative determination of the high-frequency vibration influence is performed based on the time domain evaluation parameters monitored in real time during each motion cycle, and the steps for obtaining the high-frequency vibration influence determination result include: first, obtaining the time domain evaluation parameter reference data from a preset motion posture database, specifically including: critical peak acceleration, critical root mean square acceleration and critical acceleration of each key point of the human body; performing a ratio approach calculation on the peak acceleration and root mean square acceleration in each motion cycle with the critical peak acceleration and critical root mean square acceleration, respectively, to obtain the peak acceleration influence parameter and the root mean square acceleration influence parameter; performing a ratio approach calculation on the acceleration of each key point of the human body with the critical acceleration of each key point of the human body, and then performing a ratio approach calculation on the peak acceleration and the critical root mean square acceleration, respectively, to obtain the peak acceleration influence parameter and the root mean square acceleration influence parameter; performing a ratio approach calculation on the acceleration of each key point of the human body with the critical acceleration of each key point of the human body, and then performing a ratio approach calculation on the peak acceleration and the critical root mean square acceleration, respectively, to obtain the peak acceleration influence parameter and the root mean square acceleration influence parameter; The results of the approximation calculation are averaged to obtain the acceleration influencing parameters of the key points of the human body. The peak acceleration influencing parameters, the root mean square acceleration influencing parameters and the acceleration influencing parameters of the key points of the human body are weighted respectively using the weight ratio of the time domain evaluation parameters. The weighted processing results are then coupled to obtain the time domain evaluation index within each motion cycle. The time domain evaluation parameter weight ratio includes the peak acceleration weight ratio, the root mean square acceleration weight ratio and the acceleration weight ratio of the key points of the human body. The time domain evaluation index represents the quantitative data of the degree of influence of the time domain evaluation parameters on the accuracy of high-frequency data. The time domain evaluation parameters include the peak acceleration, the root mean square acceleration and the acceleration of each key point of the human body.

[0037] The time domain evaluation index in each motion cycle is obtained as follows:

[0038]

[0039] In the formula, TD j represents the time domain evaluation index in the jth motion cycle, μ represents the peak acceleration weight ratio, represents the weight ratio of the root mean square acceleration, ρ represents the weight ratio of the acceleration of the key points of the human body, PA j It represents the peak acceleration in the jth motion cycle, which refers to the maximum instantaneous value of acceleration in the motion cycle. PA0 represents the critical peak acceleration, SA j It represents the root mean square acceleration in the jth motion cycle, which refers to the average intensity of the acceleration change in the motion cycle. SA0 represents the critical root mean square acceleration, AA ij Indicates the acceleration of the i-th key point of the human body in the j-th motion cycle, AA i0 represents the critical acceleration of the i-th human key point.

[0040] μ, and ρ are the weight ratios of peak acceleration, root mean square acceleration, and human key point acceleration preset in the motion posture database, respectively. These weight ratios are numerical indicators that measure the influence of the above-mentioned time domain evaluation parameters on the time domain evaluation index. Specifically, there is a mapping relationship table for each of the peak acceleration, root mean square acceleration, and human key point acceleration, which records each possible time domain evaluation parameter value and its corresponding weight ratio. These mapping relationships can be one-to-one or many-to-one. For example, in actual applications, when it is necessary to evaluate the time domain evaluation index within a certain motion cycle, the measured peak acceleration, root mean square acceleration, and human key point acceleration can be input into their respective corresponding mapping relationship tables, and the corresponding device performance index weight ratios of these values ​​can be quickly found. The weight ratio ranges from 0 to 1.

[0041] Peak acceleration, RMS acceleration, and acceleration at key points on the human body can be directly measured using IMU sensors, and the three are interrelated. For example, peak acceleration and RMS acceleration jointly describe the intensity of acceleration. Peak acceleration focuses on reflecting extreme conditions during movement, while RMS acceleration comprehensively considers overall acceleration fluctuations. Together, they provide a dual perspective on acceleration intensity and stability during movement. A general increase in the acceleration amplitude at key points on the human body indicates an increase in peak acceleration, an increase in overall human acceleration, and an increase in RMS acceleration. When the acceleration amplitude at a key point on the human body increases, if the amplitude changes in an upward direction, the acceleration at that key point increases; if the amplitude changes in a downward direction, the acceleration at that key point decreases. By analyzing the acceleration of different parts of the body, the acceleration of key points can refine each key point of movement and provide more accurate motion data. Combined with peak acceleration and RMS acceleration, they can provide in-depth analysis of the degree of acceleration or deceleration at different parts of the body during movement, further optimizing the assessment of movement cycles. The time domain evaluation index obtained through comprehensive analysis can accurately evaluate the changes in acceleration during the motion cycle. Optimizing high-frequency data based on the time domain evaluation index can reduce the impact of high-frequency vibration on data acquisition, thereby achieving more accurate high-frequency data monitoring and analysis.

[0042] Then, a first time domain evaluation threshold and a second time domain evaluation threshold are obtained from a preset motion posture database; the time domain evaluation index in each motion cycle is compared with the first time domain evaluation threshold and the second time domain evaluation threshold respectively; if the time domain evaluation index in a certain motion cycle is less than or equal to the first time domain evaluation threshold, the corresponding high-frequency vibration impact determination result is recorded as the first vibration impact; if the time domain evaluation index in a certain motion cycle is greater than the first time domain evaluation threshold and less than or equal to the second time domain evaluation threshold, the corresponding high-frequency vibration impact determination result is recorded as the second vibration impact; if the time domain evaluation index in a certain motion cycle is greater than the second time domain evaluation threshold, the corresponding high-frequency vibration impact determination result is recorded as the third vibration impact.

[0043] Finally, based on the high-frequency vibration impact judgment result, the high-frequency data obtained by the IMU sensor is dynamically corrected: if the high-frequency vibration impact judgment result of the IMU sensor in a certain motion cycle is the first vibration impact, no additional processing is performed; if the high-frequency vibration impact judgment result of the IMU sensor in a certain motion cycle is the second vibration impact, the motion data of the corresponding high-frequency data is corrected according to the difference between the second threshold of the time domain evaluation and the time domain evaluation index in the motion cycle. The motion data includes the acceleration data of each key point of the human body, the angular velocity data of each key point of the human body, the angle data of each key point of the human body, the torso posture angle data and the velocity data of each key point of the human body. The high-frequency noise component in the high-frequency data is extracted using the Fast Fourier Transform (FFT). The motion data of the corrected high-frequency data = the motion data of the original high-frequency data - the correction weight coefficient × the high-frequency noise component, wherein, If the high-frequency vibration impact judgment result of the IMU sensor in a certain movement cycle is the third vibration impact, an IMU sensor data abnormality prompt will be issued to determine whether the low-pass filter has been turned on. The low-pass filter can eliminate high-frequency components and retain low-frequency signals, thereby reducing false signals caused by high-frequency swinging of the limbs. If so, the low-pass filter opening time will be extended to the next movement cycle. Otherwise, the low-pass filter will be turned on and the Kalman filter will be used for signal denoising.

[0044] In this embodiment, the present invention significantly improves the reliability of IMU data in high-frequency motion scenarios through an intelligent hierarchical vibration suppression strategy: the time domain evaluation index is compared with the dual threshold, and the three levels of first vibration influence, second vibration influence and third vibration influence are dynamically divided to achieve accurate quantitative grading of vibration interference; for the second vibration influence, the high-frequency noise component is extracted by fast Fourier transform, and the acceleration and angular velocity data are dynamically adjusted in combination with the correction weight coefficient to avoid the phase distortion problem of the traditional filtering algorithm; when the third vibration influence occurs, the low-pass filter and Kalman filter are automatically activated for dual denoising, which not only eliminates the high-frequency artifacts of limb swing but also retains the effective motion signal, solving the problem of IMU data drift caused by strenuous exercise.

[0045] Furthermore, taking the case where both high-frequency data and human key point data have been corrected as an example, the steps of obtaining a detection action time series based on the corrected high-frequency data and the corrected human key point data include: first, inputting the corrected high-frequency data as a time series into an IMU-long short-term memory (LSTM) network to learn the time series characteristics of the IMU sensor during motion, representing the changing trend of motion acceleration and angular velocity, and obtaining time series characteristics; then using a graph convolutional network (GCN) to extract spatial features of the corrected human key point data to obtain spatial posture features, which include the spatial coordinates of each human key point, the connection relationship between each key point, and motion characteristics; obtaining a weight vector of the time series features and a weight vector of the spatial posture features of each human key point from a preset motion posture database; and then using an attention mechanism to obtain the weight of the time series features and the weight of the spatial posture features based on the time series features, the spatial posture features, the weight vector of the time series features, and the weight vector of the spatial posture features of each human key point, respectively. The weight of the time series features and the weight of the spatial posture features are obtained as follows:

[0046]

[0047] In the formula, δ represents the weight of the time series feature, Represents the weight of the spatial posture feature, W is the weight vector of the time series feature, h is the time series feature representation, W i is the weight vector of the spatial posture feature of the i-th human key point, h i is the spatial posture feature representation of the i-th human key point.

[0048] Through dynamic time warping, the weights of the time series features and the weights of the spatial posture features are used to splice the time series features and the spatial posture features. For example, the elements corresponding to the time series features and the spatial posture features at each moment are connected one by one. The connected vector contains information on both time and space, forming a multimodal feature vector. Due to the high dimension of the multimodal feature vector, the computational complexity and memory consumption are high. At the same time, the multimodal feature vector may contain repeated or unimportant information. Therefore, the total dimension of the multimodal feature vector is input into the fully connected layer, and activation functions such as ReLU are used to perform nonlinear operations on the multimodal feature vector. The multimodal feature vector is reduced in dimensionality according to task requirements (such as 10% of the input dimension) to obtain the detection action time series. The steps of optimizing the obtained detection action time sequence to obtain the optimized detection action time sequence include: obtaining a sequence spacing threshold from a preset motion posture database; comparing the sequence spacings of the detection action time sequence with the sequence spacing threshold respectively; if a sequence spacing of the detection action time sequence does not exceed the sequence spacing threshold, the sequence segment is marked as a normal sequence segment; if a sequence spacing of the detection action time sequence exceeds the sequence spacing threshold, it is prompted that the sequence segment is abnormal, and the sequence segment is marked as an abnormal sequence segment, and the abnormal sequence segment is optimized according to the continuous segment length to obtain the optimized detection action time sequence: obtaining a segment length threshold from a preset motion posture database; comparing the lengths of each continuous segment of the abnormal sequence segment with the segment length threshold respectively; if the abnormal If the length of a continuous segment of the sequence segment does not exceed the segment length threshold, the continuous segment is marked as the first abnormal sequence segment, and the median of the features of the adjacent sequence segments of the continuous segment is used to replace the current value. The first abnormal sequence segment is corrected according to the features of the adjacent sequence segments of the continuous segment; if the length of a continuous segment of the abnormal sequence segment exceeds the segment length threshold, the continuous segment is marked as the second abnormal sequence segment, and the second abnormal sequence segment is optimized using the detection action time series information. The probability distribution of the sequence segment is learned by the variational autoencoder (VAE) through the encoder-decoder structure to generate a continuous and smooth new sequence segment. The generative adversarial network (GAN) is used to generate the new sequence segment. The Adversarial Network continuously traverses through the generator and discriminator. The generator generates realistic sequence fragments based on the input normal sequence fragment information or noise information, and updates its parameters so that the sequence fragments generated by the generator are closer and closer to the real data distribution. The discriminator distinguishes whether the input information is real and updates its parameters so that the discriminator can better distinguish between true and false data. The generator and discriminator optimize each other in the alternating process, and finally generate realistic sequence fragments, obtaining the optimized detection action time series.

[0049] In this embodiment, the present invention significantly improves the efficiency and robustness of motion posture analysis through an intelligent timing optimization algorithm: dynamic time warping is used to weightedly splice spatiotemporal features, and then dimensionality reduction is performed through a fully connected layer to reduce the computational complexity while retaining key information; abnormal fragments are intelligently identified based on the sequence spacing threshold, and the first abnormal sequence fragment is quickly repaired using a median filter. For the second abnormal sequence fragment, VAE and GAN are combined to generate smooth data that conforms to the laws of motion mechanics. VAE ensures the rationality of data distribution, and GAN enhances the authenticity of generated data. The two work together to solve the motion distortion problem caused by traditional interpolation methods, realize intelligent repair of motion sequences, and make the evaluation results of complex movements both highly accurate and physiologically reasonable, which is particularly suitable for high-dynamic movement analysis in competitive sports.

[0050] Furthermore, the step of evaluating the eligibility of multimodal motion postures based on core motion parameters obtained from the optimized motion time series includes: core motion parameters include coordinates of key body points, trunk posture angles, and key body joint angles, with trunk posture angles including pitch, roll, and yaw angles. A change in center of gravity position is obtained based on the mass ratios of key body points preset in an existing anatomical model and the coordinates of key body points during the athlete's pre- and post-training motion cycles; a change in rotation is obtained based on the trunk posture angles during the athlete's pre- and post-training motion cycles; and a change in posture accuracy is obtained based on the angles of key body joints during the athlete's pre- and post-training motion cycles. Athlete training refers to real-life training to improve an athlete's physical fitness and skills, such as physical training, tactical training, and technical training. A change in motion posture is obtained based on the change in center of gravity position, the change in rotation, and the change in posture accuracy.

[0051] The method for obtaining the change in motion posture is as follows:

[0052] In the formula, MP represents the change of motion posture, τ represents the weight ratio of the change of center of gravity position, The weight ratio of the curl change, ω represents the weight ratio of the posture accuracy change, and GP represents the center of gravity position change. The coordinates of the key points of the human body at each time monitoring point in the pre-training exercise cycle and the post-training exercise cycle can be obtained through Euclidean calculation and mean operation. The center of gravity position coordinates of the key points of the human body are: (x i ,y i ,z i ) is the coordinate of the key point of the i-th human body, (x 0i ,y 0i ,z 0i) is the initial coordinate of the i-th human key point, where the initial coordinates are different in different postures. For example, when the subject is in an upright neutral position, the center of gravity is located 1-2 cm in front of the second sacral vertebra, while in a quadruped support posture, the center of gravity will move downward and closer to the center of the body trunk, usually located within the vertical projection range close to the junction of the thorax and pelvis, m i represents the mass ratio of the i-th key point of the human body. The change in the center of gravity coordinates varies with different movements. For example, assuming the subject is 1.75m tall, the center of gravity height is approximately 0.96m (Z axis) when standing, and the horizontal position (X / Y axis) is close to the body midline. During the standing long jump, the center of gravity position will drop when the subject squats to prepare for the jump, and the horizontal position will shift slightly backward. When preparing to throw, the center of gravity position will shift toward the supporting leg. When carrying a heavy object, the center of gravity position will shift toward the load direction. RC represents the change in rotation, which is calculated as follows: Where, (θ pa ,θ ra ,θ ya ) is the trunk posture angle at the ath time monitoring point in the pre-training exercise cycle, (θ pa,0 ,θ ra,0 ,θ ya,0 ) is the trunk posture angle at the ath time monitoring point in the exercise cycle after training, a is the number of each time monitoring point, a = 1, 2, 3, ..., A, A is the total number of time monitoring points; AC represents the change in posture accuracy, which can be obtained by calculating the average value of the difference between the joint angles at each time monitoring point in the exercise cycle before training and the exercise cycle after training. The calculation formula of the joint angle is: Where θ is the joint angle, and are the vectors between two adjacent joints.

[0053] τ, and ω are the weight ratios of the center of gravity position change, rotation change and attitude accuracy change preset in the motion posture database, respectively. These weight ratios are numerical indicators that measure the influence of the above parameters on the motion posture change. Specifically, there is a mapping relationship table for each center of gravity position change, rotation change and attitude accuracy change, which records each possible parameter value and its corresponding weight ratio. These mapping relationships can be one-to-one or many-to-one. For example, in actual applications, when it is necessary to evaluate the change in a certain motion posture, the measured center of gravity position change, rotation change and attitude accuracy change can be input into their respective corresponding mapping relationship tables, and the weight ratios corresponding to these values ​​can be quickly found, where the weight ratio ranges from 0 to 1.

[0054] A motion posture change threshold is obtained from a preset motion posture database, and the motion posture change amount is compared with the motion posture change threshold. If the motion posture change amount exceeds the motion posture change threshold and each core motion parameter is within the preset standard core motion parameter range of the motion posture database, the learning effect is marked as valid, and the effective learning effect is evaluated according to the position of each human key point and the standard key point position. The qualification of the multimodal motion posture: the posture position difference is obtained according to the position of each human key point and the standard position of each human key point in the learning group; the posture position difference threshold is obtained from the preset motion posture database, and the posture position difference is compared with the posture position difference threshold. If the posture position difference does not exceed the posture position difference threshold, the motion posture is recorded as qualified. If the posture position difference exceeds the posture position difference threshold, the motion posture is recorded as unqualified, and a position deviation report of each key point is automatically generated. Otherwise, the learning effect is marked as invalid.

[0055] The attitude position difference is obtained as follows:

[0056]

[0057] Where AP represents the attitude position difference, (x 1i ,y 1i ,z 1i ) represents the position of the i-th human key point in the learning group, (x 2i ,y 2i ,z 2i ) represents the standard position of the i-th human key point.

[0058] In this embodiment, the present invention realizes accurate quantitative evaluation of the learning effect of movement posture through an intelligent threshold judgment mechanism: by comparing the change amount of movement posture with the preset threshold, combined with double verification of the core movement parameter range, it is ensured that only movement changes that conform to the laws of biomechanics are marked as valid learning data; based on the intelligent comparison of the key point position difference and the database threshold, the movement posture is objectively classified as "qualified / unqualified", replacing the traditional subjective scoring method; when the posture is unqualified, the system automatically generates a key point position deviation report (such as the right knee is overextended by 2.5cm), providing data support for training correction, making the training effect evaluation both scientific and operational.

[0059] In summary, the embodiments of the present application monitor the external lighting parameters in real time and make quantitative judgments, dynamically adjust the camera initialization and human key point data correction to reduce the impact of sudden lighting changes on visual data; at the same time, the IMU high-frequency vibration is quantitatively judged based on the time domain evaluation parameters, and high-frequency data correction is selectively triggered to suppress false signals; then, according to the data correction status, the multimodal data is fused and processed to generate an optimized detection action time series, and finally high-precision core motion parameters are extracted, thereby achieving improved real-time accuracy and robustness of dynamic motion posture evaluation.

[0060] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0062] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1A step that specifies a function in one or more boxes.

[0064] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0065] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A multi-modal motion posture dynamic assessment method based on IMU and structured light features, characterized in that: The following steps are involved: Quantitatively determine the impact of external light according to external light parameters monitored in real time before each motion cycle, and obtain an external light impact determination result. Based on the external light impact determination result, determine whether to perform a camera initialization operation and dynamically correct the key point data of the human body. The dynamic correction of the key point data of the human body means optimizing the human posture image acquired by the camera according to the external light parameters to reduce the impact of external light on the key point data of the human body; Quantitatively determine the impact of high-frequency vibration based on time-domain evaluation parameters monitored in real time during each motion cycle to obtain a high-frequency vibration impact determination result, and determine whether to perform high-frequency data correction based on the high-frequency vibration impact determination result. The high-frequency data correction means optimizing the high-frequency data acquired by the IMU sensor based on the time-domain evaluation parameters during each motion cycle to reduce the impact of high-frequency vibration on data acquisition; Determine whether the high-frequency data and the human body key point data have been corrected. If both have been corrected, obtain a detection action time series based on the corrected high-frequency data and the corrected human body key point data. If the high-frequency data has been corrected but the human body key point data has not been corrected, obtain a detection action time series based on the corrected high-frequency data and the human body key point data. If the high-frequency data has not been corrected but the human body key point data has been corrected, obtain a detection action time series based on the high-frequency data and the corrected human body key point data. Otherwise, obtain a detection action time series directly based on the high-frequency data and the human body key point data. The obtained detection action time series is optimized to obtain an optimized detection action time series, and the eligibility of the multimodal motion posture is evaluated based on the core motion parameters obtained from the optimized detection action time series.

2. The multi-modal motion posture dynamic assessment method based on IMU and structured light features as claimed in claim 1, characterized in that: The step of performing quantitative determination of the external light impact based on the external light parameters monitored in real time before each movement cycle to obtain the external light impact determination result includes: Obtaining external light parameter reference data from a preset motion posture database, specifically including: critical external light intensity, reference external light reflectivity, allowable deviation external light reflectivity, and reference flicker frequency; The external light intensity obtained before each movement cycle is calculated with respect to the critical external light intensity to obtain the external light intensity influencing parameter; The relative deviation conformity calculation is performed on the external light reflectance of each key point of the human body obtained before each movement cycle, the reference external light reflectance and the allowable deviation external light reflectance, and then the relative deviation conformity calculation results are averaged to obtain the external light reflectance influencing parameter; The reference flicker frequency and the flicker frequency obtained before each motion cycle are subjected to relative deviation conformity calculation with the reference flicker frequency to obtain flicker frequency influencing parameters; The external light intensity influencing parameter, the external light reflectivity influencing parameter, and the flicker frequency influencing parameter are weighted using the external light parameter weight ratio, and the weighted processing results are coupled to obtain the external light index of each motion cycle, wherein the external light parameter weight ratio includes the external light intensity weight ratio, the external light reflectivity weight ratio, and the flicker frequency weight ratio. The external light index represents quantitative data of the degree of influence of the external light parameters on the accuracy of the human body key point data, and the external light parameters include the external light intensity, the external light reflectivity of each human body key point, and the flicker frequency; Obtaining a first external illumination threshold and a second external illumination threshold from a preset motion posture database; The external light index of each movement cycle is compared with the first external light threshold and the second external light threshold respectively. If the external light index of a certain movement cycle is less than or equal to the first external light threshold, the corresponding external light impact determination result is recorded as the first light impact; If the ambient light index of a certain motion cycle is greater than the first ambient light threshold and less than or equal to the second ambient light threshold, the corresponding ambient light impact determination result is recorded as the second ambient light impact; If the ambient light index of a certain motion cycle is greater than the second ambient light threshold, the corresponding ambient light impact determination result is recorded as the third ambient light impact.

3. The multi-modal motion posture dynamic assessment method based on IMU and structured light features as claimed in claim 2, characterized in that: The step of determining whether to perform camera initialization and dynamic correction of human key point data based on the external light illumination influence determination result includes: If the camera's ambient light illumination impact determination result is the first illumination impact, no additional processing is performed; If the camera's external light illumination influence determination result is the second illumination influence, then the difference between the second external light threshold and the external light index is marked as the external light deviation index, the external light deviation index is matched with the human body key point data correction values ​​corresponding to each external light deviation index preset in the motion posture database to obtain the human body key point data correction values, and the human body key point data is corrected based on the human body key point data correction values ​​to obtain the corrected human body key point data; If the camera's external light illumination impact judgment result is the third illumination impact, the corresponding human body key point data will be marked as invalid data, and it will be determined whether the camera filter device has been turned on. If so, the camera filter device turn-on time will be extended to the next motion cycle; otherwise, the camera filter device will be turned on and dynamic illumination compensation will be performed.

4. The multi-modal motion posture dynamic assessment method based on IMU and structured light features as claimed in claim 1, characterized in that: The step of performing quantitative determination of the high-frequency vibration impact based on the time domain evaluation parameters monitored in real time during each motion cycle to obtain the high-frequency vibration impact determination result includes: Obtain time domain evaluation parameter reference data from a preset motion posture database, specifically including: critical peak acceleration, critical root mean square acceleration, and critical acceleration of each key point of the human body; The peak acceleration and root mean square acceleration in each motion cycle are respectively subjected to a proportion approaching operation with the critical peak acceleration and critical root mean square acceleration to obtain the peak acceleration influencing parameters and the root mean square acceleration influencing parameters; Perform a ratio approximation calculation on the acceleration of each key point of the human body and the critical acceleration of each key point of the human body, and then perform average processing on the results of the ratio approximation calculation to obtain the acceleration influence parameters of the key points of the human body; The peak acceleration influencing parameters, the root mean square acceleration influencing parameters and the human body key point acceleration influencing parameters are weighted respectively using the time domain evaluation parameter weight ratio, and the weighted processing results are coupled to obtain the time domain evaluation index within each motion cycle, wherein the time domain evaluation parameter weight ratio includes the peak acceleration weight ratio, the root mean square acceleration weight ratio and the human body key point acceleration weight ratio, and the time domain evaluation index represents the quantitative data of the degree of influence of the time domain evaluation parameters on the accuracy of the high-frequency data, and the time domain evaluation parameters include the peak acceleration, the root mean square acceleration and the acceleration of each human body key point; Acquire a first time domain evaluation threshold and a second time domain evaluation threshold from a preset motion posture database; The time domain evaluation index in each motion cycle is compared with the first time domain evaluation threshold and the second time domain evaluation threshold. If the time domain evaluation index in a certain motion cycle is less than or equal to the first time domain evaluation threshold, the corresponding high-frequency vibration impact determination result is recorded as the first vibration impact. If the time domain evaluation index within a certain motion cycle is greater than the first time domain evaluation threshold and less than or equal to the second time domain evaluation threshold, the corresponding high-frequency vibration impact determination result is recorded as the second vibration impact; If the time domain evaluation index within a certain motion cycle is greater than the second time domain evaluation threshold, the corresponding high-frequency vibration impact determination result is recorded as the third vibration impact.

5. The multi-modal motion posture dynamic assessment method based on IMU and structured light features as claimed in claim 4, characterized in that: The step of dynamically correcting the high-frequency data obtained by the IMU sensor based on the high-frequency vibration impact determination result includes: If the high-frequency vibration impact determination result of the IMU sensor within a certain motion cycle is the first vibration impact, no additional processing is performed; If the high-frequency vibration influence determination result within a certain motion cycle of the IMU sensor is the second vibration influence, the motion data of the corresponding high-frequency data is corrected according to the difference between the second time domain evaluation threshold and the time domain evaluation index within the motion cycle, wherein the motion data includes acceleration data of each key point of the human body, angular velocity data of each key point of the human body, angle data of each key point of the human body, torso posture angle data, and velocity data of each key point of the human body; If the high-frequency vibration impact of the IMU sensor within a certain motion cycle is determined to be the third vibration impact, an IMU sensor data abnormality prompt will be issued to determine whether the low-pass filter has been turned on. If so, the low-pass filter on time will be extended to the next motion cycle. Otherwise, the low-pass filter will be turned on and signal denoising will be performed.

6. The multimodal motion posture dynamic assessment method using IMU and structured light features as claimed in claim 1, characterized in that: The step of obtaining a detection action time sequence based on the corrected high-frequency data and the corrected human body key point data comprises: Obtain time series features based on the corrected high-frequency data; Obtain spatial posture features based on the corrected human body key point data; Obtaining the weight vector of the time series feature and the weight vector of the spatial posture feature of each key point of the human body from a preset motion posture database; The weight of the time series feature and the weight of the spatial posture feature are obtained respectively according to the time series feature, the spatial posture feature, the weight vector of the time series feature and the weight vector of the spatial posture feature of each key point of the human body; The time series features and the spatial posture features are concatenated using the weights of the time series features and the weights of the spatial posture features to form a multimodal feature vector. The multimodal feature vector is then subjected to dimensionality reduction to obtain the detection action time series.

7. The multimodal motion posture dynamic assessment method using IMU and structured light features as claimed in claim 6, characterized in that: The step of optimizing the obtained detection action time sequence to obtain an optimized detection action time sequence comprises: Obtaining a sequence spacing threshold from a preset motion posture database; The intervals of each sequence in the detection action time series are compared with the sequence interval threshold. If the interval of a sequence in the detection action time series does not exceed the sequence interval threshold, the sequence segment is marked as a normal sequence segment. If a sequence spacing of the detection action time series exceeds the sequence spacing threshold, it indicates that the sequence segment is abnormal and the sequence segment is marked as an abnormal sequence segment. The abnormal sequence segment is optimized according to the length of the continuous segment to obtain the optimized detection action time series.

8. The multimodal motion posture dynamic assessment method using IMU and structured light features as claimed in claim 7, characterized in that: The step of optimizing the abnormal sequence segments according to the length of the continuous segments to obtain the optimized detection action time sequence comprises: Obtaining a segment length threshold from a preset motion posture database; The lengths of each continuous segment of the abnormal sequence segment are compared with the segment length threshold. If the length of a continuous segment of the abnormal sequence segment does not exceed the segment length threshold, the continuous segment is marked as the first abnormal sequence segment, and the first abnormal sequence segment is corrected according to the characteristics of the adjacent sequence segments of the continuous segment; If the length of a continuous segment of the abnormal sequence segment exceeds the segment length threshold, the continuous segment is marked as a second abnormal sequence segment, and the second abnormal sequence segment is optimized using the detection action time series information to obtain an optimized detection action time series.

9. The multi-modal motion posture dynamic assessment method based on IMU and structured light features as claimed in claim 1, characterized in that: The step of evaluating the eligibility of the multimodal motion posture based on the core motion parameters obtained from the optimized detection action time series includes: The core motion parameters include the coordinates of key points of the human body, the trunk posture angle and the angles of key joints of the human body, and the trunk posture angle includes pitch angle, roll angle and yaw angle; The change in the center of gravity position is obtained based on the preset mass ratio of each key point of the human body and the coordinates of each key point of the human body before and after the exercise cycle; The change in rotation was obtained based on the trunk posture angles during the exercise cycle before and after training; According to the key joint angles of the human body in the movement cycle before and after training, the change in posture accuracy is obtained; The motion posture change is obtained according to the center of gravity position change, the rotation change and the posture accuracy change; Obtain a motion posture change threshold from a preset motion posture database, compare the motion posture change amount with the motion posture change threshold, and if the motion posture change amount exceeds the motion posture change threshold and the core motion parameter is within the preset standard core motion parameter range, then mark the learning effect as valid. Evaluate the eligibility of the effective learning effect multimodal motion posture based on the position of each human body key point and the standard key point position; Otherwise, the learning effect is marked as invalid.

10. The multi-modal motion posture dynamic assessment method based on IMU and structured light features according to claim 9, characterized in that: The step of evaluating the eligibility of the effective learning effect multimodal motion posture according to the positions of the key points of each human body and the standard key point positions includes: According to the positions of the key points of each human body in the learning group and the standard positions of the key points of each human body, the posture position difference is obtained; Obtain the posture position difference threshold from the preset motion posture database, compare the posture position difference with the posture position difference threshold, if the posture position difference does not exceed the posture position difference threshold, the motion posture is recorded as qualified, if the posture position difference exceeds the posture position difference threshold, the motion posture is recorded as unqualified.

Images