Method and system for intelligent evaluation of motor skill action posture

Abstract

The application discloses a motion skill action posture intelligent evaluation method and system, adopts a multi-view camera array to synchronously collect a motion process video, extracts a human body skeleton two-dimensional key point through an improved algorithm, combines multi-view geometric constraints and Kalman filtering to reconstruct a three-dimensional human body posture sequence, constructs a standard action posture template library, realizes posture alignment of an evaluation object and a standard template through body proportion normalization processing, completes action stage division and phased posture evaluation through a dynamic time warping algorithm, calculates posture deviation and generates a graded deviation diagnosis report, and finally realizes superimposed visual feedback of evaluation postures and standard postures through augmented reality technology. The application can realize high-precision, quantifiable and phased intelligent evaluation of motion postures, eliminate the interference of individual morphological differences on evaluation results, accurately locate action deviation nodes, and provide intuitive interactive correction guidance.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and intelligent information processing technology, specifically to a method and system for intelligent evaluation of motor skills and postures. Background Technology

[0002] With the development of mass fitness and the professionalization of competitive sports, the standardized and precise assessment of movement postures has become a core element in improving training effectiveness and preventing sports injuries. Current mainstream movement posture assessment methods have the following significant shortcomings:

[0003] 1) Manual assessment relies on the coach's visual observation and experience judgment, which has the problems of strong subjectivity, low quantification, inability to capture the details of high-speed movements, and poor consistency of assessment results, making it difficult to achieve accurate movement correction;

[0004] 2) The pose assessment scheme based on monocular vision is susceptible to the effects of human self-occlusion and the limitations of shooting angle. The accuracy of two-dimensional coordinates in restoring three-dimensional pose is insufficient, and it cannot accurately represent the action deviation in the spatial dimension.

[0005] 3) Existing technologies do not fully consider the differences in body proportions between the evaluation object and the standard template. Directly comparing postures will introduce evaluation errors caused by individual morphology, resulting in distorted evaluation results.

[0006] 4) Most solutions only score the complete movement as a whole, and cannot break down and evaluate the core stages of the movement such as preparation, force exertion, braking, and recovery. It is difficult to locate the key links of movement deviation and cannot provide targeted correction guidance.

[0007] 5) The feedback of the assessment results is limited in form, mainly consisting of numerical reports, lacking intuitive visualization. Trainees find it difficult to quickly understand the difference between their own posture and the standard movement, thus limiting the improvement of training efficiency.

[0008] Therefore, there is an urgent need for an intelligent evaluation method and system for motor skills and postures to achieve a high-precision, quantifiable, phased, and visualized intelligent evaluation solution for the entire process. Summary of the Invention

[0009] This invention provides an intelligent evaluation method and system for motor skill movements and postures, which solves the problems of strong subjectivity in manual evaluation, low accuracy of three-dimensional posture reconstruction, interference of evaluation results by individual morphological differences, inability to perform phased positioning deviations, and unintuitive feedback in the existing technology.

[0010] To achieve the above objectives, the present invention provides the following technical solution: an intelligent evaluation method for motor skill posture, comprising the following steps:

[0011] S1. Acquire video of the motion process and obtain multi-view motion image sequences;

[0012] S2. Perform human skeleton key point detection on the multi-view motion image sequence, extract the coordinates of two-dimensional skeleton key points, and reconstruct the three-dimensional human posture sequence through multi-view geometric constraints.

[0013] S3. Construct a standard motion posture template library to store keyframe posture parameters and motion trajectory envelopes for various motion actions;

[0014] S4. Perform body proportion normalization processing on the three-dimensional human posture sequence and the corresponding standard posture in the standard action posture template library to achieve posture alignment.

[0015] S5. Divide the aligned complete motion into multiple stages, evaluate the posture of each stage, calculate the posture deviation value, and generate a deviation diagnosis report for each stage based on the posture deviation value.

[0016] S6. Using augmented reality technology, the three-dimensional human posture sequence is superimposed and displayed with the corresponding standard posture skeleton to complete the intelligent assessment of motor skill posture.

[0017] Preferably, in step S1, a multi-view camera array is used to synchronously acquire motion video. The number of cameras deployed in the array is 3-10, the shooting angle between adjacent cameras is 30°-60°, the acquired video resolution is 1080P or higher, and the frame rate is 25-60fps. During the acquisition process, the frame synchronization of all cameras is achieved through a time synchronization module, and the synchronization error does not exceed 5ms.

[0018] Preferably, in step S2, the human skeleton key point detection adopts an improved human pose estimation algorithm that integrates HRNet and Transformer, and the specific steps include:

[0019] S21. Preprocess the multi-view motion image sequence, including image denoising, size normalization and color gamut correction, wherein the size of the image after size normalization is [H×W], and the values ​​of H and W are in the range of 320-1024 pixels.

[0020] S22. Input the preprocessed image into the improved HRNet network, extract image features through multi-scale feature fusion, and then capture global features through the Transformer encoder to output the two-dimensional coordinates of G human skeleton key points. Where i = 1, 2, ..., G, corresponding to the key skeletal nodes of the head, neck, shoulder, elbow, wrist, hip, knee, and ankle, respectively. The horizontal coordinates of the key points The vertical coordinates of the key points;

[0021] S23. Reconstruct a three-dimensional human pose sequence based on multi-view geometric constraints, and calculate the three-dimensional coordinates of each skeletal key point using triangulation. The specific formula is as follows:

[0022] ,

[0023] in, The two-dimensional coordinates of the i-th key point acquired by the first camera. The two-dimensional coordinates of the i-th key point acquired by the second camera; , These are the focal lengths of the two cameras, ranging from 5 to 50 mm. , , where A and B are the horizontal and vertical coordinates of the image center of the first camera, respectively; B is the baseline distance between the two cameras, ranging from 0.5 to 5m. , These are the z-axis components of the unit direction vectors from the 1st and 2nd cameras to the i-th key point, respectively.

[0024] S24. The reconstructed 3D coordinates are smoothed, and noise is eliminated using the Kalman filter algorithm to obtain a continuous 3D human pose sequence. The state equation and observation equation of the Kalman filter are as follows:

[0025] ,

[0026] ,

[0027] in, Let A be the 3D pose state vector of the k-th frame, containing the 3D coordinates of all skeletal keypoints; A is the state transition matrix, with a value of the identity matrix. The noise is the process noise, which follows a mean of 0 and a variance of . Gaussian distribution, The value range is 1e-6-1e-4; is the observation vector of the k-th frame, i.e., the three-dimensional coordinates obtained by triangulation; P is the observation matrix, with a value of the identity matrix; To observe the noise, it follows a mean of 0 and a variance of . Gaussian distribution, The value range is 1e-5-1e-3.

[0028] Preferably, in step S3, the method for constructing the standard action pose template library includes:

[0029] S31: Select four major categories of sports movements: track and field, gymnastics, ball games, and martial arts. Select several typical movements for each category to construct a movement classification system.

[0030] S32: Collect multi-view videos of professional athletes performing various typical movements, extract three-dimensional human posture sequences, and select sequences whose movement completion reaches a preset threshold as standard posture samples.

[0031] S33: Extract keyframes from standard pose samples. Use the K-means clustering algorithm to cluster the 3D pose sequence. The number of clusters is 5-20. The frame corresponding to each cluster center is used as the keyframe of the action.

[0032] S34: Extract the pose parameters of each keyframe, including the angles of each skeletal joint and the ratio of limb segment lengths. Simultaneously, fit the motion trajectory using a Bézier curve to obtain the motion trajectory envelope. The formula for the Bézier curve is:

[0033] ,

[0034] Where B(t) is the coordinate of the trajectory point at time t; n is the order of the Bézier curve, which ranges from 3 to 5. It is the number of combinations; The coordinates of the trajectory control points are determined by the 3D coordinates of the skeletal keypoints in the keyframe; t is the normalized time parameter.

[0035] S35: Store the keyframe pose parameters, motion trajectory envelopes, and motion classification information of various actions into the standard motion pose template library, establish an indexing mechanism, and achieve fast retrieval.

[0036] Preferably, in step S4, the body proportion normalization process uses a normalization algorithm based on human height and limb segment length, and the specific steps include:

[0037] S41: Calculate the height H of the subject being evaluated, determined by the three-dimensional distance between the key points at the top of the head and the key points on the soles of the feet, using the following formula:

[0038] ,

[0039] in, The three-dimensional coordinates of the key points at the top of the head. The three-dimensional coordinates of key points on the sole of the foot;

[0040] S42: Calculate the length of each limb segment Where j = 1, 2, ..., 12, and the limb segments include neck-shoulder, shoulder-elbow, elbow-wrist, hip-shoulder, hip-knee, and knee-ankle (one segment on each side), the formula is:

[0041] ,

[0042] in, , These are the three-dimensional coordinates of the key points of the bones at both ends of the corresponding limb segment;

[0043] S43: Calculate the length of each limb segment Normalized to relative length The formula is:

[0044] ,

[0045] S44: Calculate the relative angles of each bone joint. For k=1,2,...,10, the joints include the neck, shoulder, elbow, wrist, hip, and knee (one on each side). The vector dot product method is used for calculation, and the formula is:

[0046] ,

[0047] in, , These are the vectors corresponding to the limb segments at both ends of the joint. For vector dot product, , They are vectors , The modulus length;

[0048] S45: Assess the relative length of the object. and relative angle Align with the normalized parameters of the corresponding actions in the standard action posture template library to eliminate the influence of individual morphological differences.

[0049] Preferably, in step S5, the motion segmentation adopts a motion phase division algorithm based on Dynamic Time Warping (DTW), dividing the complete motion into four phases: preparation phase, exertion phase, braking phase, and recovery phase. Specific steps include:

[0050] S51. Convert the aligned 3D human pose sequence into a pose feature vector sequence. Where T is the number of frames in the attitude sequence. Let be the pose feature vector of frame t, which includes the relative angles of all joints and the relative lengths of all limb segments;

[0051] S52. Calculate the first-order difference of the attitude feature vector sequence:

[0052] ,

[0053] The formula reflecting the rate of attitude change is:

[0054] ,

[0055] Where M is the dimension of the pose feature vector. Let m be the m-th component of the pose feature vector of frame t;

[0056] S53, Based on the attitude change rate The thresholds for dividing the four phases were determined, with the attitude change rate thresholds for the preparation and recovery phases being: The attitude change rate thresholds during the power generation and braking phases are: ,and , The value range is 0.01-0.05. The value range is 0.08-0.2;

[0057] S54. The DTW algorithm is used to calculate the similarity between the object's pose feature sequence and the standard action pose feature sequence. The DTW distance formula is:

[0058] ;

[0059] Where d(i,j) is the Euclidean distance between the pose feature vectors of the evaluation object in frame i and the standard action in frame j, and DTW(i,j) is the cumulative DTW distance between the previous i frames and the previous j frames.

[0060] S55. Based on the DTW distance and attitude change rate, each stage is scored. The scoring formula is as follows:

[0061] ,

[0062] in, Let be the score for the s-th stage, where s = 1, 2, 3, 4, corresponding to the preparation, exertion, braking, and recovery stages, respectively. Let dW distance be the distance in stage s. This is an adjustment coefficient, with a value range of 0.1-0.5;

[0063] S56. Calculate the attitude deviation value E using a weighted summation method. The formula is as follows:

[0064] ,

[0065] in, Let be the weight of the s-th stage. During the initial exertion phase, the weight ranges from 0.3 to 0.5, while during the remaining phases, the weight ranges from 0.1 to 0.3.

[0066] Preferably, in step S5, the method for generating the deviation diagnosis report includes:

[0067] 1) For each stage, extract the top 3-5 joints with the largest attitude deviation values ​​as critical deviation joints;

[0068] 2) Calculate the difference between the actual angle and the standard angle of the critical deviation joint. The formula is:

[0069] ,

[0070] in, To assess the actual relative angles of the key deviation joints of the object, The relative angles of the joints corresponding to the standard movements;

[0071] 3) Based on the difference The magnitude of the deviation is used to classify the deviation level as minor deviation ( ), moderate deviation (5°≤ <15°) and severe deviation ( ≥15°);

[0072] 4) By combining the key deviation joints, deviation levels and corresponding movement stages, a deviation diagnosis report is generated, which clarifies the location and degree of deviation and provides preliminary correction suggestions.

[0073] Preferably, in step S6, the specific method for augmented reality overlay display includes:

[0074] S61: The reconstructed 3D human pose sequence and the corresponding standard poses in the standard action pose template library are converted into 2D image coordinates through perspective projection. The projection formula is:

[0075] ,

[0076] Where (x', y') are the coordinates of the projected two-dimensional image, (X, Y, Z) are the three-dimensional coordinates, and f is the projection focal length. , The coordinates of the image center;

[0077] S62: Use different colors to mark the object's pose skeleton and the standard pose skeleton. The object's pose skeleton is red and the standard pose skeleton is green. The skeleton line width is 2-5 pixels.

[0078] S63: Display the superimposed posture image on the terminal device in real time, and play the motion process video simultaneously. It supports pause and rewind operations, making it easy for users to observe the details of posture deviation.

[0079] S64: Associate the deviation diagnosis report with the overlay image. Clicking on the deviation joint will display the deviation details and correction suggestions for that joint.

[0080] The present invention also provides an intelligent evaluation system for motor skill movement posture, the system comprising:

[0081] The video acquisition module is used to deploy a multi-view camera array, synchronously acquire video of the motion process, obtain multi-view motion image sequences, and achieve camera frame synchronization;

[0082] The pose reconstruction module is used to detect human skeleton key points in multi-view motion image sequences, extract the coordinates of two-dimensional skeleton key points, and reconstruct three-dimensional human pose sequences through multi-view geometric constraints and Kalman filtering algorithm.

[0083] The template library construction module is used to build a standard movement posture template library, collect the three-dimensional posture sequences of professional athletes' standard movements, extract key frame posture parameters and motion trajectory envelopes, and establish an indexing mechanism;

[0084] The pose alignment module is used to normalize the body proportions of the 3D human pose sequence and the standard pose, calculate the relative length and relative angle, and eliminate individual morphological differences.

[0085] The phase evaluation module is used to divide the complete motion into four phases using the dynamic time warping algorithm, calculate the DTW distance and score for each phase, and determine the posture deviation value.

[0086] The deviation diagnosis module is used to extract key deviation joints, calculate joint angle differences, classify deviation levels, and generate deviation diagnosis reports.

[0087] The AR feedback module is used to perform perspective projection and overlay display of the evaluation object's pose skeleton and the standard pose skeleton, and associate it with the deviation diagnosis report to achieve real-time visual feedback.

[0088] The control module is electrically connected to each of the above modules and is used to control the coordinated operation of each module, receive the output data of each module, and complete the scheduling and management of the overall evaluation process.

[0089] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0090] 1. This invention achieves highly accurate and robust three-dimensional posture temporal reconstruction of human motion; it adopts a multi-view synchronous acquisition architecture, integrates multi-scale features and a human keypoint detection algorithm with global context awareness, and combines three-dimensional coordinate calculation with multi-view geometric constraints and temporal Kalman filtering smoothing processing to stably capture the human skeleton posture details during high-speed motion and output a continuous, low-noise three-dimensional human posture sequence, thus building a high-precision spatial temporal data foundation for motion posture assessment.

[0091] 2. This invention achieves non-discriminatory alignment and refined quantitative evaluation of movement postures throughout the entire process; based on the relative parameter normalization processing of human height and limb segment characteristics, it realizes non-discriminatory posture alignment between different evaluation objects and standard action templates; combined with the similarity matching algorithm of action sequence stage division and dynamic time warping, it completes the staged quantitative scoring and weighted deviation calculation of the entire action process, realizing the full-dimensional accurate quantitative representation of movement posture from the whole to the part, ensuring the consistency and credibility of the evaluation results.

[0092] 3. This invention achieves targeted diagnosis and intuitive interactive feedback of motion posture deviation; by locating key deviation joints and quantifying deviation levels, it generates targeted motion deviation diagnosis results; combined with augmented reality technology's on-screen posture overlay rendering and data association mechanism, it can intuitively present the spatial difference between the evaluation posture and the standard posture, while realizing interactive query of deviation details and correction guidance, which greatly improves the efficiency and feasibility of motion posture correction. Attached Figure Description

[0093] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0094] In the attached diagram:

[0095] Figure 1 This is a flowchart of the intelligent evaluation method for motor skill movement posture of the present invention;

[0096] Figure 2 This is a schematic diagram of the modules of the intelligent evaluation system for motor skills and postures of the present invention. Detailed Implementation

[0097] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0098] This invention provides a method and system for intelligent evaluation of motor skill movements and postures. Through a complete technical solution including multi-view visual acquisition, high-precision three-dimensional posture reconstruction, individual normalized posture alignment, phased dynamic evaluation, graded deviation diagnosis, and AR visualization feedback, it achieves objective, accurate, efficient, and intelligent evaluation of motor skill movements and postures.

[0099] Example 1

[0100] like Figure 1 As shown, this embodiment discloses an intelligent evaluation method for motor skill movement posture. As a preferred implementation, the evaluation object is a basketball standing one-handed shoulder shot. The specific implementation steps are as follows:

[0101] S1. A multi-view camera array is constructed using four industrial cameras. The shooting angle between adjacent cameras is 45°, and they are evenly deployed in a ring around the shooting area. The camera focal length is set to 16mm, and the baseline distance B is set to 2m. The video resolution is set to 4K, the frame rate is 30fps, and the frame synchronization of the four cameras is achieved through the PTP precision time synchronization protocol with a synchronization error ≤2ms. The entire process of the evaluation object completing the standing one-handed shoulder shot is captured to obtain a multi-view motion image sequence.

[0102] S2. Perform human skeleton key point detection and 3D pose reconstruction on the acquired multi-view motion image sequence. The specific steps are as follows:

[0103] S21. Preprocess the multi-view motion image sequence, use Gaussian filtering to denoise the image, normalize the image size to 512×512 pixels, and correct the color gamut to the sRGB standard color gamut.

[0104] S22. Input the preprocessed image into the improved HRNet network, extract high-resolution image features through multi-scale feature fusion, and then capture global contextual features through the Transformer encoder to output the two-dimensional coordinates of 17 human skeleton key points.

[0105] Among them, 17 key points correspond to the COCO Human Body Key Point Standard, namely: top of head, nose, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle, left foot sole, and right foot sole, covering the core skeletal nodes of the head, neck, shoulders, elbows, wrists, hips, knees, and ankles;

[0106] S23. Based on Zhang Zhengyou's calibration method, the camera's intrinsic and extrinsic parameters are calibrated in advance. Through multi-view geometric constraints, the three-dimensional coordinates (X, Y, F, Z) of each skeleton key point are calculated using triangulation. i ,Y i Z i The calculation formula is consistent with the aforementioned technical solution;

[0107] S24. The reconstructed 3D coordinates are smoothed using the Kalman filter algorithm. Both the state transition matrix A and the observation matrix P are set to identity matrices. The process noise variance σ... w 2 Set to 5e-5, observation noise variance σ v 2 Setting it to 5e-4 eliminates reconstruction noise and jitter, resulting in a continuous and smooth three-dimensional human pose sequence.

[0108] S3. In this embodiment, the template library collects 20 sets of standard action samples from national first-level basketball players for the stationary one-handed shoulder shot. The completion rate of the actions is ≥95% after review by national-level coaches. The standard posture sequence is clustered using the K-means clustering algorithm, with 10 clusters. The keyframes corresponding to the cluster centers are extracted. The angles of each skeletal joint and the ratio of limb segment lengths in the keyframes are extracted as posture parameters. The motion trajectories of the wrist, elbow, and shoulder in the shooting action are fitted using a 4th-order Bézier curve to obtain the motion trajectory envelope. The above parameters and action classification information are stored in the template library, and a B+ tree index is established to achieve millisecond-level fast retrieval of actions.

[0109] S4. Normalize the body proportions of the 3D human posture sequence of the evaluation object and the standard shooting posture in the template library to achieve posture alignment. The specific steps are as follows:

[0110] S41, through formula The height H of the subject being evaluated is calculated and determined by the three-dimensional Euclidean distance between the key points on the top of the head and the key points on the soles of the feet.

[0111] S42, through formula Calculate the length L of the 12 core limb segments. 12 This includes left neck-left shoulder, right neck-right shoulder, left shoulder-left elbow, right shoulder-right elbow, left elbow-left wrist, right elbow-right wrist, left hip-left shoulder, right hip-right shoulder, left hip-left knee, right hip-right knee, left knee-left ankle, and right knee-right ankle;

[0112] S43. Normalize the absolute length of each limb segment to its relative length L relative to the height H. j Eliminate the impact of height differences;

[0113] S44. Calculate the relative angle θ of the 10 core joints using the vector dot product method. k This includes the left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, and right knee;

[0114] S45. Rigidly align the relative length and relative angle parameters of the evaluation object with the normalized parameters of the standard shooting motion to eliminate the interference of spatial pose and individual shape differences on subsequent evaluation.

[0115] S5. Divide the aligned shooting motion sequence into four stages: preparation, power generation, braking, and follow-through, and conduct a stage-by-stage posture evaluation. The specific steps are as follows:

[0116] S51. Convert the aligned 3D human pose sequence into a pose feature vector sequence V=[v1,v2,...,v...]. TEach frame's feature vector contains 12 relative lengths of limb segments and 10 relative angles of joints, with a feature dimension M=22;

[0117] S52, through formula Calculate the first-order difference ΔV of the pose feature vector sequence to obtain the pose change rate for each frame:

[0118] ;

[0119] S53. Set the stage division threshold, and the change rate threshold Δv for the preparation and recovery stages. th1 =0.03, the threshold value of the rate of change Δv between the acceleration and braking phases. th2 =0.1, based on the threshold, the shooting motion is completely divided into four stages;

[0120] S54. The DTW algorithm is used to calculate the similarity between the object's posture feature sequence and the standard shooting motion feature sequence. Sakoe-Chiba constraints are added to reduce the computational load, and the DTW distance is obtained for each stage. s ;

[0121] S55. Set the adjustment coefficient α=0.3, and calculate the action score for each stage using the scoring formula: ;

[0122] S56. Set the weights for each stage: w2=0.4 for the exertion stage, w1=0.2 for the preparation stage, w3=0.25 for the braking stage, and w4=0.15 for the recovery stage. The total weights are 1. Calculate the overall attitude deviation using the weighted summation formula. .

[0123] Simultaneously, a deviation diagnosis report is generated based on the calculated attitude deviation values. The specific steps are as follows:

[0124] 1) For each movement phase, extract the top 3 joints with the largest deviation values ​​as key deviation joints, and extract the right elbow, right shoulder, and right wrist as key deviation joints during the force exertion phase.

[0125] 2) Calculate the difference between the actual angle and the standard angle of the critical deviation joint:

[0126] ,

[0127] Let the actual value of the right elbow flexion angle be 90°, the standard value be 120°, and the difference be Δθ = 30°;

[0128] 3) Based on the difference, the deviation level is divided into: Δθ<5° is a slight deviation, 5°≤Δθ<15° is a moderate deviation, and Δθ≥15° is a severe deviation. The above-mentioned right elbow angle difference of 30° is judged as a severe deviation.

[0129] 4) Combine key deviation joints, deviation levels and movement stages to generate a standardized diagnostic report, clearly indicating that the right elbow flexion angle is seriously insufficient during the shooting power phase, and give targeted correction suggestions: conduct stationary ball holding and power generation imitation training, focusing on controlling the right elbow flexion angle to be kept in the range of 110°-130°, and strengthen muscle memory with slow motion replay.

[0130] S6. Using AR (Augmented Reality) technology to achieve gesture overlay display and visual feedback, the specific steps are as follows:

[0131] S61. Based on the pinhole camera model, the three-dimensional posture sequence of the evaluation object and the standard shooting posture sequence are converted into two-dimensional image coordinates through perspective projection, and the projection focal length is consistent with the focal length of the acquisition camera.

[0132] S62. Use red lines to draw the posture skeleton of the evaluation object, and green lines to draw the standard posture skeleton. Set the line width to 3 pixels, and highlight the joints with serious deviations in yellow.

[0133] S63. The superimposed posture image is displayed synchronously on the touch tablet terminal, and the original motion video is played synchronously. It supports frame-by-frame pause, slow motion, and rewind operations, making it easy for users to observe the posture deviation at each stage of the action.

[0134] S64. Link the deviation diagnosis report with the overlay image. When the user clicks on the highlighted deviation joint, the angle difference, deviation level and detailed correction suggestions of the joint will pop up, completing the intelligent assessment and feedback of the whole process.

[0135] Example 2

[0136] This embodiment discloses an intelligent evaluation system for motor skill movements and postures, used to implement the evaluation method in Embodiment 1, such as... Figure 2 As shown, the specific system architecture and implementation method are as follows:

[0137] This system includes a video acquisition module, a posture reconstruction module, a template library construction module, a posture alignment module, a stage evaluation module, a deviation diagnosis module, an AR feedback module, and a control module.

[0138] The video acquisition module includes four global shutter industrial cameras, a gigabit Ethernet switch, a PTP time synchronization server, and an industrial-grade control computer. The industrial cameras are deployed on adjustable tripods to allow for flexible adjustment of the shooting angle and baseline distance. They are connected to the control computer via gigabit Ethernet, and the PTP time synchronization server is connected to the switch to achieve microsecond-level time synchronization of all cameras with a synchronization error of ≤2ms, thus completing the synchronous acquisition of motion video and the output of image sequences.

[0139] The pose reconstruction module is deployed in an edge computing terminal equipped with an NVIDIA RTX 4060 GPU. It has a built-in preprocessing unit, an improved HRNet+Transformer key point detection unit, a triangulation 3D reconstruction unit, and a Kalman filter smoothing unit. The preprocessing unit completes image denoising, size normalization, and color gamut correction. The key point detection unit outputs the 2D coordinates of 17 skeletal key points. The 3D reconstruction unit calculates the 3D coordinates based on multi-view geometric constraints. The smoothing unit completes noise elimination and sequence smoothing. Finally, it outputs a continuous 3D human pose sequence.

[0140] The template library construction module is deployed on a cloud server and includes a motion classification unit, a standard sample collection unit, a keyframe extraction unit, a trajectory fitting unit, and an indexing unit. The motion classification unit constructs a classification system for four major categories of motions: track and field, gymnastics, ball games, and martial arts. The standard sample collection unit stores standard three-dimensional posture sequences of professional athletes. The keyframe extraction unit completes keyframe selection through K-means clustering. The trajectory fitting unit completes the construction of motion trajectory envelopes through Bézier curves. The indexing unit establishes a B+ tree index to achieve rapid retrieval and retrieval of standard motions.

[0141] The posture alignment module, stage evaluation module, and deviation diagnosis module are all deployed in the edge computing terminal. They respectively complete body proportion normalization and posture alignment, action stage division and stage scoring, deviation value calculation and diagnostic report generation. The core algorithm is completely consistent with the aforementioned technical solution, realizing real-time processing and output of evaluation data.

[0142] The AR feedback module includes a touch screen, AR glasses terminal, and an interactive processing unit. The interactive processing unit completes the perspective projection and overlay rendering of the three-dimensional posture, and the touch screen and AR glasses realize the real-time display of the overlay image. At the same time, it supports user interaction. By clicking on the deviated joint, the user can view the corresponding diagnostic details and correction suggestions.

[0143] The control module adopts an embedded ARM main control unit, which is electrically connected to each of the above modules through RS485 and Ethernet interfaces respectively. It has a built-in process scheduling unit, data interaction unit and exception handling unit to realize the coordinated work of each module, bidirectional data transmission and automated scheduling of the entire evaluation process, and ensure the stable operation of the system.

[0144] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent evaluation of motor skill movements and postures, characterized in that, Includes the following steps: S1. Acquire video of the motion process and obtain multi-view motion image sequences; S2. Detect key points of human skeleton in multi-view motion image sequence, extract the coordinates of two-dimensional skeleton key points, and reconstruct three-dimensional human pose sequence through multi-view geometric constraints. S3. Construct a standard motion posture template library to store keyframe posture parameters and motion trajectory envelopes for various motion actions; S4. Perform body proportion normalization processing on the 3D human posture sequence and the corresponding standard posture in the standard action posture template library to achieve posture alignment. S5. Divide the aligned complete motion into multiple stages, evaluate the posture of each stage, calculate the posture deviation value, and generate a deviation diagnosis report for each stage based on the posture deviation value. S6. Overlay the three-dimensional human posture sequence with the corresponding standard posture skeleton to complete the intelligent assessment of motor skill posture.

2. The intelligent evaluation method for motor skill movement posture according to claim 1, characterized in that: In step S1, a multi-view camera array is used to synchronously acquire motion video. During the acquisition process, frame synchronization of all cameras is achieved through a time synchronization module.

3. The intelligent evaluation method for motor skill posture according to claim 1, characterized in that: In step S2, the human skeleton keypoint detection adopts an improved human pose estimation algorithm that integrates HRNet and Transformer. The specific steps include: S21. Preprocess the multi-view motion image sequence, including image denoising, size normalization and color gamut correction; S22. Input the preprocessed image into the improved HRNet network, extract image features through multi-scale feature fusion, and then capture global features through the Transformer encoder to output the two-dimensional coordinates of G human skeleton key points. Where i = 1, 2, ..., G, corresponding to key skeletal nodes including the head, neck, shoulder, elbow, wrist, hip, knee, and ankle, respectively. The horizontal coordinates of the key points The vertical coordinates of the key points; S23. Reconstruct a three-dimensional human pose sequence based on multi-view geometric constraints, and calculate the three-dimensional coordinates of each skeletal key point using triangulation. The specific formula is as follows: ， in, The two-dimensional coordinates of the i-th key point acquired by the first camera. The two-dimensional coordinates of the i-th key point acquired by the second camera; , These are the focal lengths of the two cameras, respectively. , , where A and B are the horizontal and vertical coordinates of the image center of the first camera, respectively; B is the baseline distance between the two cameras. , These are the z-axis components of the unit direction vectors from the 1st and 2nd cameras to the i-th key point, respectively. S24. The reconstructed 3D coordinates are smoothed, and noise is eliminated using the Kalman filter algorithm to obtain a continuous 3D human pose sequence. The state equation and observation equation of the Kalman filter are as follows: ， ， in, Let A be the 3D pose state vector of the k-th frame, containing the 3D coordinates of all skeletal keypoints; A is the state transition matrix. The noise is the process noise, which follows a mean of 0 and a variance of . Gaussian distribution; Let be the observation vector of the k-th frame; P is the observation matrix; To observe noise.

4. The intelligent evaluation method for motor skill movement posture according to claim 1, characterized in that: In step S3, the method for constructing the standard motion pose template library includes: S31: Select four major categories of sports movements: track and field, gymnastics, ball games, and martial arts. Select several typical movements for each category to construct a movement classification system. S32: Collect multi-view videos of professional athletes performing various typical movements, extract three-dimensional human posture sequences, and select sequences whose movement completion reaches a threshold as standard posture samples. S33: Extract keyframes from standard pose samples, and use the K-means clustering algorithm to cluster the 3D pose sequence. The frame corresponding to each cluster center is used as the keyframe of the action. S34: Extract the pose parameters of each keyframe, including the angles of each skeletal joint and the ratio of limb segment lengths. Simultaneously, fit the motion trajectory using a Bézier curve to obtain the motion trajectory envelope. The formula for the Bézier curve is: ， Where B(t) is the coordinate of the trajectory point at time t; n is the order of the Bézier curve; It is the number of combinations; Here are the coordinates of the trajectory control points, and t is the normalized time parameter; S35: Store the keyframe pose parameters, motion trajectory envelopes, and motion classification information of various actions into the standard motion pose template library and establish an indexing mechanism.

5. The intelligent evaluation method for motor skill movement posture according to claim 1, characterized in that: In step S4, the body proportion normalization process uses a normalization algorithm based on human height and limb segment length. The specific steps include: S41: Calculate the height H of the subject being evaluated, determined by the three-dimensional distance between the key points at the top of the head and the key points on the soles of the feet, using the following formula: ， in, The three-dimensional coordinates of the key points at the top of the head. The three-dimensional coordinates of key points on the sole of the foot; S42: Calculate the length of each limb segment Where j = 1, 2, ..., 12, and the limb segments include neck-shoulder, shoulder-elbow, elbow-wrist, hip-shoulder, hip-knee, and knee-ankle, as shown in the formula: ， in, , These are the three-dimensional coordinates of the key points of the bones at both ends of the corresponding limb segment; S43: Calculate the length of each limb segment Normalized to relative length The formula is: ， S44: Calculate the relative angles of each bone joint. For k=1,2,...,10, the joints include the neck, shoulder, elbow, wrist, hip, and knee. The vector dot product method is used for calculation, and the formula is: ， in, , These are the vectors corresponding to the limb segments at both ends of the joint. For vector dot product, , They are vectors , The modulus length; S45: Assess the relative length of the object. and relative angle Align with the normalized parameters of the corresponding actions in the standard action pose template library.

6. The intelligent evaluation method for motor skill movement posture according to claim 1, characterized in that: In step S5, the motion segmentation adopts a motion phase division algorithm based on dynamic time warping, dividing the complete motion into four phases: preparation phase, force exertion phase, braking phase, and recovery phase. The specific steps include: S51. Convert the aligned 3D human pose sequence into a pose feature vector sequence. Where T is the number of frames in the attitude sequence. Let be the pose feature vector of frame t, which includes the relative angles of all joints and the relative lengths of all limb segments; S52. Calculate the first-order difference of the attitude feature vector sequence: ， The formula reflecting the rate of attitude change is: ， Where M is the dimension of the pose feature vector. Let m be the m-th component of the pose feature vector of frame t; S53, Based on the attitude change rate Determine the threshold values ​​for the four stages; S54. The DTW algorithm is used to calculate the similarity between the object's pose feature sequence and the standard action pose feature sequence. The DTW distance formula is: ； Where d(i,j) is the Euclidean distance between the pose feature vectors of the evaluation object in frame i and the standard action in frame j, and DTW(i,j) is the cumulative DTW distance between the previous i frames and the previous j frames. S55. Based on the DTW distance and attitude change rate, each stage is scored. The scoring formula is as follows: ， in, For the score of the s-th stage, Let dW distance be the distance in stage s. This is the adjustment coefficient; S56. Calculate the attitude deviation value E using a weighted summation method. The formula is as follows: ， in, Let be the weight of the s-th stage.

7. The intelligent evaluation method for motor skill movement posture according to claim 1, characterized in that: In step S5, the method for generating the deviation diagnosis report includes: 1) For each stage, extract the top 3-5 joints with the largest attitude deviation values ​​as critical deviation joints; 2) Calculate the difference between the actual angle and the standard angle of the critical deviation joint. The formula is: ， in, To assess the actual relative angles of the key deviation joints of the object, The relative angles of the joints corresponding to the standard movements; 3) Based on the difference The magnitude of the deviation is used to classify the deviation level into slight deviation, moderate deviation, and severe deviation; 4) By combining the key deviation joints, deviation levels and corresponding movement stages, a deviation diagnosis report is generated, which clarifies the location and degree of deviation and provides preliminary correction suggestions.

8. The intelligent evaluation method for motor skill movement posture according to claim 1, characterized in that: In step S6, the specific methods for augmented reality overlay display include: S61: The reconstructed 3D human pose sequence and the corresponding standard poses in the standard action pose template library are converted into 2D image coordinates through perspective projection. The projection formula is: ， Where (x', y') are the coordinates of the projected two-dimensional image, (X, Y, Z) are the three-dimensional coordinates, and f is the projection focal length. , The coordinates of the image center; S62: Use different color markings to evaluate the object's pose skeleton and the standard pose skeleton; S63: Display the superimposed posture image on the terminal device in real time, and simultaneously play the motion process video. S64: Associate the deviation diagnosis report with the overlay image.

9. A motor skill movement posture intelligent evaluation system, characterized in that, The system for implementing the intelligent evaluation method for motor skill movement postures according to any one of claims 1-8, the system comprising: The video acquisition module is used to deploy a multi-view camera array, synchronously acquire video of the motion process, obtain multi-view motion image sequences, and achieve camera frame synchronization; The pose reconstruction module is used to detect human skeleton key points in multi-view motion image sequences, extract the coordinates of two-dimensional skeleton key points, and reconstruct three-dimensional human pose sequences through multi-view geometric constraints and Kalman filtering algorithm. The template library construction module is used to build a standard movement posture template library, collect the three-dimensional posture sequences of professional athletes' standard movements, extract key frame posture parameters and motion trajectory envelopes, and establish an indexing mechanism; The pose alignment module is used to normalize the body proportions of a 3D human pose sequence and a standard pose, and to calculate the relative length and relative angle. The phase evaluation module is used to divide the complete motion into four phases using the dynamic time warping algorithm, calculate the DTW distance and score for each phase, and determine the posture deviation value. The deviation diagnosis module is used to extract key deviation joints, calculate joint angle differences, classify deviation levels, and generate deviation diagnosis reports. The AR feedback module is used to perform perspective projection and overlay display of the evaluation object's pose skeleton and the standard pose skeleton, and associate it with the deviation diagnosis report to achieve real-time visual feedback. The control module is electrically connected to each module and is used to control the coordinated work of each module, receive the output data of each module, and complete the scheduling and management of the overall evaluation process.

Images