Ba Duan Jin training and evaluation system based on 3D human posture estimation from monocular video

Abstract

The present invention provides a Ba Duan Jin training evaluation system based on monocular video 3D human posture estimation, which belongs to the field of artificial intelligence and computer vision technology, including a hardware support subsystem for realizing hardware support for the acquisition, transmission and processing of training videos; a 3D human posture motion capture subsystem, which is responsible for extracting 3D human key points from monocular videos; an action difference recognition and feedback subsystem, which realizes the recognition and feedback of action deviations by calculating the key point differences between the user and the standard action; an exercise evaluation and conclusion report subsystem, which scientifically evaluates the whole set of continuous actions completed by the user and generates a personalized evaluation report. Compared with the prior art, the present invention can realize scientific Ba Duan Jin training without manual intervention, which not only improves the popularity of Ba Duan Jin exercises, but also effectively promotes traditional culture.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and computer vision, and in particular relates to an Eight-Section Brocade training and evaluation system based on monocular video 3D human body posture estimation. Background Art

[0002] Ba Duan Jin is an ancient health-preserving Qigong. As Ba Duan Jin's movements are gentle, beginners are prone to deviations in posture, angle, rhythm, etc. when mastering its movements, and often find it difficult to achieve the expected practice results. Irregular movements will not only reduce the fitness and health-preserving effects of Ba Duan Jin, but in severe cases will also lead to muscle strain, joint injury, and spinal health problems. The current Ba Duan Jin learning method mainly relies on video or text guidance, lacks real-time feedback and posture correction functions, making it difficult for users to get immediate personalized guidance when practicing.

[0003] Therefore, there is an urgent need for an intelligent training guidance and evaluation system to provide users with timely and convenient personalized posture correction suggestions, so that they can achieve effective Ba Duan Jin practice guidance at home, in the community and other places. With the rapid development of artificial intelligence and computer vision technology, these technologies have broad application potential in the field of human motion recognition and evaluation. Combining computer vision technology with Ba Duan Jin training guidance provides a good development direction for improving the accuracy, scientificity and popularity of Ba Duan Jin practice. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a Ba Duan Jin training evaluation system based on monocular video 3D human posture estimation, which solves the problems of inaccurate posture evaluation and lack of real-time correction feedback in Ba Duan Jin training.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is: a Ba Duan Jin training and evaluation system based on monocular video 3D human posture estimation, comprising:

[0006] The 3D human posture motion capture subsystem is used to receive the monocular video of the user's practice and extract the coordinates of the key joints of the human body and the key nodes of the face in 3D space through the 3D human posture estimation pre-training model;

[0007] The action difference recognition and feedback subsystem is used to calculate the action standardization judgment parameters according to the coordinates in the 3D space by setting the standard judgment rules of the key actions, and compare the user's action differences according to the action standardization judgment parameters, and feed back the non-standard actions to the user end, wherein the action standardization judgment parameters are used as the standard judgment rules for static key actions;

[0008] The exercise evaluation and conclusion report subsystem is used to evaluate the user's entire set of continuous movements, combine the static key movement difference comparison results to obtain the scoring results, and generate an evaluation report based on the scoring results to complete the evaluation of Ba Duan Jin training.

[0009] Furthermore, the Ba Duan Jin training evaluation system further includes a hardware support subsystem; the hardware support subsystem includes:

[0010] The camera is used to collect monocular video during the user's training process; the small computer is used to compress and transmit the collected monocular video in real time and receive feedback data; the communication module is used to transmit the user's video data and return the action evaluation results; the remote server is used to calculate the difference between the user's action and the standard action, generate an evaluation report, and feed it back to the user end.

[0011] The beneficial effects of the present invention are as follows: the hardware support subsystem in the present invention is used to realize the hardware support for the acquisition, transmission and processing of training videos; the 3D human posture motion capture subsystem is responsible for extracting 3D human key points from monocular videos; the action difference recognition and feedback subsystem realizes the recognition and feedback of action deviations by calculating the key point differences between the user and the standard action; the exercise evaluation and conclusion report subsystem scientifically evaluates the whole set of continuous actions completed by the user and generates a personalized evaluation report. Compared with the prior art, the present invention can realize scientific Ba Duan Jin training without manual intervention, which not only improves the popularity of Ba Duan Jin exercises, but also effectively promotes traditional culture.

[0012] Furthermore, the monocular video of the user's practice is received, and the coordinates of the key joints of the human body and the key nodes of the face in 3D space are extracted through the 3D human posture estimation pre-training model, which is specifically as follows:

[0013] Receive the monocular video from the user, use the YOLOv8 model to correct the user's position, and compress the image;

[0014] After position correction and compression processing, the coordinates of the key joints of the human body and the key facial nodes in 3D space are extracted through the 3D human posture estimation pre-training model.

[0015] The beneficial effects of the above further solution are: the above processing can effectively improve the accuracy and processing efficiency of video acquisition. Position correction ensures the correct positioning of the user in the video, avoiding posture estimation errors caused by improper camera angles or users deviating from the center of the picture. Compression processing effectively reduces the storage and transmission pressure of video data, and improves the system's performance in real time and efficiency.

[0016] Furthermore, the standard judgment rules include:

[0017] Action standardization decision parameter of the first rule :

[0018]

[0019] in, represents the normal vector of plane ABC, represents the horizontal plane normal vector;

[0020] Action standardization decision parameter of the second rule and :

[0021]

[0022]

[0023]

[0024] in, Representing a collection The mean absolute error of all points, set It is a set of key points of wrist joint, elbow joint, shoulder joint, hip joint, knee joint and ankle joint in three-dimensional space. Representing a collection Middle The key point coordinates, Represents the set by the least squares method The coordinates of a point on the fitted line obtained by fitting all the points in Represents the direction vector of the fitted line The coordinates of

[0025] The action standardization judgment parameters of the third and seventh rules are :

[0026]

[0027] in, represents the forearm vector, represents the palm vector;

[0028] The action standardization judgment parameters of the fourth rule, the fifth rule, the thirteenth rule and the fourteenth rule are and :

[0029]

[0030]

[0031] in, represents the body vector, represents the thigh vector, represents the calf vector;

[0032] Action Standardization Judgment Parameters of Rule 6 :

[0033]

[0034] in, Represents the body plane The normal vector of represents the arm vector;

[0035] The action standardization judgment parameters of the eighth and twelfth rules are and :

[0036]

[0037]

[0038] in, Indicates the center of both ears To the middle of both eyes The vector of Indicates the midpoint of both eyes To the tip of the chin A vector of

[0039] The action standardization judgment parameters of the ninth and eleventh rules are and :

[0040]

[0041]

[0042] in, Normal vectors representing the wrist, elbow, shoulder, hip, knee, and ankle joints fitted to the same plane;

[0043] The sum of the vector moduli in the tenth rule is :

[0044]

[0045] in, , , , and Both represent vectors projected onto the horizontal plane;

[0046] Action Standardization Decision Parameters of Rule 15 , the geometric center point of each symmetrical joint is required At heel level:

[0047]

[0048]

[0049]

[0050]

[0051] in, , , , Both indicate the range of the heel in the horizontal plane. The set of geometric centers representing symmetric joints The point coordinates of They represent the x-axis coordinate points of the heels, They represent the z-axis coordinate points of the heel, It means to find the maximum value. Indicates finding the minimum value.

[0052] The beneficial effect of the above further scheme is that the present invention, through the above design, quantitatively analyzes human body movements, which not only provides efficient data support for subsequent movement difference analysis and evaluation, but also can further quantify the degree of deviation of user movements through standardized parameters.

[0053] Furthermore, the expression of the action difference is as follows:

[0054]

[0055] in, represents the action difference, k represents the number of eigenvalues ​​selected to measure the action, and They represent the kth feature value of the i-th frame in the standard action sequence and the j-th frame in the user action, respectively.

[0056] The beneficial effect of the above further scheme is that the present invention can effectively align the continuous actions of the user and the standard action sequence that are inconsistent in time through the above design. The present invention is based on the dynamic time warping algorithm DTW. The system calculates the difference between the user action sequence and the standard action sequence, and matches similar action frames at different times together, thereby solving the problem of misjudgment caused by different action completion times.

[0057] Furthermore, the expression of the scoring result is as follows:

[0058]

[0059]

[0060]

[0061]

[0062] in, Indicates the rating results. represents the static key action score, represents the dynamic connection action score, Indicates user Frame action and standard action The alignment cost of the frame, Indicates user Frame action and standard action The alignment cost of the frame, Indicates user Frame action and standard action The alignment cost of the frame, Indicates user Frame action and standard action The alignment cost of the frame, Indicates the number of parameters for actions that the user has not performed. Indicates action differences.

[0063] The beneficial effect of the above further solution is: a comprehensive evaluation of user actions is achieved, taking into account both the accuracy of static key actions and the fluency of dynamic continuous actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is the first schematic diagram of the key frame in the first paragraph.

[0065] Figure 2 This is the second schematic diagram of the key frame in the first paragraph.

[0066] Figure 3 This is the first schematic diagram of the key frame in the second paragraph.

[0067] Figure 4 This is the second schematic diagram of the key frame in the second paragraph.

[0068] Figure 5 This is a schematic diagram of the key frames in the third paragraph.

[0069] Figure 6 This is a schematic diagram of the key frames in the fourth paragraph.

[0070] Figure 7 This is a schematic diagram of the key frames in the fifth paragraph.

[0071] Figure 8 This is a schematic diagram of the key frames in the seventh paragraph.

[0072] Fig. 9 This is a schematic diagram of the key frames in the eighth paragraph.

[0073] Fig.10 Schematic diagram of 133 key points of the human body.

[0074] Fig.11 This is a schematic diagram of error correction for the second rule.

[0075] Fig.12 This is the error correction diagram of the fourth rule.

[0076] Fig.13 It is a system block diagram of the present invention.

[0077] Explanation of the reference numerals: 0—nose; 1—left eye; 2—right eye; 3—left ear; 4—right ear; 5—left shoulder; 6—right shoulder; 7—left elbow; 8—right elbow; 9—left wrist; 10—right wrist; 11—left hip; 12—right hip; 13—left knee; 14—right knee; 15—left ankle; 16—right ankle; 17—left big toe; 18—left little toe; 19—left heel; 20—right big toe ; 21—right little toe; 22—right heel; 31—chin; 91—base of left palm; 95—tip of left thumb; 99—tip of left index finger; 103—tip of left middle finger; 107—tip of left ring finger; 111—tip of left little finger; 112—base of right palm; 116—tip of right thumb; 120—tip of right index finger; 124—tip of right middle finger; 128—tip of right ring finger; 132—tip of right little finger. DETAILED DESCRIPTION

[0078] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0079] Example

[0080] like Fig.13 As shown, the present invention provides a Ba Duan Jin training evaluation system based on monocular video 3D human body posture estimation, which includes:

[0081] The 3D human posture motion capture subsystem is used to receive the monocular video of the user's training and extract the coordinates of the key joints and facial key nodes of the human body in 3D space through the 3D human posture estimation pre-training model. Specifically, it receives the monocular video of the user's training and uses the YOLOv8 model to correct the user's position and compress the image; after position correction and compression, the coordinates of the key joints and facial key nodes of the human body in 3D space are extracted through the 3D human posture estimation pre-training model;

[0082] The action difference recognition and feedback subsystem is used to calculate the action standardization judgment parameters according to the coordinates in the 3D space by setting the standard judgment rules of the key actions, and compare the user's action differences according to the action standardization judgment parameters, and feed back the non-standard actions to the user end, wherein the action standardization judgment parameters are used as the standard judgment rules for static key actions;

[0083] The exercise evaluation and conclusion report subsystem is used to evaluate the user's entire set of continuous movements, combine the static key movement difference comparison results to obtain the scoring results, and generate an evaluation report based on the scoring results to complete the evaluation of Ba Duan Jin training.

[0084] In this embodiment, the Ba Duan Jin training evaluation system further includes a hardware support subsystem;

[0085] The hardware support subsystem includes: a camera for collecting monocular video during the user's training process; a small computer for real-time compression and transmission of the collected monocular video and receiving feedback data; a communication module for transmitting the user's video data and the returned action evaluation results; a remote server for calculating the difference between the user's action and the standard action, generating an evaluation report, and feeding it back to the user end.

[0086] In this embodiment, the present invention aims to provide a convenient and professional Ba Duan Jin exercise guidance system, which is dedicated to meeting the needs of users to perform standardized training anytime and anywhere, and includes the following four subsystems:

[0087] 1. Hardware support subsystem: including camera, small computer, communication module and remote server. The camera is used to collect videos of users during practice; the small computer is used to compress and transmit the collected videos in real time and receive feedback data; the communication module is used to efficiently transmit the user's video data and the returned action evaluation results; the remote server is responsible for calculating the difference between the user's action and the standard action, generating a detailed evaluation report, and feeding it back to the user end. The hardware support subsystem provides users with necessary equipment support, including smartphones or smart TVs (with cameras) and other terminal devices with Internet communication functions, mainly used for collecting action data and receiving feedback information. The server is equipped with a remote server to provide basic support for the core calculation and algorithm deployment of the remaining subsystems.

[0088] In this embodiment, the standard demonstration of Ba Duan Jin is downloaded, and the video is divided into eight sections according to the Ba Duan Jin movements. The video is then imported into the 3D human posture motion capture system in sequence. 133 key nodes are automatically extracted through the 3D human posture estimation pre-training model RTMW3D (Real-Time Multi-person 3D Whole-body pose estimation models), such as Fig.10 As shown, there are 48 upper limb joints, 12 lower limb joints and 73 relatively fixed facial key points. These key nodes are connected in sequence to form the human body's skeletal structure or facial contour. The action feature parameters are extracted by calculating the angles between the limbs in the key frames and the distances between the joints and the specified plane. Based on the above parameters, fifteen action specification standards are obtained, and the values ​​calculated by the standard actions are used as the thresholds for action judgment. In addition, in the 3D human posture motion capture system, the side view of the standard demonstration action is extracted and fed back to the user, which makes up for the deficiency that the standard Ba Duan Jin only provides a frontal perspective demonstration.

[0089] In this embodiment, the user watches the standard Ba Duan Jin frontal movement and the extracted side movement demonstration through the display device for follow-up practice. The camera collects the user image in real time and corrects the position of the user in the video screen. The YOLOv8 model is used to quickly detect the human body bounding box to ensure that the user's action is in the middle of the screen. According to the voice prompt, the user completes the following actions in sequence, "raise both hands above the head", "raise both hands horizontally, move a large step to the right", "raise both hands horizontally, move a large step to the left". After the human body bounding box detection is passed, the system starts to record the video of the user's follow-up practice process. The recorded video is processed by a small computer device, including compression to a specified size (384×288) to optimize storage and transmission performance. Subsequently, the compressed video data is transmitted to the remote server for further analysis and archiving. The user's follow-up video extracts 133 key points in the 3D human posture motion capture system through the RTMW3D model (Real-Time Multi-person 3D Whole-body poseestimation models), including key points of the upper limbs, lower limbs and face.

[0090] 2. 3D human posture motion capture subsystem: Receive user monocular video, detect the human body bounding box through computer vision algorithm, and extract the coordinates of the key bone nodes and facial key nodes of the human body in 3D space. This subsystem can accurately capture and locate the key points of the human body for subsequent posture analysis and difference comparison.

[0091] In this embodiment, the 3D human posture motion capture subsystem is deployed on the server side, which is responsible for extracting the three-dimensional posture key points and bone structure information of the human body from the practice video. The RTMW3D model (Real-Time Multi-person3D Whole-body pose estimation models) is used to analyze each frame of video data and extract the position coordinates of 133 key points (including face, torso, limbs, hands, feet, etc.). These key points are used to calculate standard action parameters, such as angles and distances, to provide data support for action standardization analysis. The specific calculation rules are as follows:

[0092] First, assume that the three-dimensional space coordinates of the key points are as follows: Shoulder joint A: left shoulder joint ( ) and right shoulder joint ( ); Elbow joint B( )or( );Wrist joint C( )or( ); the tip of the middle finger is D( )or( ); Tip of thumb ( )or( ); index finger tip ( )or( ); Ring finger tip ( )or( ); Little finger tip ( )or( ); Hip joint E: left hip joint ( ) and right hip joint ( );Knee joint F( )or( ); Ankle joint G( )or( ); right ear ( ); left ear ( The midpoint of the line connecting the two ears is ; Right eye ( ); left eye ( The midpoint of the line connecting the two eyes is ; Chin tip ( ); the left heel is ( ); the left big toe is ( ), right big toe ( ), right heel ( ).

[0093] First paragraph:

[0094] First rule: If Figure 1 As shown in the figure, the forearm and the upper arm must be kept in the same horizontal plane. To achieve standardized action judgment. According to the three-dimensional spatial coordinates of the key points, the vector of the upper arm is constructed. and forearm vector . Define plane ABC and the normal vector of plane ABC Normal vector to the horizontal plane Angle , the specific calculation is as follows:

[0095]

[0096]

[0097]

[0098]

[0099]

[0100] in, represents the normal vector of plane ABC, Represents the horizontal plane normal vector.

[0101] The action standardization judgment parameter is , When it is close to 0°, it means that the upper arm and the forearm are closer to the same horizontal position, the action is more standard, and the standard action is extracted. , the threshold value of this action standardization is defined as the angle If the angle exceeds this range, the action is considered to deviate from the standard.

[0102] The second rule, such as Figure 2 As shown in , the arms, torso and lower limbs must be in a straight line, and the straight line must be perpendicular to the horizontal plane. To achieve action standardization judgment, assume that the set , the set P contains the key points of the wrist joint, elbow joint, shoulder joint, hip joint, knee joint and ankle joint in three-dimensional space. All points of the set in three-dimensional space are fitted by the least squares method to obtain a fitting straight line, which makes the distance from all points to the straight line the shortest. The expression of the fitting straight line is:

[0103]

[0104] in, represents a point on the fitted line, Represents the direction vector of the fitted line The coordinates of .

[0105] Sum the difference between the distance from all points in the set P to the straight line and the mean absolute error MAE :

[0106]

[0107]

[0108] The direction vector of the line Normal vector to the horizontal plane The angle is :

[0109]

[0110] in, Representing a collection Middle The coordinates of key points.

[0111] The action standard judgment parameters are ,when When it is close to 0, it means that the arms, torso and lower limbs are closer to a straight line. The action standard judgment parameters are , when the action standard judgment parameter is When it is close to 0°, it means that the posture of the arms, torso and lower limbs tends to be perpendicular to the ground. , the threshold value of this action standardization is defined as If the range is exceeded, the action is considered to be deviating from the standard. Fig.11 The following is a schematic diagram of error correction for this rule.

[0112] The third rule: Figure 2 As shown in the figure, at the base of the palm, the forearm and the palm are at 90°. To achieve standardized action judgment, the forearm vector is constructed based on the three-dimensional spatial coordinates of the key points. , palm vector , forearm vector With palm vector Angle , the specific calculation is as follows:

[0113]

[0114]

[0115] The action standardization judgment parameter is , When it is close to 90°, it means that the forearm and palm are closer to the vertical position, the action is more standard, and the standard action is extracted. , the threshold value of this action standardization is defined as the angle If the angle exceeds this range, it is considered that the action deviates from the standard.

[0116] Second paragraph:

[0117] Rule 4: If Figure 3 As shown in the figure, the body is straight and the knees are slightly bent. The focus of this action is on the trunk and lower limbs to achieve standardized judgment of the action. According to the three-dimensional spatial coordinates of the key points, the body vector is constructed. , thigh vector , Calf vector , the angle between the torso and the thigh , the angle between the thigh and calf , the specific calculation is as follows:

[0118]

[0119]

[0120]

[0121]

[0122]

[0123] The action standardization judgment parameter is and . Standard action extraction , the threshold value of this action standardization is defined as the angle If the angle exceeds this range, the action is considered to be deviated from the standard. Fig.12 The following is a schematic diagram of error correction for Rule 4.

[0124] Rule 5: If Figure 4 As shown, the torso is centered and the knees are slightly bent (same as the fourth rule). To achieve standardized judgment of the movement, calculate the angle between the torso and the thigh. , the angle between the thigh and calf The calculation method is the same as the fourth rule. The action standardization judgment parameter is and , action standardization judgment parameters for standard action extraction , the threshold value of this action standardization is defined as the angle If the angle exceeds this range, it is considered that the action deviates from the standard.

[0125] Rule 6: If Figure 4 As shown in the figure, the straightened arm must be perpendicular to the normal vector of the body plane. To achieve standardized judgment of the action. Construct the arm vector , the vector from the left shoulder joint to the right side , the vector from the right shoulder joint to the right hip joint , Body Plane The normal vector , the angle between the arm vector and the body plane normal vector The specific calculation is as follows:

[0126]

[0127]

[0128]

[0129] The action standardization judgment parameter is , standard action extraction , the threshold value of this action standardization is defined as If the angle exceeds this range, it is considered that the action deviates from the standard.

[0130] Rule 7: If Figure 4 As shown, the straightened arm must be perpendicular to the palm (same as the third rule). To achieve standardized judgment of the action, calculate the angle between the arm and the palm , the calculation method is the same as the third rule. The action standardization judgment parameter is , standard action extraction , the threshold value of this action standardization is defined as the angle If the angle exceeds this range, it is considered that the action deviates from the standard.

[0131] Rule 8: If Figure 4 As shown, the head rotates only in yaw, to the side of the outstretched hand, without pitch or roll rotation. To the middle of both eyes The vector is , which is related to the arm vector The angle is , the midpoint of the line connecting the two eyes To the tip of the chin The vector is , which is the same as the horizontal plane normal vector Angle The specific calculation is as follows:

[0132]

[0133]

[0134]

[0135]

[0136] The action standardization judgment parameter is and , When it is close to 0°, it means that the head has rotated yaw to a position closer to parallel with the shoulder. When it is close to 0°, it means that the head has no pitch rotation and roll rotation, and the standard action extraction , , the threshold value of this action standardization is defined as If the angle exceeds this range, it is considered that the action deviates from the standard.

[0137] Paragraph 3:

[0138] Rule 9: If Figure 5As shown, the human body remains upright, and the wrist, elbow, shoulder, hip, knee and ankle joints should be in the same plane. In order to achieve standardized judgment of movements, a plane is obtained so that the sum of the distances from the wrist, elbow, shoulder, hip, knee and ankle joints to the plane is minimized. Assume that the plane equation is:

[0139]

[0140] in, , , represents the coordinates of any point in space, and d represents the constant term of the plane.

[0141] Using the least squares method, fit the 12 key points in set P to the plane. Represents the normal vector of the plane , calculate the normal vector and the horizontal plane normal vector Angle , the distances of all points to the plane :

[0142]

[0143]

[0144] in, The point belongs to the set P, the set .

[0145] The action standardization judgment parameter is , . When it is close to 0°, it means that the angle between the body and the ground is closer to vertical. When it is close to 0, it means that all key points are in the same plane. Standard action extraction , the threshold value of this action standardization is defined as If it exceeds this range, it is considered that the action deviates from the standard.

[0146] Rule 10: If Figure 5 As shown in the figure, the raised fingertips are aligned vertically with the shoulder joint. , index finger tip , middle finger tip , ring finger tip , Little finger tip and shoulder joint Vector projected on the horizontal plane , , , and The sum of all vector moduli is , the specific calculation is as follows:

[0147]

[0148]

[0149]

[0150]

[0151]

[0152]

[0153] The action standardization judgment parameter is . When it is close to 0, it means that the fingertips and shoulders are nearly aligned in the vertical direction, and the standard action extraction ,The threshold value of this action standardization is defined as, If it exceeds this range, it is considered that the action deviates from the standard.

[0154] Paragraph 4:

[0155] Rule 11: If Figure 6 As shown, the body should be in the same plane (same as the ninth rule). To achieve standardized judgment of the action, the sum of the distances from the key points to the fitted plane is calculated. , the angle between the plane and the horizontal plane normal vector , the calculation method is the same as the ninth rule. The action standardization judgment parameter is and , standard action extraction , ,The threshold value of this action standardization is defined as, If it exceeds this range, it is considered that the action deviates from the standard.

[0156] Rule 12: If Figure 6 As shown, the head only makes yaw rotation, without pitch rotation and roll rotation (same as the eighth rule). To achieve standardized action judgment, calculate the angle between the vector from the midpoint of the two ears to the midpoint of the two eyes and the arm vector. , the angle between the vector from the midpoint of the line between the two eyes to the tip of the chin and the normal vector of the horizontal plane . Calculation method rule 8, action standardization judgment parameter is , . Standard action extraction , The threshold value for this action standardization is defined as the angle , If the angle exceeds this range, the action is considered to deviate from the standard.

[0157] Paragraph 5:

[0158] Rule 13: If Figure 7 As shown, the body is bent and the knees are kept bent (the fourth rule). To achieve standardized judgment of the action, the angle between the torso and the thigh is calculated. , the angle between the thigh and calf , the calculation method is the same as the fourth rule. The action standardization judgment parameter is and , standard action extraction , , the threshold value of this action standardization is defined as the angle If the angle exceeds this range, it is considered that the action deviates from the standard.

[0159] Paragraph 7:

[0160] Rule 14: If Figure 8 As shown, the body is centered and the knees are slightly bent (same as the fourth rule). To achieve standardized judgment of the movement, calculate the angle between the torso and the thigh. , the angle between the thigh and calf The calculation method is the same as the fourth rule. The action standardization judgment parameter is and , standard action extraction , the threshold value of this action standardization is defined as the angle If the angle exceeds this range, the action is considered to deviate from the standard.

[0161] Paragraph 8:

[0162] Rule 15: If Fig. 9 As shown, the toes are pointed, the heels are together, the heels are raised and the body is kept upright. In order to achieve standardized judgment of the movements, the left heel is , the right heel is , left big toe , right big toe and the geometric center set (Geometric center of two knee joints , the geometric center of the two hip joints , the geometric center of the two shoulder joints , the geometric center of the two ears , the geometric center of the two elbow joints , the geometric center of the two wrist joints , the geometric center of the two heels ). Construct the heel vector in three-dimensional space and toe vector , calculate the distance between the heels , Toe distance , the distance between the heel and the toes and the height of the heel lift , the specific calculation is as follows:

[0163]

[0164]

[0165]

[0166]

[0167]

[0168]

[0169]

[0170]

[0171]

[0172]

[0173]

[0174]

[0175]

[0176] Heel level range:

[0177]

[0178]

[0179] Judgment condition: Each geometric center point is required In scope:

[0180]

[0181] in, , , , Both indicate the range of the heel in the horizontal plane. The set of geometric centers representing symmetric joints The point coordinates of They represent the x-axis coordinate points of the heels, They represent the z-axis coordinate points of the heel, It means to find the maximum value. Indicates finding the minimum value.

[0182] The action standardization judgment parameter is .when When it is close to 0, it means that the heel and toes are close together. When it is greater than 0, it means the heel is in the lifted state. When it is equal to 0, it means that the body has no obvious deviation. Standard action extraction , , The normalized threshold for this action is defined as If it exceeds this range, it is considered that the action deviates from the standard.

[0183] 3. Action difference recognition and feedback subsystem: The 3D coordinates of key nodes extracted by the 3D human posture capture subsystem are used to calculate the differences in rotation angles and bending angles between the user and the standard demonstration in specific actions, generate real-time feedback data based on the calculation results, and transmit it to the user end.

[0184] In this embodiment, the action difference identification and feedback subsystem is deployed on the server side to analyze the difference between the user action and the standard action, and provide visual feedback to help the user adjust the action in time and improve the practice effect. After receiving the standard action judgment threshold and user action judgment parameters generated by the 3D human posture motion capture subsystem, this subsystem determines whether the standardized judgment parameters of the user action exceed the set threshold through comparative analysis. If the deviation of a key point or action angle exceeds the standard threshold, this subsystem will record the key point information involved in the deviation and feed it back to the user side. The user side will mark the bones corresponding to the deviation key points in red, so that the user can intuitively perceive the specific location and degree of the action problem.

[0185] 4. Exercise evaluation and conclusion report subsystem: After the user completes the entire set of Ba Duan Jin exercises, this subsystem performs dynamic time regularization analysis on the coordinates of the user's 3D key bone nodes and the coordinates of the standard movements, scores the entire set of continuous movements, and generates a comprehensive final score combined with the scores of key static movements, and finally forms a complete training conclusion report. The report includes the user's non-standard movements and the corresponding bone joints, static scores, dynamic scores, and final scores.

[0186] The exercise evaluation and conclusion reporting subsystem is deployed on the server side. The system combines the irregular action results reported in the action difference identification and feedback subsystem to score and analyze the user's overall action. The core function of this system is to comprehensively evaluate static key actions and dynamic continuous actions. Since Ba Duan Jin Health Qigong emphasizes the coordination of breathing and movement, and the difference in breathing length among people leads to different speeds of movements. In this subsystem, a dynamic time warping algorithm is introduced to deal with the problem of inconsistent movement speed. The dynamic time warping algorithm can ignore the difference in movement completion speed through nonlinear time alignment. The minimum alignment cost of the user's action sequence and the standard action sequence is calculated to evaluate the difference between the user's action and the standard action. In feature selection, the arm vector is extracted With body vector Angle (left) and (Right). Body vector Vector with lower limbs Angle (left) and (Right) As the main feature of action evaluation. Suppose the standard action set , where n represents the number of frames of the standard action video. User action sequence set , where m represents the number of frames of the user action video. The four features of the i-th frame in sequence X and the j-th frame in Y difference The definition is as follows:

[0187]

[0188] in, represents the action difference, k represents the number of eigenvalues ​​selected to measure the action, and They represent the kth feature value of the i-th frame in the standard action sequence and the j-th frame in the user action, respectively.

[0189]

[0190] in, Indicates user Frame action and standard action The alignment cost of the frame, Indicates user Frame action and standard action The alignment cost of the frame, Indicates user Frame action and standard action The alignment cost of the frame, Indicates user Frame action and standard action Alignment cost of frames. Alignment cost The larger the alignment cost is, the greater the distance between the two sequences is, that is, the lower the similarity between the user action and the standard action is; conversely, the smaller the alignment cost is, the higher the action similarity is, thus achieving quantitative scoring of user actions.

[0191] There are 28 static key action standardized parameters in total, and the action parameters that the user has not made standard are recorded as M. Static key action score:

[0192]

[0193] Dynamic continuous action score:

[0194]

[0195] Comprehensive score:

[0196]

[0197] Finally, an evaluation report is generated based on the scoring results. The report includes non-compliant movements and the corresponding bone joints, static scores, dynamic scores, and final scores.

[0198] In this embodiment, in the action difference identification and feedback subsystem, the dynamic time warping algorithm is used to calculate the alignment cost between the user action sequence and the standard key action sequence, so as to identify which frame of static key action the user is currently performing. At the same time, the difference between the user action and the standard action in the key action is compared. If the user's action characteristics deviate from the preset threshold range, the action is determined to be non-compliant, and the key nodes of the human body whose deviation exceeds the threshold are recorded. After receiving the feedback results, the small computer will present these key nodes and the corresponding bone connection lines in red on the front-end display device, and intuitively prompt the user to correct the action deviation. After completing the entire exercise, the exercise evaluation and conclusion report subsystem combines the non-standard action results reported in the action difference identification and feedback subsystem to conduct a comprehensive evaluation of static key actions and dynamic continuous actions. In order to ignore the difference in action speed between individuals, the dynamic time warping algorithm is used to evaluate the continuous action using dynamic continuous actions. The generated report includes the user's non-standard actions and the joints and bones that need to be focused on, the static key action scores, the dynamic continuous action scores and the final scores.

Claims

1. A Ba Duan Jin training and evaluation system based on monocular video 3D human posture estimation, characterized in that: include: The 3D human posture motion capture subsystem is used to receive the monocular video of the user's practice and extract the coordinates of the key joints of the human body and the key nodes of the face in 3D space through the 3D human posture estimation pre-training model; The action difference recognition and feedback subsystem is used to calculate the action standardization judgment parameters according to the coordinates in the 3D space by setting the standard judgment rules of the key actions, and compare the user's action differences according to the action standardization judgment parameters, and feed back the non-standard actions to the user end, wherein the action standardization judgment parameters are used as the standard judgment rules for static key actions; The exercise evaluation and conclusion report subsystem is used to evaluate the user's entire set of continuous movements, combine the static key movement difference comparison results to obtain the scoring results, and generate an evaluation report based on the scoring results to complete the evaluation of Ba Duan Jin training.

2. The Ba Duan Jin training and evaluation system based on monocular video 3D human posture estimation according to claim 1 is characterized in that: The Ba Duan Jin training and evaluation system also includes a hardware support subsystem; The hardware support subsystem includes: Camera, used to collect monocular video during user practice; A small computer used to compress and transmit the collected monocular video in real time and receive feedback data; Communication module, used to transmit the user's video data and the returned action evaluation results; The remote server is used to calculate the difference between the user's action and the standard action, generate an evaluation report, and feed it back to the user end.

3. The Ba Duan Jin training and evaluation system based on monocular video 3D human posture estimation according to claim 1 is characterized in that: The receiving of the monocular video of the user's follow-up training and the extraction of the coordinates of the key joints of the human body and the key nodes of the face in the 3D space through the 3D human posture estimation pre-training model are specifically as follows: Receive the monocular video from the user, use the YOLOv8 model to correct the user's position, and compress the image; After position correction and compression processing, the coordinates of the key joints of the human body and the key facial nodes in 3D space are extracted through the 3D human posture estimation pre-training model.

4. The Ba Duan Jin training and evaluation system based on monocular video 3D human posture estimation according to claim 1, characterized in that: The standard judgment rules include: Action standardization decision parameter of the first rule : in, represents the normal vector of plane ABC, represents the horizontal plane normal vector; Action standardization decision parameter of the second rule and : in, Representing a collection The mean absolute error of all points, set It is a set of key points of wrist joint, elbow joint, shoulder joint, hip joint, knee joint and ankle joint in three-dimensional space. Representing a collection Middle The key point coordinates, Represents the set by the least squares method The coordinates of a point on the fitted line obtained by fitting all the points in Represents the direction vector of the fitted line The coordinates of The action standardization judgment parameters of the third and seventh rules are : in, represents the forearm vector, represents the palm vector; The action standardization judgment parameters of the fourth rule, the fifth rule, the thirteenth rule and the fourteenth rule are and : in, represents the body vector, represents the thigh vector, represents the calf vector; Action Standardization Judgment Parameters of Rule 6 : in, Represents the body plane The normal vector of represents the arm vector; The action standardization judgment parameters of the eighth and twelfth rules are and : in, Indicates the center of both ears To the middle of both eyes The vector of Indicates the midpoint of both eyes To the tip of the chin A vector of The action standardization judgment parameters of the ninth and eleventh rules are and : in, Normal vectors representing the wrist, elbow, shoulder, hip, knee, and ankle joints fitted to the same plane; The sum of the vector moduli in the tenth rule is : in, , , , and Both represent vectors projected onto the horizontal plane; Action Standardization Decision Parameters of Rule 15 , the geometric center point of each symmetrical joint is required At heel level: in, , , , Both indicate the range of the heel in the horizontal plane. The set of geometric centers representing symmetric joints The point coordinates of They represent the x-axis coordinate points of the heels, They represent the z-axis coordinate points of the heel, It means to find the maximum value. Indicates finding the minimum value.

5. The Ba Duan Jin training and evaluation system based on monocular video 3D human posture estimation according to claim 1, characterized in that: The expression of the action difference is as follows: in, represents the action difference, k represents the number of eigenvalues ​​selected to measure the action, and They represent the kth feature value of the i-th frame in the standard action sequence and the j-th frame in the user action, respectively.

6. The Ba Duan Jin training and evaluation system based on monocular video 3D human posture estimation according to claim 1, characterized in that: The expression of the scoring result is as follows: in, Indicates the rating results. represents the static key action score, represents the dynamic connection action score, Indicates user Frame action and standard action The alignment cost of the frame, Indicates user Frame action and standard action The alignment cost of the frame, Indicates user Frame action and standard action The alignment cost of the frame, Indicates user Frame action and standard action The alignment cost of the frame, Indicates the number of parameters for actions that the user has not performed. Indicates action differences.

Images