Sports evaluation method, device, electronic equipment and storage medium

Abstract

The present application proposes a motion evaluation method, device, electronic device and storage medium, the method comprising: classifying the motion attributes of the image frames of the motion video of the test object, determining the motion attributes corresponding to each image frame; the motion attributes corresponding to the image frame represent the posture of the test object presented in the image frame; according to the motion attributes corresponding to each image frame, determining whether each group of actions of the test object during the motion is compliant, and obtaining the normative evaluation results of each group of actions; according to the normative evaluation results of each group of actions, determining the motion evaluation results of the corresponding test object. By adopting the technical solution of the present application, the image frames collected during the test can be analyzed for motion, determining whether the motion of the test object is compliant, and counting the actions according to the normative evaluation results of the actions, which is more accurate than manual evaluation, and not only performs quantitative evaluation, but also realizes normative evaluation of actions, improving the accuracy of motion evaluation and action evaluation.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and particularly to a motion evaluation method, device, electronic device, and storage medium. Background Art

[0002] People are paying more and more attention to physical health, and the awareness of national fitness is also continuously strengthened. In daily exercises, the most common sports items include sit-ups, push-ups, etc. When performing exercises, exercisers usually use counting methods to evaluate their exercises in order to calculate their exercise scores, thereby stimulating their exercise enthusiasm.

[0003] In the prior art, counting and evaluating exercise actions can be done by a counter observing the actions of the exerciser to count the number. However, manual counting is subjective and prone to errors, affecting the accuracy of the evaluation. It is also possible to install infrared receiving devices in the test area to achieve optoelectronic counting. However, optoelectronic counting is easily interfered by obstacles and cannot judge the accuracy of exercise actions, thus affecting the accuracy of the evaluation.

[0004] Therefore, how to improve the accuracy of exercise action evaluation is a technical problem that needs to be urgently solved by those skilled in the art. Summary of the Invention

[0005] Based on the above-mentioned defects and deficiencies of the prior art, this application proposes a motion evaluation method, device, electronic device, and storage medium, which can improve the accuracy of exercise action evaluation.

[0006] The first aspect of this application provides a motion evaluation method, including:

[0007] Classify the action attributes of the image frames of the exercise video of the test object to determine the action attributes corresponding to each image frame; the action attributes corresponding to the image frame represent the posture of the test object presented in the image frame.

[0008] According to the action attributes corresponding to each image frame, determine whether each group of actions of the test object during exercise is compliant, and obtain the evaluation result of the normativity of each group of actions.

[0009] According to the evaluation result of the normativity of each group of actions, determine the motion evaluation result corresponding to the test object.

[0010] The second aspect of this application provides a motion evaluation device, including:

[0011] An action attribute classification module, configured to classify the action attributes of the image frames of the exercise video of the test object to determine the action attributes corresponding to each image frame; the action attributes corresponding to the image frame represent the posture of the test object presented in the image frame.

[0012] An action compliance analysis module, configured to determine whether each group of actions during the movement of the test object is compliant according to the action attributes corresponding to each image frame, and obtain a normative evaluation result for each group of actions;

[0013] A motion evaluation module, configured to determine a motion evaluation result corresponding to the test object according to the normative evaluation result of each group of actions.

[0014] A third aspect of the present application provides an electronic device, including: a memory and a processor;

[0015] Wherein, the memory is connected to the processor and is used to store programs;

[0016] The processor is configured to implement the above-mentioned motion evaluation method by running the program in the memory.

[0017] A fourth aspect of the present application provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned motion evaluation method is implemented.

[0018] The motion evaluation method proposed in the present application classifies the action attributes of the image frames of the motion video of the test object to determine the action attributes corresponding to each image frame; the action attributes corresponding to the image frame represent the posture of the test object presented in the image frame; according to the action attributes corresponding to each image frame, it is determined whether each group of actions during the movement of the test object is compliant, and a normative evaluation result for each group of actions is obtained; according to the normative evaluation result of each group of actions, a motion evaluation result corresponding to the test object is determined. By adopting the technical solution of the present application, the action analysis of the image frames collected during the test can be performed to determine whether the actions of the test object are compliant, and the action count can be performed according to the normative evaluation result of the actions, which is more accurate than manual evaluation, and not only the quantity evaluation is performed, but also the action specification evaluation is realized, improving the accuracy of motion evaluation and action evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0020] Figure 1 is a schematic flowchart of a motion evaluation method provided by an embodiment of the present application;

[0021] Figure 2 is a schematic flowchart of the process of obtaining the bone point coordinates of the test object in each image frame provided by an embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of the processing flow of the comparison candidate object provided by the embodiment of the present application and the skeletal point norm;

[0023] Figure 4 It is a schematic diagram of the processing flow for determining the action attributes corresponding to each image frame provided by the embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of the processing flow for determining the normative evaluation results of each group of actions provided by the embodiment of the present application;

[0025] Figure 6 It is a schematic diagram of the processing flow for determining each image frame corresponding to each group of actions provided by the embodiment of the present application;

[0026] Figure 7 It is a curve graph of the change in body bending angle before correction provided by the embodiment of the present application;

[0027] Figure 8 It is a schematic diagram of the body bending angle when the body is not lying flat provided by the embodiment of the present application;

[0028] Figure 9 It is a schematic diagram of the body bending angle when the body is lying flat provided by the embodiment of the present application;

[0029] Figure 10 It is a curve graph of the corrected change in body bending angle provided by the embodiment of the present application;

[0030] Figure 11 It is a schematic diagram of the working process of the motion state machine provided by the embodiment of the present application;

[0031] Figure 12 It is a schematic diagram of the structure of a motion evaluation device provided by the embodiment of the present application;

[0032] Figure 13 It is a schematic diagram of the structure of an electronic device provided by the embodiment of the present application. Detailed implementation manners

[0033] The technical solution of the embodiment of the present application is applicable to the application scenario of motion evaluation. By adopting the technical solution of the embodiment of the present application, action counting can be performed according to the normative evaluation results of actions, which is more accurate than manual evaluation. Moreover, not only quantity evaluation is carried out, but also action norm evaluation is realized, improving the accuracy of motion evaluation and action evaluation.

[0034] In daily exercise, conducting exercise evaluation on the exercise object can stimulate the exercise enthusiasm of the exercise object and test the exercise results of the exercise object. For common and simple exercise items such as sit-ups and push-ups, in the prior art, exercise evaluation methods such as manual counting, photoelectric counting, or terminal IMU counting are usually adopted.

[0035] Among them, the exercise evaluation method of manual counting is that the counter observes whether the actions of the tester during the exercise process are standard. If a set of actions corresponding to one exercise cycle are all standard, the counter will count once. However, manual counting is time-consuming and laborious, and has strong subjectivity, is prone to cheating, and is not easy to detect minor violations, resulting in low accuracy and evaluation efficiency of exercise evaluation.

[0036] The exercise evaluation method of photoelectric counting is to count through sensors such as infrared. An infrared transmitter and an infrared receiver are installed on both sides of the test area, and whether there is occlusion is judged by the on / off of the infrared receiver, and counting is carried out in a cycle according to the infrared on / off state. For example, for the photoelectric counting of sit-ups, when the tester lies flat, the tester's body blocks the infrared sensor on the lower side, and the lower infrared is disconnected to judge that it is the lying flat state at this time; when the tester sits up, the tester's body blocks the infrared sensor on the upper side, and the upper infrared sensor is disconnected to judge that it is the sitting up state at this time; when the infrared signals are received by both infrared receivers, it is judged that it is the intermediate process state at this time. The cycle of each group of sit-up actions is determined through the cycle of the infrared on / off state, so as to realize the counting of sit-ups. However, infrared induction is easily interfered by occluders, thus affecting the counting, and infrared induction can only count through the sensed exercise cycle, but cannot judge whether each group of actions is compliant, resulting in low accuracy of exercise evaluation.

[0037] The exercise evaluation method of terminal IMU counting is to wear an IMU device (inertial measurement device) on the tester, such as an accelerometer, a gyroscope, etc., record the sensor data such as the acceleration (accelerometer) and / or angular velocity (gyroscope) of the tester, divide the periodically changing data into periods, obtain the exercise cycle, and thus perform cycle counting. This method also cannot perform action compliance detection, resulting in low accuracy of exercise evaluation.

[0038] In view of the above deficiencies of the prior art and the problem of low accuracy of exercise evaluation in reality, the inventor of the present application has conducted research and experiments, and proposed an exercise evaluation method, which can realize the evaluation of the action standardization of the test object, and perform action counting according to the evaluation result of the action standardization, not only performing quantity evaluation, but also realizing action standard evaluation, and improving the accuracy of exercise evaluation.

[0039] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0040] An embodiment of the present application provides a motion evaluation method. Refer to Figure 1 As shown, the method includes:

[0041] S101. Classify the action attributes of the image frames of the motion video of the test object to determine the action attributes corresponding to each image frame.

[0042] Specifically, for the test object undergoing motion evaluation, it is necessary to collect a video of the test object to obtain the motion video of the test object, extract the image frames from the motion video, and classify the action attributes of each extracted image frame, so as to analyze the posture of the test object presented in each image frame, that is, determine the action attributes corresponding to each image frame. Among them, in this embodiment, all image frames in the motion video can be extracted for action attribute classification, or the extraction time interval of the image frames can be set. For example, if the extraction time interval is set to 50 ms, then an image frame is extracted from the motion video every 50 ms. In this embodiment, it is preferably to use an RGB camera to collect the motion video of the test object.

[0043] In this embodiment, the action attribute corresponding to the image frame represents the posture of the test object presented in the image frame. For example, in the motion video collected during the sit-up motion evaluation, the posture of the test object presented in the image frame, that is, the action attribute may include at least one of the following: knee-touching state, knee-bending state, head-holding state, lying-flat state, and the orientation state of the test object, etc. In the motion video collected during the push-up motion evaluation, the posture of the test object presented in the image frame, that is, the action attribute may include at least one of the following: elbow-bending state, straight-arm state, back-straight state, and the orientation state of the test object, etc.

[0044] In this embodiment, first, it is necessary to determine the test object in the image frame of the motion video, select several target points from the test object in the image frame, and based on the positions of the target points, the posture of the test object can be analyzed, so as to realize the analysis of the action attributes of the test object. Among them, the target points are mainly selected at the positions of joints or parts with obvious changes when the test object performs various actions during the motion, so that through the changes of the target points in the test object in the motion video, the changes of the actions can be judged, and according to the positions of the target points in the image frame, the current posture of the test object can be judged.

[0045] Further, this step specifically includes:

[0046] First, perform human skeleton point detection on each image frame in the motion video of the test object to obtain the skeleton point coordinates of the test object in each image frame.

[0047] In this embodiment, the technology of human skeleton key point detection can be used to obtain human skeleton points from the image frames in the motion video and determine the coordinates of each human skeleton point, so as to obtain the skeleton point coordinates of the test object in the image frame. Among them, the detected skeleton points can include any of the following: top of the head, tip of the nose, left ear, right ear, chin, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left metacarpophalangeal joint, right metacarpophalangeal joint, left fingertip, right fingertip, left hip, right hip, left knee, right knee, left ankle, right ankle, left toe tip, right toe tip, left heel, right heel, etc.

[0048] Second, according to the skeleton point coordinates of the test object in each image frame, classify the action attributes of the test object in each image frame to determine the action attributes corresponding to each image frame.

[0049] After obtaining the skeleton point coordinates of the test object in each image frame, by analyzing the skeleton point coordinates of the test object, the posture presented by the test object can be determined, the action attribute classification can be realized, and the action attributes corresponding to each image frame can be obtained. For example, for the sit-up exercise, it can be analyzed whether the test object touches the knees, bends the knees, holds the head, lies flat, or faces left or right.

[0050] Among them, in the sit-up exercise, the skeleton point coordinates of the left elbow, right elbow, left knee, right knee, etc. can be collected to analyze the knee-touching state, the skeleton point coordinates of the left hip, right hip, left knee, right knee, left ankle, right ankle, etc. can be collected to analyze the knee-bending state, the skeleton point coordinates of the top of the head, tip of the nose, left ear, right ear, chin, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left metacarpophalangeal joint, right metacarpophalangeal joint, left fingertip, right fingertip, etc. can be collected to analyze the head-holding state, the skeleton point coordinates of the left shoulder, right shoulder, left hip, right hip, left ankle, right ankle, left toe tip, right toe tip, left heel, right heel, etc. can be collected to analyze the lying-flat state, and the skeleton point coordinates of the top of the head, left ankle, right ankle, left toe tip, right toe tip, left heel, right heel, etc. can be collected to analyze the facing state.

[0051] S102. According to the action attributes corresponding to each image frame, determine whether each group of actions of the test object during exercise is compliant, and obtain the evaluation result of the normativity of each group of actions.

[0052] Specifically, in this embodiment, the actions of the test object in each image frame can be analyzed periodically according to the bone point coordinates of the test object in each image frame. Each period corresponds to a set of actions, so that the image frames corresponding to each set of actions in the motion video of the test object can be analyzed. Then, according to the action attributes corresponding to each image frame, it is determined whether each set of actions complies with the regulations, so as to obtain the standardization evaluation result of each set of actions.

[0053] In the sit-up exercise, from the lying flat action to the sitting straight action, and then from the sitting straight action to the lying flat action is one cycle. To determine the standardization of the sit-up action, it is necessary to detect whether the knees are touched between the lying flat action and the sitting straight action, whether the head is held between the lying flat action and the knee-touching action, whether the knees are bent between the lying flat action and the knee-touching action, and whether the body is lying flat at the start of a set of actions. The non-compliance situations in each set of actions are recorded in the standardization evaluation result of each set of actions. For example, if the knee-touching state in the action attributes of each image frame between the lying flat action and the sitting straight action in a set of actions indicates that the knees are not touched, it is determined that the standardization evaluation result of this set of actions includes the non-compliance of not touching the knees. In addition, according to the analysis of the action attributes of each image frame in each set of actions, the standardization evaluation result may also include non-compliance of not bending the knees, non-compliance of not lying flat, non-compliance of not holding the head, etc.

[0054] S103. Determine the motion evaluation result corresponding to the test object according to the standardization evaluation result of each set of actions.

[0055] Specifically, in this embodiment, the motion of the test object can be counted according to the standardization evaluation result of each set of actions in the motion video, so as to determine the motion evaluation result of the test object according to the counting result and the standardization evaluation result of each set of actions.

[0056] Further, this step specifically includes: First, regard the actions without non-compliance in the standardization evaluation result as standard actions; Second, record the number of standard actions in the motion video of the test object, and use the number of standard actions and the standardization evaluation result of each set of actions as the motion evaluation result of the test object.

[0057] The motion evaluation includes timed motion evaluation and limit counting motion evaluation. Among them, the timed motion evaluation is to count the motion actions within a fixed time duration, and the limit counting motion evaluation is to record the maximum number of motion actions of the test object. That is, in the timed motion evaluation, the counting stops after the time is up, while in the limit counting motion evaluation, the counting stops after the test object stops moving. For the timed motion evaluation, only the actions within the timed range in the motion video need to be judged for compliance and counted, and for the limit counting motion evaluation, all actions in the motion video need to be judged for compliance and counted.

[0058] As can be seen from the above introduction, the motion evaluation method proposed in the embodiments of the present application classifies the action attributes of the image frames of the motion video of the test object to determine the action attributes corresponding to each image frame; the action attribute corresponding to the image frame represents the posture of the test object presented in the image frame; according to the action attributes corresponding to each image frame, it is determined whether each group of actions of the test object during motion is compliant, and a normative evaluation result for each group of actions is obtained; according to the normative evaluation result of each group of actions, the motion evaluation result corresponding to the test object is determined. By adopting the technical solution of this embodiment, the action analysis can be performed on the image frames collected during the test, it can be determined whether the actions of the test object are compliant, and the action counting can be performed according to the normative evaluation result of the action, which is more accurate than manual evaluation. Moreover, not only the quantity evaluation is performed, but also the action specification evaluation is realized, improving the accuracy of motion evaluation and action evaluation.

[0059] As an alternative implementation, as shown in Figure 2 In step S101 of the above, for each image frame in the motion video of the test object, human skeleton point detection is performed to obtain the skeleton point coordinates of the test object in each image frame, including the following steps:

[0060] S201. Perform human skeleton point detection on each image frame in the motion video of the test object to obtain the skeleton point coordinates of all candidate objects in each image frame.

[0061] Specifically, since other people besides the test object may be captured when shooting the motion video, when performing human skeleton point detection on the image frames containing other people besides the test object, the skeleton point coordinates of the test object and those of other people may be obtained, and it is impossible to determine which one is the test object, and thus the skeleton point coordinates of the test object cannot be determined. Therefore, in this embodiment, all objects in the image frame can be regarded as candidate objects, and then through the human skeleton point detection method, the skeleton point coordinates of each candidate object in the image frame are obtained.

[0062] S202. By comparing the skeleton point coordinates of each candidate object in each image frame with the skeleton point coordinates of the skeleton point norm corresponding to each image frame determined in advance, the skeleton point coordinates of the test object in each image frame are obtained.

[0063] Specifically, in this embodiment, the skeleton point norm corresponding to each image frame is determined in advance. Only when the similarity between the candidate object and the skeleton point norm is relatively high can it be indicated that the candidate object is the test object. Therefore, it is necessary to compare the skeleton point coordinates of each candidate object in the image frame with the skeleton point coordinates of the skeleton point norm corresponding to this image frame, determine the similarity between each candidate object and the skeleton point norm, and use the candidate object corresponding to the maximum similarity as the test object, so as to obtain the skeleton point coordinates of the test object.

[0064] Among them, the bone point coordinates of the bone point norm corresponding to each image frame are determined by performing a moving average calculation using the bone point coordinates of the norm image frame in the motion video. The norm image frame is an image frame in the image frames before the said image frame where the posture of the test object is the same as that of the bone point norm. In this embodiment, the bone point norm includes at least one of a lying flat norm, a sitting straight norm, and an intermediate norm. Among them, the sitting straight norm corresponding to the image frame is the moving average of the bone points of the test object in the image frame where the test object touches the knees and the body bending angle is the smallest among the image frames before the said image frame; the lying flat norm corresponding to the image frame is the moving average of the bone points of the test object in the image frame where the test object lies flat and the body bending angle is the largest among the image frames before the said image frame; the intermediate norm corresponding to the image frame is the moving average of the bone points of the test object in the image frame where the body bending angle of the test object is between 100° and 120° among the image frames before the said image frame.

[0065] When the similarity between the bone points of the test object in the previous image frame of the current image frame and the bone point norm is low (i.e., the posture of the test object is different from that of the bone point norm), the bone point coordinates of the bone point norm corresponding to the current image frame are the same as the bone point coordinates of the bone point norm corresponding to the previous image frame of the current image frame; when the similarity between the bone points of the test object in the previous image frame of the current image frame and the bone point norm is high (i.e., the posture of the test object is the same as that of the bone point norm), the moving average calculation formula for the bone point coordinates of the bone point norm corresponding to the current image frame is:

[0066] comm_pose1 = α * comm_pose0 + (1 - α) * pose

[0067] Among them, comm_pose1 represents the bone point coordinates of the bone point norm corresponding to the current image frame, pose represents the bone point coordinates of the previous image frame of the current image frame; comm_pose0 represents the bone point coordinates of the bone point norm corresponding to the previous image frame of the current image frame; α represents the moving coefficient.

[0068] In this embodiment, by setting the bone point norms in three states and comparing all candidate objects in the image frame with the bone point norms, the test object is found from the candidate objects, realizing the tracking of the test object in each image frame, and can improve the accuracy of determining the bone point coordinates of the test object, thereby improving the accuracy of motion evaluation.

[0069] As an alternative embodiment, refer to Figure 3As shown, another embodiment of the present application discloses that in step S202, by comparing the bone point coordinates of each candidate object in each image frame with the bone point coordinates of the bone point norm corresponding to each image frame determined in advance, the bone point coordinates of the test object in each image frame are obtained, including the following steps:

[0070] S301. Calculate the similarity between each candidate object in each image frame and the corresponding bone point norm by using the bone point coordinates of each candidate object in each image frame and the bone point coordinates of the bone point norm corresponding to each image frame.

[0071] Specifically, in this embodiment, the similarity between each candidate object in the image frame and the bone point norm can be calculated by using the bone point coordinates of each candidate object in the image frame and the bone point coordinates of the bone point norm corresponding to the image frame. Among them, if the bone point norm corresponding to the image frame includes three states, then the similarity between the candidate object and the bone point norm of each state needs to be calculated.

[0072] Further, this step specifically includes:

[0073] First, normalize the bone point coordinates of each candidate object in each image frame and the bone point coordinates of the bone point norm corresponding to each image frame to obtain the normalized bone point coordinates of each candidate object in each image frame and the normalized bone point coordinates of the bone point norm corresponding to each image frame.

[0074] In this embodiment, it is necessary to normalize the bone point coordinates of each candidate object in the image frame and the bone point coordinates of the corresponding bone point norm to obtain the normalized bone point coordinates of each candidate object in the image frame and the normalized bone point coordinates of the corresponding bone point norm, so as to avoid affecting the accuracy of the bone point coordinates detected from the image frame due to the influence of the perspective of the motion video acquisition device on the collected image.

[0075] In this embodiment, it is preferably to normalize the bone points by using the thigh length and the hip center point between the left hip and the right hip. The normalization calculation formula is as follows:

[0076] norm_pose = (pose - hip_center) / leg_len * facter

[0077] hip_center = (hip_left + hip_right) / 2

[0078] knee_center = (knee_left + knee_right) / 2

[0079] leg_len = Euclidean(hip_center, knee_center)

[0080] Among them, norm_pose represents the normalized skeletal point coordinates, pose represents the original skeletal point coordinates, hip_center represents the hip center point coordinates in the candidate object or skeletal point norm corresponding to the original skeletal point coordinates, leg_len represents the distance between the hip center point and the knee center point in the candidate object or skeletal point norm corresponding to the original skeletal point coordinates (i.e., the thigh length), factor represents the scale factor, hip_left represents the left hip skeletal point coordinates in the candidate object or skeletal point norm corresponding to the original skeletal point coordinates, hip_right represents the right hip skeletal point coordinates in the candidate object or skeletal point norm corresponding to the original skeletal point coordinates, knee_center represents the knee center point coordinates in the candidate object or skeletal point norm corresponding to the original skeletal point coordinates, knee_left represents the left knee skeletal point coordinates in the candidate object or skeletal point norm corresponding to the original skeletal point coordinates, and knee_right represents the right knee skeletal point coordinates in the candidate object or skeletal point norm corresponding to the original skeletal point coordinates.

[0081] In addition, this embodiment can also use other skeletal points to normalize all skeletal point coordinates. For example, the upper body length and the skeletal points on the neck can be used. For the sit-up exercise, the accuracy of normalizing the skeletal points using the thigh length and the hip center point between the left hip and the right hip is higher.

[0082] Second, combine all the normalized skeletal point coordinates of each candidate object in each image frame into a skeletal point vector of each candidate object in each image frame, and combine all the normalized skeletal point coordinates of the corresponding skeletal point norm of each image frame into a skeletal point vector of the corresponding skeletal point norm of each image frame.

[0083] After normalizing the skeletal point coordinates of each candidate object in the image frame and the skeletal point coordinates of the corresponding skeletal point norm, determine the normalized skeletal point coordinate sequence of each candidate object in the image frame and the normalized skeletal point coordinate sequence of the corresponding skeletal point norm, and then straighten the skeletal point coordinate sequence to obtain a skeletal point vector. If there are 30 skeletal point coordinates in the candidate object or skeletal point norm, then the corresponding skeletal point coordinate sequence is 30*2-dimensional, and the straightened skeletal point vector is 60*1-dimensional. For example, the normalized skeletal point coordinate sequence is ((x0, y0), (x1, y1), ……, (xh, yh)), and the skeletal point vector obtained after straightening this coordinate sequence is (x0, y0, x1, y1, ……, xh, yh), where k is the number of skeletal point coordinates in the candidate object or skeletal point norm.

[0084] Thirdly, calculate the vector similarity between the skeleton point vectors of each candidate object in each image frame and the skeleton point vectors of the corresponding skeleton point norms as the similarity between each candidate object and the corresponding skeleton point norm in each image frame.

[0085] In this embodiment, the Pearson similarity calculation method can be used to calculate the vector similarity between the skeleton point vectors of the candidate object and the skeleton point vectors of the skeleton point norm, and this vector similarity is used as the similarity between the candidate object and the skeleton point norm. Among them, the calculation formula for calculating the Pearson similarity of two skeleton point vectors m and n is as follows:

[0086]

[0087] Among them, r represents the vector similarity between the skeleton point vectors m and n, h represents the number of vector elements in the skeleton point vector, m i represents the i-th vector element in the skeleton point vector m, n i represents the i-th vector element in the skeleton point vector n, represents the average value of the vector elements in the skeleton point vector m, represents the average value of the vector elements in the skeleton point vector n.

[0088] S302. Use the skeleton point coordinates of the candidate object with the largest similarity to the corresponding skeleton point norm in each image frame as the skeleton point coordinates of the test object in each image frame.

[0089] Specifically, compare the similarities between each candidate object and each skeleton point norm in the image frames obtained in the above steps, and use the candidate object with the largest similarity value as the test object in this image frame, so as to obtain the skeleton point coordinates of the test object in each image frame. When comparing similarities, first compare the similarities between a candidate object and each skeleton point norm, and take the largest similarity as the target similarity corresponding to this candidate object. Use this method to determine the target similarities corresponding to all candidate objects in the image frame, and then compare the target similarities of all candidate objects, and use the candidate object corresponding to the maximum value of the target similarity as the test object.

[0090] Among them, the determination formula for the test object is as follows:

[0091]

[0092] Among them, id represents the identifier of the test object, represents the vector similarity between the normalized skeleton point coordinate vector of the i-th candidate object in the image frame and the normalized skeleton point coordinate vector of the corresponding sitting norm of the image frame, It represents the vector similarity between the normalized skeletal point coordinate vector of the i-th candidate object in the image frame and the normalized skeletal point coordinate vector of the corresponding flat norm of the image frame. It represents the vector similarity between the normalized skeletal point coordinate vector of the i-th candidate object in the image frame and the normalized skeletal point coordinate vector of the corresponding intermediate norm of the image frame, and k represents the number of candidate objects in the image frame.

[0093] As an optional implementation manner, refer to Figure 4 As shown, another embodiment of the present application discloses that in the above step S101, according to the skeletal point coordinates of the test object in each image frame, the action attributes of the test object in each image frame are classified to determine the action attributes corresponding to each image frame, including the following steps:

[0094] S401. Normalize all the skeletal point coordinates of the test object in each image frame to obtain the corresponding normalized skeletal point coordinates of each image frame.

[0095] Specifically, in this embodiment, it is necessary to normalize the skeletal point coordinates of the test object in the image frame to obtain the normalized skeletal point coordinates of the test object in each image frame, so as to avoid being affected by the perspective of the acquisition device for the images collected by the motion video acquisition device, thereby affecting the accuracy of the skeletal point coordinates detected from the image frame.

[0096] In this embodiment, it is preferably to normalize the skeletal points by using the thigh length and the hip center point between the left hip and the right hip. The normalization calculation method is the same as the normalization method in the above embodiment, and will not be specifically elaborated in this embodiment. This embodiment can also use other skeletal points to normalize all the skeletal point coordinates. For example, use the upper body length and the skeletal points on the neck, etc. For the sit-up exercise, the accuracy of normalizing the skeletal points by using the thigh length and the hip center point between the left hip and the right hip is higher.

[0097] S402. Combine all the normalized skeletal point coordinates corresponding to each image frame into the skeletal point vector corresponding to each image frame.

[0098] Specifically, after normalizing the skeletal point coordinates of the test object in the image frame, determine the normalized skeletal point coordinate sequence of the test object in the image frame, and then straighten the normalized skeletal point coordinate sequence to obtain the skeletal point vector of the test object in the image frame. Among them, the specific method of straightening the normalized skeletal point coordinate sequence is the same as the straightening method in the above embodiment, and will not be specifically elaborated in this embodiment.

[0099] S403. Classify the action attributes of the skeletal point vectors corresponding to each image frame to determine the action attributes corresponding to each image frame.

[0100] Specifically, after determining the bone point vectors of the test object in each image frame in this embodiment, the action attributes of the test object can be classified according to the bone point vectors of the test object to determine the action attributes corresponding to each image frame. In this embodiment, an action attribute classification model can be pre-trained using sample bone point vectors. Among them, the action attributes corresponding to the image frame include at least one attribute element, and the value of each attribute element is used to indicate whether the test object in the image frame shows a posture corresponding to the attribute element.

[0101] For example, the action attributes may include at least one attribute element among the knee-touching state, lying-flat state, knee-bending state, head-holding state, and orientation state. In this embodiment, 1 and 0 can be used to represent yes and no. If the value of the knee-touching state is 1, it means the test object in the image frame touches the knee; if the value of the knee-touching state is 0, it means the test object in the image frame does not touch the knee. If the value of the lying-flat state is 1, it means the test object in the image frame is lying flat; if the value of the lying-flat state is 0, it means the test object in the image frame is not lying flat. If the value of the knee-bending state is 1, it means the test object in the image frame bends the knee; if the value of the knee-bending state is 0, it means the test object in the image frame does not bend the knee. If the value of the head-holding state is 1, it means the test object in the image frame holds the head; if the value of the head-holding state is 0, it means the test object in the image frame does not hold the head. If the value of the orientation state is 1, it means the test object in the image frame faces right; if the value of the orientation state is 0, it means the test object in the image frame faces left.

[0102] In this embodiment, the pre-trained action attribute classification model includes a multi-layer perceptron network and a classification neural network. The bone point vector of the test object in the image frame is input into the multi-layer perceptron network, and the multi-layer perceptron network can perform action attribute analysis on the bone point vector to output an action attribute vector, where the action attribute vector contains the numerical values corresponding to each action attribute element. If there are 30 bone point coordinates of the test object in the image frame and the action attributes include 5 action attribute elements, the dimension of the bone point vector is a 60-dimensional vector. Then, the input layer of the multi-layer perceptron network is 60-dimensional, the middle layer is 64-dimensional, and the output layer is 5-dimensional. The action attribute vector output by the multi-layer perceptron network is a 5-dimensional vector. The classification neural network performs sigmoid operation on the action attribute vector output by the multi-layer perceptron network to implement the normalization operation of the action attribute vector, obtaining numerical values normalized from 0 to 1. Each numerical value corresponds to a threshold. If it is greater than the threshold, 1 is output; if it is less than or equal to the threshold, 0 is output, thereby obtaining five numerical values. These five numerical values respectively represent whether the test object shows a posture corresponding to the numerical value.

[0103] As an alternative implementation, refer to Figure 5As shown in the figure, another embodiment of the present application discloses that in step S102, according to the action attributes corresponding to each image frame, it is determined whether each group of actions during the movement of the test object is compliant, and a normative evaluation result for each group of actions is obtained, including the following steps:

[0104] S501. Determine each image frame corresponding to each group of actions from the movement video of the test object.

[0105] Specifically, in order to evaluate the movement of the test object, it is necessary to conduct a normative evaluation and a counting evaluation on all movement actions during the movement of the test object. Therefore, in this embodiment, it is first necessary to determine the image frames corresponding to each group of actions in the movement video of the test object.

[0106] In this embodiment, the action state of the test object in the image frame of the movement video can be detected. According to the periodic transformation of the action states corresponding to each image frame, the period of each group of actions can be determined, and the image frames within the period of each group of actions are used as the image frames corresponding to each group of actions. For example, in the sit-up movement, the period from the lying flat action to the sitting straight action and then from the sitting straight action to the lying flat action is the period of a group of actions.

[0107] S502. Determine whether each group of actions during the movement of the test object is compliant according to the action attributes corresponding to each image frame corresponding to each group of actions, and obtain a normative evaluation result for each group of actions.

[0108] Specifically, in this embodiment, it is necessary to perform compliance analysis on each group of actions by using the action attributes corresponding to all the image frames included in each group of actions, so as to obtain the normative evaluation results of each group of actions. The cycle of each group of actions can be divided into the upper half cycle and the lower half cycle. The action states of the image frames in the upper half cycle are different from those of the image frames in the lower half cycle. In this embodiment, the cycle of each group of actions can be divided according to the action states corresponding to the respective image frames in a group of actions. The image frames corresponding to the first action state in a group of actions are used as the image frames of the upper half cycle of the action, and the image frames corresponding to the second action state are used as the image frames of the lower half cycle of the action. For example, in the motion evaluation of sit-ups, the action states of the image frames between the lying flat inflection point and the sitting up inflection point in a group of actions are all in the sitting up state, and these image frames are the image frames of the upper half cycle; the action states of the image frames between the sitting up inflection point and the lying flat inflection point are all in the lying flat state, and these image frames are the image frames of the lower half cycle. Among them, the lying flat inflection point is the image frame after the image frame with the largest body curvature of the test object in a group of actions and the image frame with the tendency to sit up; the sitting up inflection point is the image frame after the image frame with the smallest body curvature of the test object in a group of actions and the image frame with the tendency to lie down. In the motion evaluation of sit-ups, the first compliance analysis of a group of actions is to analyze the image frames of the lower half cycle of the previous group of actions corresponding to this group of actions, and the second compliance analysis of a group of actions is to analyze the image frames of the upper half cycle of this group of actions.

[0109] Further, for two adjacent groups of actions, where the second group of actions is the subsequent group of actions adjacent to the first group of actions, the specific steps for the compliance analysis of the second group of actions are as follows:

[0110] Determine whether the second group of actions is compliant according to the first compliance analysis result corresponding to the first group of actions and the action attributes corresponding to the respective image frames in the first action state of the second group of actions, so as to obtain the normative evaluation result of the second group of actions.

[0111] The image frames corresponding to a set of actions include: the image frames corresponding to the first action state and the image frames corresponding to the second action state adjacent to the first action state. In the sit-up exercise, they are the image frames corresponding to the sitting-up state and the lying-flat state. In this embodiment, by performing a first compliance analysis on the action attributes corresponding to each image frame of the second action state in the first set of actions, the first compliance analysis result corresponding to the first set of actions is determined. By performing a second compliance analysis on the action attributes corresponding to each image frame of the first action state in the second set of actions, the second compliance analysis result corresponding to the second set of actions is determined. The first compliance analysis result corresponding to the first set of actions and the second compliance analysis result corresponding to the second set of actions are used as the normative evaluation result of the second set of actions. And the first compliance analysis result analyzed from the action attributes corresponding to each image frame of the second action state in the second set of actions is the content included in the normative evaluation result of the third set of actions adjacent to the second set of actions.

[0112] In the sit-up exercise, by performing a second compliance analysis on the action attributes corresponding to each image frame of the first action state in the second set of actions, the second compliance analysis result corresponding to the second set of actions is obtained, which specifically includes: If the knee-touching state of each image frame in the first action state (i.e., the sitting-up state) of the second set of actions is not knee-touching, it indicates that the test subject did not touch the knees during the second set of actions, and it is determined that the second compliance analysis result corresponding to the second set of actions includes a non-knee-touching violation. If there are image frames with a knee-touching state of knee-touching among the image frames of the first action state (i.e., the sitting-up state) of the second set of actions, then analyze the knee-bending state and the head-holding state of the image frames to be analyzed in the second set of actions, where the image frames to be analyzed are the image frames with a knee-touching state of knee-touching and the image frames before the image frames with a knee-touching state of knee-touching among the image frames of the first action state. If there are image frames with a knee-bending state of not knee-bending among the image frames to be analyzed in the second set of actions, it indicates that there is a situation of not knee-bending before knee-touching when the test subject performs the second set of actions, and it is determined that the second compliance analysis result corresponding to the second set of actions includes a non-knee-bending violation. If there are image frames with a head-holding state of not head-holding among the image frames to be analyzed in the second set of actions, it indicates that there is a situation of not head-holding before knee-touching when the test subject performs the second set of actions, and it is determined that the second compliance analysis result corresponding to the second set of actions includes a non-head-holding violation.

[0113] By performing a first compliance analysis on the action attributes corresponding to each image frame of the second action state in the first set of actions, the first compliance analysis result corresponding to the first set of actions is determined, which specifically includes: If the lying-flat state of each image frame in the second action state (i.e., the lying-flat state) of the first set of actions is not lying flat, it is determined that the first compliance analysis result corresponding to the first set of actions includes a non-lying-flat violation, indicating that when the test subject starts the second set of actions, they are not lying flat. Therefore, there is a non-lying-flat violation in the second set of actions.

[0114] As an alternative embodiment, refer to Figure 6 As shown, another embodiment of the present application discloses that for the above step S501, to determine each image frame corresponding to each group of actions from the motion video of the test object, the following steps are included:

[0115] S601. Analyze the action state corresponding to each image frame according to the bone point coordinates of the test object in each image frame of the motion video.

[0116] Specifically, in this embodiment, the body bending angle of the test object in each image frame of the motion video can be calculated according to the bone point coordinates of the test object in each image frame, and then the current motion speed of the test object in each image frame can be calculated according to the body bending angle of the test object in each image frame. According to the body bending angle and motion speed of the test object in each image frame, analyze the action state corresponding to the image frame.

[0117] Further, this step specifically includes:

[0118] First, judge whether the image frame meets the pre-set state switching condition according to the bone point coordinates of the test object in the image frame.

[0119] In this embodiment, the pre-set state switching condition is the condition for switching the action state corresponding to the image frame to an action state different from the action state corresponding to the previous image frame of the image frame. When a group of actions includes two action states, the image frames corresponding to a group of actions include: the image frames corresponding to the first action state and the image frames corresponding to the second action state adjacent to the first action state. If the action state corresponding to the previous image frame of the current image frame is the first action state, determine that the state switching condition is the state switching condition for switching to the second action state, and judge whether the current image frame meets the state switching condition for switching to the second action state; if the current image frame has no previous image frame (i.e., the first image frame), or the action state corresponding to the previous image frame of the current image frame is the second action state or no action state, determine that the state switching condition is the state switching condition for switching to the first action state, and judge whether the current image frame meets the state switching condition for switching to the first action state.

[0120] Specifically, to determine whether an image frame meets the state transition condition, it is first necessary to calculate the body bending angle of the test object in the current image frame based on the bone point coordinates of the test object in the current image frame, calculate the body bending angle of the test object in the previous image frame of the current image frame based on the bone point coordinates of the test object in the previous image frame of the current image frame, and then calculate the movement speed (i.e., angular velocity) of the test object in the current image frame according to the difference between the body bending angles of the test object in the two image frames and the difference between the acquisition times of the two image frames. Among them, the body bending angle of the test object in the image frame can be the angle between the vector from the hip to the ankle and the vector from the hip to the shoulder as the body bending angle.

[0121] After determining the movement speed of the test object in the current image frame in this embodiment, it is necessary to compare the movement speed with a preset speed threshold. If the movement speed of the test object in the current image frame is not less than the preset speed threshold, it is determined that the current image frame does not meet the state transition condition. If the movement speed of the test object in the current image frame is less than the preset speed threshold, it means that the test object may meet the state transition condition, that is, the current image frame may be an inflection point of state transition, which is a candidate inflection point for state transition. At this time, it is necessary to determine whether the test object in the current image frame has a trend of action state transition according to the action state corresponding to the previous image frame of the current image frame and the body bending angle of the test object in the current image frame. If the test object in the current image frame has a trend of action state transition, it means that the current image frame as a candidate inflection point is an inflection point of state transition and meets the state transition condition. If the test object in the current image frame does not have a trend of action state transition, it means that the current image frame as a candidate inflection point is not an inflection point of state transition and does not meet the state transition condition. For example, if the test object in the current image frame as a candidate inflection point has a trend of switching to the second action state (lying flat state), it means that the current image frame is a lying flat inflection point and meets the state transition condition of switching to the second action state. If the test object in the current image frame as a candidate inflection point has a trend of switching to the first action state (sitting up state), it means that the current image frame is a sitting up inflection point and meets the state transition condition of switching to the first action state.

[0122] Among them, determining whether there is a tendency for the action state of the test object to switch in the current image frame based on the action state corresponding to the previous image frame of the current image frame and the body bending angle of the test object in the current image frame is specifically as follows: If the action state corresponding to the previous image frame of the current image frame is the first action state (i.e., the sitting-up state), then determine the minimum body bending angle of the test object among all the image frames with the first action state (i.e., the sitting-up state) near the previous image frame, calculate the angle difference between the body bending angle of the test object in the current image frame and the minimum body bending angle. If this angle difference is greater than the preset threshold, it indicates that the test object in the current image frame has a tendency to switch to the second action state (i.e., the lying-down tendency). If the action state corresponding to the previous image frame of the current image frame is the second action state (i.e., the lying-flat state), then determine the maximum body bending angle of the test object among all the image frames with the second action state (i.e., the lying-flat state) near the previous image frame, calculate the angle difference between the body bending angle of the test object in the current image frame and the maximum body bending angle. If this angle difference is greater than the preset threshold, it indicates that the test object in the current image frame has a tendency to switch to the first action state (i.e., the sitting-up tendency).

[0123] As Figure 8 shown, the right hip bone point of the test object in the image frame is O, the right ankle bone point is A, and the right shoulder bone point is B. To calculate the body bending angle of the test object, the hip-ankle vector hip-shoulder vector between can be directly calculated. The calculation formula is:

[0124]

[0125] Since the range of the included angle α calculated by the cosine theorem is [0°, 180°], when the test object is in the posture as Figure 9 shown, the calculated included angle α is the angle between the vector clockwise to the vector However, in fact, the body bending angle of the test object should be the angle between the vector counterclockwise to the vector That is, 360° - α. At this time, the change curve graph of the body bending angle of the test object in the motion video will show an angle mutation situation in the curve graph as Figure 7 shown, which affects the accuracy of determining the action state corresponding to the image frame. Therefore, in this embodiment, it is necessary to determine the body bending angle of the test object according to the orientation state in the action attribute corresponding to the image frame and the cross product between the vector and the vector That is, when the orientation state corresponding to the image frame indicates facing right and cross < 0 (vector In the vector in the clockwise direction), and the orientation state corresponding to the image frame indicates facing left, and cross > 0 (vector in the vector are both in the counterclockwise direction), the body bending angle is 360° - α. Then the change curve graph of the body bending angle of the test object in the motion video at this time will be corrected to Figure 10 the curve graph shown.

[0126] Second, if the image frame meets the state switching condition, determine that the action state corresponding to the image frame is the action state corresponding to the state switching condition.

[0127] The action state corresponding to the state switching condition is the action state after switching the action state corresponding to the image frame according to the state switching condition. For example, if the image frame meets the state switching condition for switching to the first action state, determine that the action state corresponding to the image frame is the first action state. If the image frame meets the state switching condition for switching to the second action state, determine that the action state corresponding to the image frame is the second action state.

[0128] Third, if the image frame does not meet the state switching condition, determine that the action state corresponding to the image frame is the action state of the previous frame image frame of the image frame.

[0129] For example, if the previous frame image frame of the image frame is in the first action state and this image frame does not meet the state switching condition for switching to the second action state, determine that the action state corresponding to this image frame is the first action state. If the previous frame image frame of the image frame is in the second action state and this image frame does not meet the state switching condition for switching to the first action state, determine that the action state corresponding to this image frame is the second action state.

[0130] S602. Perform a periodic analysis on the action states corresponding to each image frame to determine each image frame corresponding to each group of actions.

[0131] Specifically, after determining the action states corresponding to each image frame in the motion video through the above steps, the actions can be divided according to the periodicity of the action states. For example, the image frames corresponding to the first action state and the image frames corresponding to the second action state adjacent to the first action state are used as the image frames corresponding to a group of actions.

[0132] As an optional implementation manner, refer to Figure 11As shown, another embodiment of the present application discloses that, for timing sports evaluation, when evaluating sit-ups, the sit-up state machine performs state analysis based on the action attributes corresponding to each image frame of the input motion video and the bone point vector of the test object. First, it circularly detects whether the preparatory action is standard. Among them, a standard preparatory action means that the corresponding knee-bending state in the image frame indicates knee-bending, the head-holding state indicates head-holding, and the lying-flat state indicates lying flat. When it is detected that the action attributes corresponding to the image frame conform to the standard preparatory action, the action state corresponding to the image frame is determined to be the preparatory state, and the timing starts. Then, it detects the lying-flat inflection point. When the lying-flat inflection point is detected, the action state of the image frame is switched to the lying-flat state. Then, it detects the sit-up inflection point. When the sit-up inflection point is detected, the action state of the image frame is switched to the sit-up state. Then, it detects the lying-flat inflection point until the timing duration reaches the evaluation duration or the test object leaves the evaluation area. Moreover, during the process of detecting the sit-up inflection point, the standardization evaluation of touching the knees, knee-bending, and head-holding is carried out, and during the detection of the lying-flat inflection point, the standardization evaluation of lying flat is carried out.

[0133] Corresponding to the above-mentioned sports evaluation method, an embodiment of the present application also proposes a sports evaluation device. Refer to Figure 12 As shown, the device includes:

[0134] An action attribute classification module 100, configured to classify the action attributes of the image frames of the motion video of the test object, and determine the action attributes corresponding to each image frame; the action attributes corresponding to the image frame represent the posture of the test object presented in the image frame;

[0135] An action compliance analysis module 110, configured to determine whether each group of actions of the test object during movement is compliant according to the action attributes corresponding to each image frame, and obtain the standardization evaluation result of each group of actions;

[0136] A sports evaluation module 120, configured to determine the sports evaluation result of the corresponding test object according to the standardization evaluation result of each group of actions.

[0137] An embodiment of the present application provides a motion evaluation device. The action attribute classification module 100 classifies the image frames of the motion video of the test object to determine the action attributes corresponding to each image frame. The action attribute corresponding to the image frame represents the posture of the test object presented in the image frame. The action compliance analysis module 110 determines whether each group of actions of the test object is compliant according to the action attributes corresponding to each image frame, and obtains the normative evaluation result of each group of actions. The motion evaluation module 120 determines the motion evaluation result corresponding to the test object according to the normative evaluation result of each group of actions. By adopting the technical solution of this embodiment, the action analysis can be performed on the image frames collected during the test, it can be determined whether the actions of the test object are compliant, and the action count can be performed according to the normative evaluation result of the actions. Compared with manual evaluation, it is more accurate, and not only the quantity evaluation is performed, but also the action specification evaluation is realized, which improves the accuracy of motion evaluation and action evaluation.

[0138] As an alternative implementation, another embodiment of the present application further discloses that the action compliance analysis module 110 includes: an action determination unit and a normative evaluation unit.

[0139] The action determination unit is used to determine each image frame corresponding to each group of actions from the motion video of the test object;

[0140] The normative evaluation unit is used to determine whether each group of actions of the test object is compliant according to the action attributes corresponding to each image frame corresponding to each group of actions, and obtain the normative evaluation result of each group of actions.

[0141] As an alternative implementation, another embodiment of the present application further discloses that the action attribute classification module 100 includes: a bone point detection unit and a classification unit;

[0142] The bone point detection unit is used to perform human bone point detection on each image frame in the motion video of the test object, and obtain the bone point coordinates of the test object in each image frame;

[0143] The classification unit is used to classify the action attributes of the test object in each image frame according to the bone point coordinates of the test object in each image frame, and determine the action attributes corresponding to each image frame.

[0144] As an alternative implementation, another embodiment of the present application further discloses that the bone point detection unit is specifically used for:

[0145] Perform human bone point detection on each image frame in the motion video of the test object to obtain the bone point coordinates of all candidate objects in each image frame;

[0146] By comparing the bone point coordinates of each candidate object in each image frame with the bone point coordinates of the bone point norm corresponding to each pre-determined image frame, the bone point coordinates of the test object in each image frame are obtained;

[0147] Among them, the bone point coordinates of the bone point norm corresponding to the image frame are determined by sliding averaging using the bone point coordinates of the norm image frame in the motion video; the norm image frame is the image frame with the same posture as the bone point norm among the image frames before the current image frame.

[0148] As an alternative implementation, another embodiment of the present application also discloses that the bone point detection unit obtains the bone point coordinates of the test object in each image frame by comparing the bone point coordinates of each candidate object in each image frame with the bone point coordinates of the bone point norm corresponding to each pre-determined image frame, including:

[0149] Using the bone point coordinates of each candidate object in each image frame and the bone point coordinates of the bone point norm corresponding to each image frame, calculate the similarity between each candidate object in each image frame and the corresponding bone point norm;

[0150] Take the bone point coordinates of the candidate object with the maximum similarity to the corresponding bone point norm in each image frame as the bone point coordinates of the test object in each image frame.

[0151] As an alternative implementation, another embodiment of the present application also discloses that the bone point detection unit uses the bone point coordinates of each candidate object in each image frame and the bone point coordinates of the bone point norm corresponding to each image frame to calculate the similarity between each candidate object in each image frame and the corresponding bone point norm, including:

[0152] Normalize both the bone point coordinates of each candidate object in each image frame and the bone point coordinates of the bone point norm corresponding to each image frame to obtain the normalized bone point coordinates of each candidate object in each image frame and the normalized bone point coordinates of the bone point norm corresponding to each image frame;

[0153] Combine all the normalized bone point coordinates of each candidate object in each image frame into the bone point vector of each candidate object in each image frame, and combine all the normalized bone point coordinates of the bone point norm corresponding to each image frame into the bone point vector of the bone point norm corresponding to each image frame;

[0154] Calculate the vector similarity between the bone point vector of each candidate object in each image frame and the bone point vector of the corresponding bone point norm as the similarity between each candidate object in each image frame and the corresponding bone point norm.

[0155] As an alternative implementation, another embodiment of the present application also discloses that the classification unit is specifically used for:

[0156] Normalize the coordinates of all the skeletal points of the test object in each image frame to obtain the normalized skeletal point coordinates corresponding to each image frame;

[0157] Combine all the normalized skeletal point coordinates corresponding to each image frame into a skeletal point vector corresponding to each image frame;

[0158] Perform action attribute classification on the skeletal point vectors corresponding to each image frame to determine the action attributes corresponding to each image frame.

[0159] As an alternative implementation, another embodiment of the present application also discloses that the classification unit performs action attribute classification on the skeletal point vectors corresponding to each image frame to determine the action attributes corresponding to each image frame, including:

[0160] Input the skeletal point vectors corresponding to each image frame into a pre-trained action attribute classification model to obtain the action attributes corresponding to each image frame;

[0161] Among them, the action attribute classification model is obtained by training the action attribute classification of sample skeletal point vectors; the action attributes include at least one attribute element, and the value of each attribute element is used to indicate whether the test object shows a posture corresponding to the attribute element.

[0162] As an alternative implementation, another embodiment of the present application also discloses an action determination unit, specifically used for:

[0163] Analyze the action state corresponding to each image frame according to the skeletal point coordinates of the test object in each image frame of the motion video;

[0164] Perform periodic analysis on the action states corresponding to each image frame to determine each image frame corresponding to each group of actions.

[0165] As an alternative implementation, another embodiment of the present application also discloses that the action determination unit analyzes the action state corresponding to the image frame according to the skeletal point coordinates of the test object in each image frame of the motion video, including:

[0166] Judge whether the image frame meets the pre-set state switching condition according to the skeletal point coordinates of the test object in the image frame;

[0167] If the image frame meets the state switching condition, determine that the action state corresponding to the image frame is the action state corresponding to the state switching condition;

[0168] If the image frame does not meet the state switching condition, determine that the action state corresponding to the image frame is the action state corresponding to the previous image frame of the image frame;

[0169] Among them, the state transition condition is the condition for switching the action state corresponding to the image frame to an action state different from the action state corresponding to the previous image frame of the image frame, and the action state corresponding to the state transition condition is the action state after switching the action state corresponding to the image frame according to the state transition condition.

[0170] As an optional implementation manner, another embodiment of the present application further discloses that the action determination unit determines whether the image frame meets a preset state transition condition according to the skeletal point coordinates of the test object in the image frame, including:

[0171] Calculate the movement speed of the test object in the image frame according to the skeletal point coordinates of the test object in the image frame and the skeletal point coordinates of the test object in the previous image frame of the image frame;

[0172] If the movement speed of the test object in the image frame is not less than the preset speed threshold, it is determined that the image frame does not meet the state transition condition;

[0173] If the movement speed of the test object in the image frame is less than the preset speed threshold, then determine whether the test object in the image frame has a tendency to switch the action state according to the action state corresponding to the previous image frame of the image frame and the body bending angle of the test object in the image frame;

[0174] If the test object in the image frame does not have a tendency to switch the action state, it is determined that the image frame does not meet the state transition condition;

[0175] If the test object in the image frame has a tendency to switch the action state, it is determined that the image frame meets the state transition condition.

[0176] As an optional implementation manner, another embodiment of the present application further discloses a specification evaluation unit, which is specifically used for:

[0177] Determine whether the second group of actions is compliant according to the first compliance analysis result corresponding to the first group of actions and the action attributes corresponding to each image frame in the first action state of the second group of actions, and obtain the specification evaluation result of the second group of actions;

[0178] Among them, the first compliance analysis result corresponding to the first group of actions is determined by performing a first compliance analysis on the action attributes corresponding to each image frame in the second action state of the first group of actions; the second group of actions is the group of actions adjacent to the first group of actions at the back, and the image frames corresponding to a group of actions include: the image frames corresponding to the first action state and the image frames corresponding to the second action state adjacent to the back of the first action state.

[0179] As an alternative implementation, another embodiment of the present application further discloses that the specification evaluation unit determines whether the second set of actions is compliant based on the first compliance analysis result corresponding to the first set of actions and the action attributes corresponding to each image frame of the first action state in the second set of actions, and obtains the specification evaluation result of the second set of actions, including:

[0180] Perform a second compliance analysis on the action attributes corresponding to each image frame of the first action state in the second set of actions to obtain a second compliance analysis result corresponding to the second set of actions;

[0181] Use the first compliance analysis result corresponding to the first set of actions and the second compliance analysis result corresponding to the second set of actions as the specification evaluation result of the second set of actions.

[0182] As an alternative implementation, another embodiment of the present application further discloses that the action attributes include: knee-touching state, knee-bending state, and head-holding state; the specification evaluation unit performs a second compliance analysis on the action attributes corresponding to each image frame of the first action state in the second set of actions to obtain a second compliance analysis result corresponding to the second set of actions, including:

[0183] If the knee-touching state of each image frame of the first action state in the second set of actions is not knee-touching, it is determined that the second compliance analysis result corresponding to the second set of actions includes a non-knee-touching violation;

[0184] If there is an image frame with a knee-touching state of knee-touching among the image frames of the first action state in the second set of actions, analyze the knee-bending state and head-holding state of the image frames to be analyzed in the second set of actions; the image frames to be analyzed are the image frames with a knee-touching state of knee-touching and the image frames before the image frames with a knee-touching state of knee-touching among the image frames of the first action state;

[0185] If there is an image frame with a non-knee-bending state among the image frames to be analyzed in the second set of actions, it is determined that the second compliance analysis result corresponding to the second set of actions includes a non-knee-bending violation;

[0186] If there is an image frame with a non-head-holding state among the image frames to be analyzed in the second set of actions, it is determined that the second compliance analysis result corresponding to the second set of actions includes a non-head-holding violation.

[0187] As an alternative implementation, another embodiment of the present application further discloses that the action attribute includes a lying-flat state; the specification evaluation unit determines the first compliance analysis result corresponding to the first set of actions by performing a first compliance analysis on the action attributes corresponding to each image frame of the second action state in the first set of actions, including:

[0188] If the lying-flat state of each image frame of the second action state in the first set of actions is not lying-flat, it is determined that the first compliance analysis result corresponding to the first set of actions includes a non-lying-flat violation.

[0189] As an alternative implementation, another embodiment of the present application further discloses that the motion evaluation module 120 is specifically configured to:

[0190] Regard the actions without violations in the normative evaluation result as standard actions;

[0191] Record the number of standard actions in the motion video of the test object, and use the number of standard actions and the normative evaluation results of each group of actions as the motion evaluation results of the test object.

[0192] The motion evaluation device provided in this embodiment belongs to the same inventive concept as the motion evaluation method provided in the above embodiments of the present application, can execute the motion evaluation method provided in any of the above embodiments of the present application, and has the corresponding functional modules and beneficial effects for executing the motion evaluation method. For the technical details not described in detail in this embodiment, reference can be made to the specific processing content of the motion evaluation method provided in the above embodiments of the present application, which will not be elaborated here.

[0193] Another embodiment of the present application further discloses an electronic device. Refer to Figure 13 as shown, the device includes:

[0194] A memory 200 and a processor 210;

[0195] Wherein, the memory 200 is connected to the processor 210 and is used to store programs;

[0196] The processor 210 is configured to implement the motion evaluation method disclosed in any of the above embodiments by running the programs stored in the memory 200.

[0197] Specifically, the above electronic device may further include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0198] The processor 210, the memory 200, the communication interface 220, the input device 230, and the output device 240 are interconnected through the bus. Among them:

[0199] The bus may include a path for transmitting information between various components of the computer system.

[0200] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the solution of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0201] The processor 210 may include a main processor and may also include a baseband chip, a modem, etc.

[0202] The memory 200 stores a program for implementing the technical solution of the present invention and may also store an operating system and other critical services. Specifically, the program may include program code, and the program code includes computer operation instructions. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk memory, a flash memory, etc.

[0203] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor, etc.

[0204] The output device 240 may include a device for allowing information to be output to a user, such as a display screen, a printer, a speaker, etc.

[0205] The communication interface 220 may include a device of any transceiver type for communicating with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0206] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the steps of the motion evaluation method provided by the embodiments of the present application.

[0207] Another embodiment of the present application also provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the motion evaluation method provided by any of the above embodiments are implemented.

[0208] For each of the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should understand that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0209] It should be noted that the embodiments in this specification are all described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for the relevant content.

[0210] The steps in the methods of the embodiments of this application can be adjusted, combined, and deleted according to actual needs.

[0211] The modules and sub-modules in the devices and terminals in the embodiments of this application can be combined, divided, and deleted according to actual needs.

[0212] In several embodiments provided by this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are only illustrative. For example, the division of modules or sub-modules is only a logical function division. In actual implementation, there can be other division methods. For example, multiple sub-modules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of devices or modules can be in electrical, mechanical, or other forms.

[0213] The modules or sub-modules described as separate components may or may not be physically separated. The components as modules or sub-modules may or may not be physical modules or sub-modules, that is, they can be located in one place, or they can be distributed to multiple network modules or sub-modules. Some or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0214] In addition, the functional modules or sub-modules in each embodiment of this application can be integrated into one processing module, or each module or sub-module can exist physically alone, or two or more modules or sub-modules can be integrated into one module. The above integrated modules or sub-modules can be implemented in the form of hardware or in the form of software functional modules or sub-modules.

[0215] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0216] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be directly implemented by hardware, software units executed by a processor, or a combination of both. The software units can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0217] Finally, it should also be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0218] The above description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A motion evaluation method, characterized in that: include: Performing human skeleton point detection on each image frame in the motion video of the test object to obtain the coordinates of the skeleton points of all candidate objects in each image frame; The skeletal point coordinates of each candidate object in each image frame are compared with the skeletal point coordinates of the skeletal point norm corresponding to each image frame, so as to obtain the skeletal point coordinates of the test object in each image frame; wherein the skeletal point coordinates of the skeletal point norm corresponding to the image frame are determined by sliding average of the skeletal point coordinates of the norm image frame in the motion video; the norm image frame is an image frame in which the posture of the test object and the skeletal point norm are the same in the image frame before the image frame; Classifying the motion attributes of the test object in each image frame according to the coordinates of the skeletal points of the test object in each image frame, and determining the motion attributes corresponding to each image frame; the motion attributes corresponding to the image frame represent the posture of the test object presented in the image frame; Determine whether each set of movements of the test object during the movement is compliant according to the movement attributes corresponding to each image frame, and obtain a normative evaluation result for each set of movements; According to the normative evaluation results of each group of movements, the movement evaluation results corresponding to the test object are determined.

2. The method according to claim 1, characterized in that: According to the action attributes corresponding to each image frame, determine whether each group of actions of the test object during movement is compliant, and obtain the normative evaluation results of each group of actions, including: Determining, from the motion video of the test subject, individual image frames corresponding to each set of actions; According to the action attributes corresponding to each image frame corresponding to each group of actions, it is determined whether each group of actions of the test object during movement is compliant, and a normative evaluation result for each group of actions is obtained.

3. The method according to claim 1, characterized in that The step of classifying the action attributes of the test object in each image frame according to the coordinates of the skeletal points of the test object in each image frame and determining the action attributes corresponding to each image frame includes: Normalizing all the bone point coordinates of the test object in each image frame to obtain normalized bone point coordinates corresponding to each image frame; All normalized bone point coordinates corresponding to each image frame are combined into a bone point vector corresponding to each image frame; The action attributes of the skeleton point vectors corresponding to each image frame are classified to determine the action attributes corresponding to each image frame.

4. The method according to claim 3, characterized in that The step of classifying the action attributes of the skeleton point vectors corresponding to each image frame to determine the action attributes corresponding to each image frame includes: Input the skeleton point vector corresponding to each image frame into the pre-trained action attribute classification model to obtain the action attribute corresponding to each image frame; The action attribute classification model is obtained by performing action attribute classification training on sample skeleton point vectors; the action attribute includes at least one attribute element, and the value of each attribute element is used to indicate whether the test object exhibits a posture corresponding to the attribute element.

5. The method according to claim 2, characterized in that: Determining, from the motion video of the test object, each image frame corresponding to each group of actions, including: Analyzing the motion state corresponding to each image frame according to the coordinates of the skeletal points of the test object in each image frame of the motion video; The action states corresponding to the image frames are periodically analyzed to determine the image frames corresponding to each group of actions.

6. The method according to claim 5, characterized in that Analyzing the motion state corresponding to each image frame according to the coordinates of the skeleton points of the test object in each image frame of the motion video includes: According to the coordinates of the skeleton points of the test object in the image frame, determining whether the image frame meets the preset state switching conditions; If the image frame meets the state switching condition, determining that the action state corresponding to the image frame is the action state corresponding to the state switching condition; If the image frame does not meet the state switching condition, determining that the action state corresponding to the image frame is the action state corresponding to the previous image frame of the image frame; Among them, the state switching condition is a condition for switching the action state corresponding to the image frame to an action state different from the action state corresponding to the previous image frame of the image frame, and the action state corresponding to the state switching condition is the action state after the action state corresponding to the image frame is switched according to the state switching condition.

7. The method according to claim 2, characterized in that The step of determining whether each group of actions is compliant based on the action attributes corresponding to each image frame corresponding to each group of actions, and obtaining the normative evaluation result of each group of actions, includes: Determine whether the second group of actions is compliant based on the first compliance analysis result corresponding to the first group of actions and the action attributes corresponding to each image frame of the first action state in the second group of actions, and obtain the normative evaluation result of the second group of actions; Among them, the first compliance analysis result corresponding to the first group of actions is determined by performing a first compliance analysis on the action attributes corresponding to each image frame of the second action state in the first group of actions; the second group of actions is a subsequent group of actions adjacent to the first group of actions, and the image frames corresponding to a group of actions include: image frames corresponding to the first action state and image frames corresponding to the second action state adjacent to the first action state.

8. The method according to claim 7, characterized in that According to the first compliance analysis result corresponding to the first group of actions and the action attributes corresponding to each image frame of the first action state in the second group of actions, whether the second group of actions are compliant is determined, and the normative evaluation result of the second group of actions is obtained, including: Performing a second compliance analysis on the action attributes corresponding to each image frame of the first action state in the second group of actions to obtain a second compliance analysis result corresponding to the second group of actions; The first compliance analysis result corresponding to the first group of actions and the second compliance analysis result corresponding to the second group of actions are used as the normative evaluation results of the second group of actions.

9. The method according to claim 7, characterized in that: The action attributes include: lying flat state; Determining a first compliance analysis result corresponding to the first group of actions by performing a first compliance analysis on the action attributes corresponding to each image frame of the second action state in the first group of actions includes: If the lying flat states of the image frames of the second action state in the first group of actions are all not lying flat, it is determined that the first compliance analysis result corresponding to the first group of actions includes a not lying flat violation.

10. The method according to claim 1, characterized in that Determining the motion evaluation result corresponding to the test object according to the normative evaluation result of each group of movements includes: The actions that do not violate the rules in the normative evaluation results are regarded as normative actions; The number of standard movements in the motion video of the test subject is recorded, and the number of standard movements and the standard evaluation results of each group of movements are used as the motion evaluation results of the test subject.

11. A sports evaluation device, characterized in that: include: An action attribute classification module is used to perform human skeleton point detection on each image frame in a motion video of a test object to obtain the skeleton point coordinates of all candidate objects in each image frame; the skeleton point coordinates of each candidate object in each image frame are compared with the skeleton point coordinates of a skeleton point norm corresponding to each image frame to obtain the skeleton point coordinates of the test object in each image frame; wherein the skeleton point coordinates of the skeleton point norm corresponding to the image frame are determined by sliding average of the skeleton point coordinates of a norm image frame in the motion video; the norm image frame is an image frame in an image frame before the image frame in which the posture of the test object is the same as that of the skeleton point norm; according to the skeleton point coordinates of the test object in each image frame, the test object in each image frame is classified into action attributes to determine the action attributes corresponding to each image frame; the action attributes corresponding to the image frame represent the posture of the test object presented in the image frame; The motion compliance analysis module is used to determine whether each set of motions of the test object during the motion is compliant according to the motion attributes corresponding to each image frame, and obtain the normative evaluation results for each set of motions; The motion evaluation module is used to determine the motion evaluation result corresponding to the test object according to the normative evaluation result of each group of movements.

12. An electronic device, characterized in that: include: Memory and processor; Wherein, the memory is connected to the processor and is used to store programs; The processor is used to implement the motion evaluation method according to any one of claims 1 to 10 by running the program in the memory.

13. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the motion evaluation method according to any one of claims 1 to 10 is implemented.

Images