Motion evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides an exercise evaluation method and device, electronic equipment and a storage medium, and belongs to the technical field of exercise evaluation.The method comprises the steps that a video image of a subject entering a test area is analyzed to achieve identity recognition, and the ready state of the subject is judged according to a preset evaluation mode to start an evaluation process; in the evaluation process, continuously analyzing video images of a subject, extracting human body key points and action attributes, and inputting the human body key points and the action attributes into a preset state machine to perform action compliance evaluation; and outputting an evaluation score of the subject according to a result of the action compliance evaluation. By optimizing resource use, strengthening action monitoring and providing real-time feedback, the problems existing in an existing system are effectively solved, and the overall efficiency of evaluation and the accuracy of user actions are improved.

Description

Technical Field

[0001] The present application relates to the technical field of motion evaluation, and in particular to a motion evaluation method, device, electronic device and storage medium. Background Art

[0002] With the increasing public health awareness, intelligent sports evaluation systems have been widely used in education and daily activities, aiming to provide more efficient, fair and timely sports data collection and performance evaluation. Current sports evaluation systems usually use manual counting, sensor counting and vision-based counting technology to record and evaluate individual sports performance.

[0003] However, most current sports evaluation systems are in single-player mode, which is inefficient when handling multi-player tests, increasing the demand for manpower and equipment. Manual counting is affected by subjective judgments, and sensor counting is expensive and complicated to operate due to special equipment. Although there are vision-based multi-player counting solutions, there is still room for improvement in technical aspects such as key point detection, human recognition, posture estimation, and counting logic. For example, the CN115482580A solution lacks counting functions, the CN116957868A solution relies on image similarity rather than traditional counting logic, the CN118230244A solution counts by analyzing angle changes, and the CN117643718A solution does not specify how to implement its counting logic. Summary of the invention

[0004] The present application provides a motion evaluation method, device, electronic device and storage medium, aiming to solve the problems existing in the existing motion evaluation system in at least one aspect, such as resource consumption, action standard monitoring and real-time feedback of violations, so as to improve the evaluation efficiency and the accuracy of user actions.

[0005] In a first aspect, the present application provides a motion assessment method, the method comprising:

[0006] Analyze the video images of the subject entering the test area to achieve identity recognition, and determine their readiness status according to the preset evaluation mode to start the evaluation process;

[0007] During the evaluation process, the video images of the subjects are continuously analyzed to extract the key points and motion attributes of the human body, and then input them into the preset state machine for motion compliance evaluation;

[0008] According to the results of the action compliance assessment, the test scores of the subjects are output.

[0009] In a second aspect, the present application further provides a motion assessment device, the device comprising:

[0010] The identity detection module is used to analyze the video images of the subject entering the test area to achieve identity recognition, and determine its readiness status according to the preset evaluation mode to start the evaluation process;

[0011] The compliance assessment module is used to continuously analyze the video images of the subjects during the evaluation process, extract the key points and action attributes of the human body, and input them into the preset state machine for action compliance assessment;

[0012] The score output module is used to output the test scores of the subjects according to the results of the action compliance assessment.

[0013] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the steps of the motion assessment method as described in any one of the first aspects are implemented.

[0014] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the motion assessment method as described in any one of the first aspects.

[0015] The present application provides a motion evaluation method, device, electronic device and storage medium, which are intended to solve the problems existing in the existing motion evaluation system in terms of resource consumption, action standard monitoring and real-time feedback of violations. The method realizes identity recognition by analyzing the video image of the subject entering the test area, reducing the large amount of hardware resources required in the traditional motion evaluation system, such as sensors and special equipment, thereby reducing resource consumption. In the evaluation process, the video image of the subject is continuously analyzed, the key points and action attributes of the human body are extracted, and they are input into the preset state machine, which can monitor the subject's actions in real time to ensure that they meet the preset standards, thereby improving the accuracy and real-time performance of action monitoring. According to the changes in the key points of the human body, the action compliance is evaluated, and the subject's evaluation score is output according to the evaluation results. This real-time feedback mechanism enables the subject to immediately understand whether his or her actions are compliant, so as to make timely adjustments, thereby improving the interactivity of the evaluation and the accuracy of the user's actions.

[0016] Therefore, this application effectively solves the problems existing in the existing system by optimizing resource utilization, strengthening action monitoring and providing real-time feedback, thereby improving the overall efficiency of the evaluation and the accuracy of user actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a flow chart of the motion evaluation method provided by the present application;

[0019] Figure 2 is a schematic diagram of the site layout provided in the embodiment of the present application;

[0020] Figure 3 is a schematic diagram of key points of the human body provided in an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a human body posture estimation model provided in an embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of the evaluation process provided in the embodiment of the present application;

[0023] Figure 6 is a schematic diagram of the counting logic provided in the embodiment of the present application;

[0024] Figure 7 It is a structural schematic diagram of the sports evaluation device provided by the present application;

[0025] Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0028] In order to solve the problems existing in the existing system in terms of resource consumption, action standard monitoring, and real-time feedback of violations, the present application provides a motion evaluation method, device, electronic device, and storage medium. The method realizes identity recognition by analyzing the video image of the subject entering the test area, reducing the large amount of hardware resources required by the traditional motion evaluation system, such as sensors and special equipment, thereby effectively reducing resource consumption. In the evaluation process, the system continuously analyzes the video image of the subject, accurately extracts the key points and action attributes of the human body, and inputs them into the preset state machine. This process can monitor the subject's actions in real time to ensure that they meet the preset standards, and improves the accuracy and real-time performance of action monitoring. The action compliance is evaluated according to the changes in the key points of the human body, and the subject's evaluation results are output in real time according to the evaluation results. This real-time feedback mechanism enables the subject to immediately understand whether his or her actions are compliant, so as to make timely adjustments, thereby enhancing the interactivity of the evaluation and the accuracy of the user's actions.

[0029] Therefore, this application not only optimizes resource utilization, strengthens motion monitoring and provides real-time feedback, but is also suitable for multiple sit-up evaluation scenarios, and can effectively improve the overall efficiency of the evaluation and the accuracy of user movements.

[0030] Combine the following Figure 1-Figure 8 The present application describes a motion assessment method, device, electronic device and storage medium.

[0031] Please refer to Figure 1 , Figure 1 The following is a flow chart of a motion evaluation method provided by the present application. A motion evaluation method comprises the following steps:

[0032] S110, analyzing the video image of the subject entering the test area to achieve identity recognition, and judging the subject's readiness status according to a preset evaluation mode to start the evaluation process.

[0033] Specifically, by analyzing the video images of the subject entering the test area, the system is able to identify the subject's identity. This involves using computer vision techniques, such as face recognition or human posture estimation, to identify individuals entering the test area. After identifying the subject, the system will determine whether the subject is ready to start the evaluation based on the preset evaluation mode. For example, check whether the subject is standing in the right position, whether he has taken the correct starting posture, or whether other necessary preparations have been completed. Once the system confirms that the subject is ready, it will start the evaluation process. This means that the system will start collecting data, monitoring the subject's movements, and evaluating according to predetermined criteria.

[0034] S120, during the evaluation process, the video images of the subject are continuously analyzed, key points and motion attributes of the human body are extracted, and they are input into a preset state machine for motion compliance evaluation.

[0035] Specifically, during the evaluation process, the system continuously analyzes the subject's video images and motion attributes, monitors his movements in real time, and ensures that every detail of the evaluation process is captured. The system accurately extracts the key point data of the human body, including joint positions, limb angles, etc., which are the key basis for evaluating the accuracy of the subject's movements. The extracted key points of the human body are then input into a preset state machine.

[0036] The state machine analyzes the input human key points and action attributes, and determines whether the subject's action meets the requirements based on the preset action standards and rules. It checks whether the human key points and action attributes meet specific conditions, such as angle, position, etc., at each state transition to evaluate the compliance of the action. The state machine will comprehensively evaluate the accuracy of the subject's action based on the changes in the human key points to determine whether it meets the evaluation standards. In this way, the state machine can ensure the accuracy and fairness of the evaluation results.

[0037] S130: Output the test subject's test score based on the result of the action compliance assessment.

[0038] Specifically, in the previous step (i.e., S120), the system has performed a compliance assessment on the subject's movements using a state machine by analyzing the subject's video image and the extracted human body key points. This includes determining whether the subject's movements meet predetermined standards or rules. After completing the action compliance assessment, the system generates an evaluation score for the subject based on the results of the assessment. The evaluation score will reflect the subject's performance during the assessment process, including performance in terms of accuracy of the movements. For example, the evaluation score can be presented in the form of a numerical score, a grade, a chart, or other forms so that the subject or the evaluator can intuitively understand the subject's performance.

[0039] The above steps S110 to S130 are described in detail below in conjunction with an embodiment.

[0040] Please refer to Figure 2 , Figure 2 It is a schematic diagram of the site layout provided in the embodiment of the present application. The present application adopts a video acquisition method, specifically a single RGB camera can be used to directly capture the video of multiple subjects in the screen, and there should be no occlusion between the subjects to ensure that the movements of each subject can be clearly captured. Since the sit-up evaluation needs to pay attention to the sitting up action and whether the legs are bent, each subject is required to face the camera sideways. This setting is to ensure that the camera can clearly capture the abdominal and leg movements of the subject.

[0041] The number of testers can be set randomly within the camera shooting area and without blocking the subjects. The system supports setting each test area on the visual panel. For example, if there are three subjects by default, the arrangement of the camera and test area is as follows: Figure 2 If the number of testers needs to be increased, for example, from three to five, the test area can be reset on the visual panel. Five test areas are determined by setting five quadrilaterals to accommodate the increased number of subjects.

[0042] In some embodiments, in S110, the step of analyzing the video image of the subject entering the test area to achieve identity recognition includes:

[0043] S111, input each frame image of the video into a preset human posture estimation model to extract the positions of each human frame and human key points, determine whether the subject has reached the test area through the intersection and union ratio of the human frame position and each test area, and then obtain the face position of the subject who has reached the test area through the corresponding human key points.

[0044] Specifically, the system inputs each frame of the video image of each subject entering the test area into the preset human pose estimation model. The human pose estimation model is a multi-person human pose estimation model that can identify the human body in the image and extract the bounding box (i.e., human body box) and human body key points (such as joint positions) of each human body. The intersection over union (IoU) between each human body box and the predefined test area is calculated. IoU is an indicator to measure the degree of overlap between two areas, and the calculation formula is the intersection area divided by the union area. If the IoU between a human body box and the test area exceeds a certain threshold, the system determines that the subject has reached the test area. Through the human pose estimation model, not only can the key point data of the human body be extracted, but also the face position of the subject who has arrived at the test area can be further determined.

[0045] Please refer to Figure 3 , Figure 3 It is a schematic diagram of key points of the human body provided in an embodiment of the present application. Figure 3 30 key points of the human body are shown, including: top of the head, nose, left ear, right ear, chin, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left palm, right palm, left middle finger, right middle finger, left hip, right hip, left knee, right knee, left ankle, right ankle, left heel, right heel, left toe, right toe, left eye, right eye, left thumb, right thumb, etc. Any one or combination thereof. These key points of the human body cover the main joints and facial features of the human body, which helps to improve the accuracy and comprehensiveness of the analysis of human posture in sports events.

[0046] When labeling key points of multiple people in a scene, due to the huge workload, this application adopts a method that combines machine labeling and manual labeling.

[0047] Method 1:

[0048] Preliminary training: First, the key point information of the single-person moving main target is manually labeled. The key point information includes human key points and human body frames (i.e., rectangular bounding boxes surrounding the human body).

[0049] Human Detection: You can use existing human detection models to detect human frames in images and calculate the IoU between the detected human frames and the manually annotated human frames.

[0050] Key point detection and screening: When the IoU is less than the set threshold, it means that the detected human frame does not match the manually annotated human frame. In this case, the existing single-person key point detection model is used to detect human key points on the detected human frame, and the existing human key point confidence assessment model is used to measure the quality of these key points. In this way, the human frame and human key points of non-moving main targets with higher quality are screened out.

[0051] Method 2:

[0052] Manual labeling: manually label human body frames on completely unlabeled image data. Key point detection and quality assessment: use a single key point detection model to detect human key points on manually labeled human body frames, and also use the human key point confidence assessment model to measure the quality of these human key points. In this way, high-quality multi-person human body frames and human key points are obtained.

[0053] The combination of these two methods aims to improve the efficiency and accuracy of labeling. Method 1 uses existing benchmark data and machine detection to assist in labeling, while method 2 performs manual labeling on new data and combines machine detection to ensure the quality of key points. In this way, the workload of manual labeling can be reduced while ensuring the quality of labeling.

[0054] Please refer to Figure 4 , Figure 4It is a schematic diagram of a human posture estimation model provided in an embodiment of the present application. The architecture of the human posture estimation model includes Backbone (backbone network), PANet (path aggregation network), and Detection head (detection head). The video frame is input to Backbone, which is a basic part in the deep learning model and can be a pre-trained convolutional neural network (such as ResNet, VGG, etc.) for extracting features from the input image. PANet is a network structure used to improve feature propagation and aggregation in feature pyramid networks (FPN). It can help the model better utilize features at different levels and improve the performance of detection and posture estimation. Detection head is responsible for detecting the position and category of the human body, the position and confidence of the key points of the human body from the features extracted by the backbone network. The detection head can include a series of convolutional layers for generating human body frames and category scores, point positions and scores. Therefore, the processing flow of the human posture estimation model includes processing of the input image, feature extraction (through Backbone), further processing and aggregation of features (through PANet), regression detection of human bodies and key points (through Detection head), and finally obtaining the human body frame and human body key points of each human body in the image through conversion post-processing. These components work together to accurately estimate the poses of multiple people from an image.

[0055] The human pose estimation model is based on computer vision technology and adopts an end-to-end architecture design. It can simultaneously identify the positions (human frames) and key points (joints) of all human bodies in an image at one time. Compared with the traditional two-stage strategy, this method has significant advantages. The traditional top-down strategy first detects and frames the human body, and then estimates the pose of a single person in each frame; while the bottom-up strategy first detects all key points and then assembles these key points into the poses of multiple people. In contrast, the one-time end-to-end human pose estimation model achieves higher processing efficiency and real-time performance by directly jointly detecting the human frame and human key points, and is suitable for scenarios that require fast and accurate processing of large-scale human data.

[0056] Specifically, each anchor (i.e., predefined reference box) in the Detection head predicts a human bounding box (used to locate the human body in the image), target confidence (indicating the probability of whether the bounding box predicted by the anchor contains a human body), category confidence (involving the category of the person, indicating the probability that the predicted bounding box contains a human body), and 30 key point information (i.e., predicted human key point coordinates and confidence scores). NMS (Non-Maximum Suppression) is then performed on the human bounding box, that is, overlapping bounding boxes are removed. When multiple bounding boxes that may belong to the same person are detected, NMS selects the one with the highest confidence and suppresses other overlapping boxes. Finally, the information of each human body, i.e., the human body box and human key points, is output as follows:

[0057]

[0058] The first 4 columns (x1, y1, x2, y2) represent the coordinates of the human body frame, which are the coordinates of the upper left and lower right corners of the bounding box. Immediately following the coordinates of the human body frame are the coordinates of 30 key points that define the posture of the human body.

[0059] Therefore, the detection head predicts the human body bounding box and key points through each anchor, then uses NMS to remove overlapping boxes, and finally outputs a formatted result, which includes the coordinates of the human body box and key points of each human body. This output format is convenient for subsequent processing and analysis, such as posture estimation, action recognition, etc.

[0060] S112, comparing the facial image with the records in the subject database to achieve identity recognition.

[0061] Specifically, after obtaining the face position, the system will compare the extracted face image with the record in the subject database. The database contains the facial feature information of the registered subjects. Through the comparison, the system can identify the subject's identity and ensure that the identity verification during the evaluation process is accurate.

[0062] After the multi-person posture estimation task is completed, the system has identified all individuals in the scene. However, these individuals may include many non-test subjects. At the same time, it is also an important consideration to ensure that multiple testers do not interfere with each other during the evaluation process. Therefore, in the multi-person evaluation system of this application, in order to ensure that each test point can be evaluated smoothly, this application also provides a human body matching solution. This solution mainly relies on the accurate identification and positioning of the human body frame during the matching process to ensure the independence of each tester and the accuracy of the evaluation. Through this solution, testers can be effectively distinguished from non-testers, and the smooth progress of the test process can be ensured. The following are two solutions, which can be selected according to hardware and actual test needs:

[0063] Solution 1:

[0064] This scheme is a rule-based matching scheme suitable for fixed-point movements where the subject's movements remain basically unchanged, such as sit-ups, sit-ups, etc. This scheme identifies the subject in each test area, and its identification steps include:

[0065] S113, inputting the video image of each subject entering the test area into a preset human posture estimation model to output a human body frame of each subject.

[0066] S114, for each test area, calculating the ratio of the overlapping portion between the human body frame of each subject and its test area to the entire test area.

[0067] Specifically, each test area is allocated according to the following formula:

[0068]

[0069] Among them, bbox 人体 Represents the detected human body frame, bbox 测试区域 Represents the test area frame, for example, the threshold in sit-ups can be set to 0.2 by default. It means that when the detected human frame accounts for greater than or equal to 20% of the test area, the human frame is considered to belong to this test area. This threshold is set to ensure that when the subject performs sit-ups, even if part of the body exceeds the test area, it will not be mistakenly excluded.

[0070] S115, if the ratio is greater than or equal to a preset threshold, the subject is assigned to the test area to identify the subject in each test area.

[0071] For example, when the proportion of the detected human body frame in the test area is greater than or equal to 0.2, the object is assigned to the test area.

[0072] If a test area is assigned multiple human frames (for example, it may be because the presser foot person also enters the test area), the system will perform further processing:

[0073] S116, when a certain test area is assigned with multiple human body frames, the Euclidean distance between the center point of each object and the center point of the test area is calculated in the horizontal direction.

[0074] S117, selecting the subject closest to the center point of the test area and confirming it as the subject of the test area.

[0075] That is, the system determines which individual is closest to the center of the test area by calculating the distance, thereby determining that the individual is the correct subject to participate in the test.

[0076] Therefore, the purpose of the above S113 to S117 is to assign the detected human body frame to the correct test area to ensure that only one subject is identified in each test area. And when dealing with the situation where multiple people enter the same test area, the correct subject can be selected. For example, when a non-tester (such as a foot presser) accidentally enters the test area, the system can accurately identify and confirm the real test subject by calculating the distance. In this way, the system ensures the accuracy and reliability of the test.

[0077] Option 2:

[0078] This solution uses image-based matching technology and is suitable for identifying moving objects in dynamic scenes. The specific implementation steps are as follows:

[0079] Initial object assignment step: According to the test area matching strategy, an initial test object is assigned to each test area. The corresponding human cropped images are obtained by locating the human frame, and the features of these images are extracted using the similarity perception model. These features are then stored in a dedicated feature library (memory bank).

[0080] Real-time feature matching step: During the video stream processing, each frame is processed by the multi-person pose estimation model, and the human body frame and cropped image of each object are output. These cropped images are input into the similarity perception model in turn to extract real-time image features. By calculating the distance between the features extracted from the current frame and the features stored in the feature library, it is determined whether each object should belong to the corresponding test area.

[0081] Continuous matching and isolation steps: Through the continuous human body matching process, ensure that the human body frame and its key point information of each test area are accurately matched, and maintain the independence of each test area in the entire evaluation and counting process to avoid mutual interference.

[0082] Through the above-mentioned initial object allocation step, real-time feature matching step, and continuous matching and isolation step, the system can efficiently and accurately manage object matching in dynamic scenes to ensure the smooth progress of the evaluation process.

[0083] Please refer to Figure 5 , Figure 5 Schematic diagram of the evaluation process provided by the embodiment of the present application. The evaluation mode includes a fixed time evaluation mode and a non-fixed time evaluation mode.

[0084] In the fixed-time evaluation mode, the system will confirm that all subjects have entered their respective test areas and that everyone has completed the preset preparation movements (for example, lying on their side and ready to start sit-ups, the system announces "Please get ready"). Once all subjects are ready, the system will start a unified timer (for example, the system announces "Start", indicating that the timer has started), marking the official start of the evaluation process. The subjects then begin to do sit-ups, and the system will continue to monitor and record their performance until the timer reaches the preset time end (for example, the system announces "Time is up"), at which time the system will stop the evaluation and output the results of each subject.

[0085] In the non-fixed time evaluation mode, the system will confirm that a subject has entered any test area and completed the preset preparation action, and then start counting the subject independently to start the evaluation process for the subject. This mode allows subjects to enter the test area and start the evaluation at their own pace, without waiting for all subjects to start synchronously. The system will time each subject entering the test area individually, and output the subject's score immediately when the subject leaves the test area or the timing ends. This flexible evaluation process design is suitable for groups of subjects who cannot start the evaluation at the same time.

[0086] In some embodiments, in S120, in the evaluation process, the step of continuously analyzing the video image of the subject and extracting key points of the human body includes:

[0087] S121, processing the video images of the subject frame by frame, and inputting each frame of the image into a human posture estimation model to extract the human body key points and human body frame of each subject.

[0088] Specifically, the human pose estimation model can extract the human key points and human body frames of each subject in one go and end-to-end.

[0089] S122, inputting the key points of the human body of each subject into a preset motion attribute classification model to obtain the motion attributes of each subject.

[0090] Specifically, the motion attribute classification model aims to identify and output the subject's current motion characteristics, which may include the following aspects: whether the knees are touching, whether the subject is lying flat, whether the knee flexion angle is within the normal range (such as approximately between 65° and 135°), whether the hands are holding the head, and whether the subject is sitting facing left or right during the test.

[0091] The action attribute classification model is a multi-layer perceptron network model. The input data of the model is processed as follows: First, the 30 key points of the human body are normalized using the thigh length and the hip center point, and then the key point information is straightened into a 60-dimensional vector. The middle layer of the model is set to 64 dimensions for feature extraction and conversion. The output layer is a 7-dimensional vector, corresponding to the classification of the above action attributes.

[0092] In terms of the choice of loss function, for the four binary attributes of touching the knee, lying flat, holding the head, and orientation, binary cross entropy loss (BCE Loss) is used for supervised learning. For the three-category attribute of knee flexion angle (including knee flexion angle too large, normal, and too small), cross entropy loss (CE Loss) is used for supervision.

[0093] Through the design of the above-mentioned action attribute classification model, the action attributes of the subjects can be effectively classified to ensure the accuracy and reliability of the evaluation results. In addition, if the subject's action does not meet the preset standards or rules (i.e., violations), the system will make a report, that is, notify the relevant personnel in some way (which can be sound, text prompts, etc.), pointing out that there is a problem with the subject's action and that it needs to be adjusted or corrected. Such a report function helps to monitor and guide the subjects in real time to ensure that they perform the test according to the correct actions.

[0094] S123, inputting the key points and action attributes of each subject into a state machine, which defines a ready state, a lying state and a sitting state, wherein the switching between the lying state and the sitting state depends on the key inflection points determined by the key points of the human body.

[0095] S124, determining key inflection points based on the input key point data and action attributes, and making compliance judgments based on them, and outputting compliance judgment results and counting results.

[0096] Specifically, the key inflection points include the sit-up inflection point and the lying-down inflection point. When the subject sits up from a lying-down state to a speed less than a preset threshold, the state machine counts the minimum body bending angle and determines the sit-up inflection point based on this angle. When the subject lies down from a sitting-up state to a speed less than a preset threshold, the state machine counts the maximum body bending angle and determines the lying-down inflection point based on this angle; based on the determined sit-up inflection point and lying-down inflection point, the state machine performs compliance judgment in combination with the action attributes, and outputs the compliance judgment result and the counting result; the compliance judgment result is used to confirm whether the subject's action meets the evaluation criteria, and the counting result is counted based on the compliant action switching to obtain the number of sit-ups performed by the subject.

[0097] Please refer to Figure 6 , Figure 6 It is a schematic diagram of the counting logic provided in the embodiment of the present application. Figure 6 The counting logic flow of inputting each group of human key points and action attributes into the corresponding state machine and entering the multi-person sit-up evaluation system is shown as follows:

[0098] First, images are extracted from the video stream, and the human key point information of each subject is identified through the human posture estimation model. Then, the detected human key point information and its corresponding human frame are put into a queue and matched with the corresponding test area queue. The human key point information matched to the area queue is input into the action attribute classification model, which outputs the action attributes of the object in each area. These action attributes are also put into a queue and matched with the area queue. The updated human key points and updated action attributes are input into the state machine. The state machine is responsible for state judgment and counting of each area (such as area 1, area 2, area 3, etc.). Specifically including:

[0099] In the ready state, the system checks whether the subject has completed the ready action and the action is in compliance. If it is in compliance, the subject enters the lying state. In the lying state, the system detects whether the subject touches the knee and whether there is any violation. If the subject touches the knee and there is no violation, the count is increased by one. The transition between the lying state and the sitting state is determined by the lying inflection point and the sitting inflection point. In the sitting state, the system detects whether the subject is lying flat.

[0100] Furthermore, for the state machine counting logic process of the multi-person sit-up evaluation system, the specific steps are as follows:

[0101] Step 1: Prepare for motion detection.

[0102] The system first enters the initial cycle to detect whether the test subject's preparation movements meet the standards. Once the preparation movements are confirmed to be compliant, the system will switch to the lying state and start a new supine-sitting cycle.

[0103] Process 2: Sit-up status detection.

[0104] During the supine-sit-up cycle, the system continuously loops to detect the sit-up keyframes and determines whether the subject touches the knees through the motion attribute classification model. When the sit-up keyframe is detected and there is no violation from the beginning of the cycle to the current moment (compliance is determined through motion attributes and key points), the system adds one to the count. In addition, the system also loops to detect the sit-up inflection point. If the sit-up inflection point is detected but the sit-up keyframe is not detected or there is a violation, it is not counted and the system switches to the sit-up state.

[0105] Process 3: Supine state detection.

[0106] The system continues to detect the supine keyframes in a loop and determines whether the object is lying flat through the motion attribute classification model. At the same time, the system also detects the lying flat inflection point in a loop. If the lying flat inflection point is detected but the supine keyframe is not detected, the system will record the non-lying flat state and start the next supine-sitting cycle, while archiving the non-lying flat information and then switching back to the lying flat state.

[0107] Execute process 2 and process 3 in a loop.

[0108] Process 4: Output.

[0109] After the counting is completed, the system will output the relevant information of each cycle, including the time from the start (lying flat) to sitting up and then to the end (lying flat), the sign of whether the lying flat touches the knee, the number of violations, whether the cycle is counted, etc. Based on this information, the system can obtain the video slice of a certain cycle for counting verification, violation review, or counting the frequency of each violation, thereby providing a comprehensive test evaluation.

[0110] Through the above counting logic process, the system can accurately track the action status of the objects in each test area and count when specific conditions are met, thereby realizing the automated evaluation of multiple people's sit-up actions.

[0111] In summary, the motion evaluation method provided by the present application is different from the traditional bottom-up scheme and top-down scheme. This method jointly detects the human body frame and human body key points in the entire test area, and then processes them through a matching scheme based on the test area. No additional human body detection steps are required, thereby saving computing resources and improving evaluation efficiency. In addition, the state machine in the present application will archive key information during the counting process. For example, when the user is not completely lying down or touching the knees, the system can capture and record the key frames closest to completely lying down or touching the knees, so that the user can view and analyze them after the test. Finally, by combining the action attribute classification judgment and the state transition of the state machine, the system can detect and broadcast violations such as legs not straightened, heads not held, and feet off the ground in real time, which helps the subjects to correct their actions in time during the test and ensure the accuracy and effectiveness of the test.

[0112] The motion evaluation device provided in the present application is described below. The motion evaluation device and the motion evaluation apparatus described below and the motion evaluation method described above can be referenced to each other.

[0113] Please refer to Figure 7 , Figure 7 A sports evaluation device 700 includes an identity detection module 710 , a compliance assessment module 720 , and a performance output module 730 .

[0114] Exemplarily, the identity detection module 710 is used to analyze the video image of the subject entering the test area to achieve identity recognition, and determine the readiness state according to the preset evaluation mode to start the evaluation process.

[0115] Exemplarily, the compliance assessment module 720 is used to continuously analyze the video images of the subject during the assessment process, extract the key points and motion attributes of the human body, and input them into a preset state machine to perform motion compliance assessment.

[0116] Exemplarily, the score output module 730 is used to output the test subject's assessment score according to the result of the action compliance assessment.

[0117] Exemplarily, the identity detection module 710 is further used to:

[0118] Input each frame of the video into a preset human pose estimation model to extract the positions of each human frame and human key points, determine whether the subject has reached the test area through the intersection and union ratio of the human frame position and each test area, and then obtain the face position of the subject who has reached the test area through the corresponding human key points;

[0119] The facial image is compared with the records in the subject database to achieve identity recognition.

[0120] Exemplarily, the identity detection module 710 is further used to:

[0121] In the fixed-time evaluation mode, after confirming that all subjects have entered the test area and completed the preset preparation actions, start the timer to start the evaluation process;

[0122] In the non-fixed time evaluation mode, after confirming that a subject has entered the test area and completed the preset preparation actions, the counting of the subject begins, thereby starting the evaluation process.

[0123] Exemplarily, the compliance assessment module 720 is further configured to:

[0124] Process the subject's video images frame by frame, and input each frame into the human posture estimation model to extract the human body key points and human body frame of each subject;

[0125] The human body key points of each subject are input into a preset motion attribute classification model to obtain the motion attributes of each subject.

[0126] Exemplarily, the compliance assessment module 720 is further configured to:

[0127] Input the key points and action attributes of each subject into a state machine, which defines a ready state, a lying state, and a sitting state, wherein the switching between the lying state and the sitting state depends on the key inflection points determined by the key points of the human body;

[0128] Based on the input key point data and action attributes, the key inflection points are determined, and compliance judgments are made accordingly, and compliance judgment results and counting results are output.

[0129] Exemplarily, the compliance assessment module 720 is further configured to:

[0130] When the subject sits up from a lying position until the moving speed is less than a preset threshold, the state machine counts the minimum body bending angle and determines the sitting-up inflection point based on this angle;

[0131] When the subject lies flat from a sitting position to a position where the moving speed is less than a preset threshold, the state machine counts the maximum body bending angle and determines the lying flat inflection point based on this angle;

[0132] Based on the determined sitting-up inflection point and lying-down inflection point, the state machine performs compliance judgment in combination with the action attributes, and outputs the compliance judgment result and the counting result; the compliance judgment result is used to confirm whether the subject's action meets the evaluation criteria, and the counting result is counted according to the compliant action switching to obtain the number of sit-ups performed by the subject.

[0133] Exemplarily, the identity detection module 710 is further used to:

[0134] Input the video image of each subject entering the test area into a preset human posture estimation model to output a human body frame of each subject;

[0135] For each test area, calculate the proportion of the overlap between each subject's body frame and its test area to the entire test area;

[0136] If the ratio is greater than or equal to a preset threshold, the subject is assigned to the test area to identify the subject in each test area.

[0137] Exemplarily, the identity detection module 710 is further used to:

[0138] When a test area is assigned multiple human frames, the Euclidean distance between the center point of each object and the center point of the test area is calculated in the horizontal direction;

[0139] The subject closest to the center point of the test area is selected and confirmed as the subject of the test area.

[0140] Exemplarily, the key points of the human body include any one of the following or a combination thereof:

[0141] Top of the head, nose, left ear, right ear, chin, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left palm, right palm, left middle finger, right middle finger, left hip, right hip, left knee, right knee, left ankle, right ankle, left heel, right heel, left toe, right toe, left eye, right eye, left thumb, right thumb.

[0142] It should be noted here that the above-mentioned motion assessment device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0143] Figure 8 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 8 As shown, the electronic device may include: a processor (Processor) 810, a communication interface (Communications Interface) 820, a memory (Memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the motion evaluation method.

[0144] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to ensure that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0145] On the other hand, the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the motion assessment method provided by the above methods.

[0146] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the motion evaluation methods provided above.

[0147] The embodiments of the present application provide an electronic device, a computer program product, and a processor-readable storage medium, on which the computer program stored enables the processor to implement all the method steps implemented in the above-mentioned method embodiments and to achieve the same technical effects. The parts and beneficial effects of this embodiment that are the same as those in the method embodiments will not be described in detail herein.

[0148] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0149] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions to ensure that a computer device (which can be a personal computer, a server, or a network device, etc.) executes the methods described in each embodiment or some parts of the embodiments.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A motion evaluation method, characterized in that: The method comprises: Analyze the video images of the subject entering the test area to achieve identity recognition, and determine their readiness status according to the preset evaluation mode to start the evaluation process; During the evaluation process, the video images of the subjects are continuously analyzed to extract the key points and motion attributes of the human body, and then input them into the preset state machine for motion compliance evaluation; According to the results of the action compliance assessment, the test scores of the subjects are output.

2. The motion evaluation method according to claim 1, characterized in that: The step of analyzing the video image of the subject entering the test area to achieve identity recognition includes: Input each frame of the video into a preset human pose estimation model to extract the positions of each human frame and human key points, determine whether the subject has reached the test area through the intersection and union ratio of the human frame position and each test area, and then obtain the face position of the subject who has reached the test area through the corresponding human key points; The facial image is compared with the records in the subject database to achieve identity recognition.

3. The motion evaluation method according to claim 1, characterized in that: The evaluation mode includes a fixed-time evaluation mode and a non-fixed-time evaluation mode. The step of determining the readiness state according to the preset evaluation mode to start the evaluation process includes: In the fixed-time evaluation mode, after confirming that all subjects have entered the test area and completed the preset preparation actions, start the timer to start the evaluation process; In the non-fixed time evaluation mode, after confirming that a subject has entered the test area and completed the preset preparation actions, the counting of the subject begins, thereby starting the evaluation process.

4. The motion evaluation method according to claim 1, characterized in that: The step of continuously analyzing the video image of the subject and extracting key points of the human body comprises: Process the subject's video images frame by frame, and input each frame into the human posture estimation model to extract the human body key points and human body frame of each subject; The human body key points of each subject are input into a preset motion attribute classification model to obtain the motion attributes of each subject.

5. The motion evaluation method according to claim 4, characterized in that: The step of inputting it into a preset state machine for action compliance evaluation includes: Input the key points and action attributes of each subject into a state machine, which defines a ready state, a lying state, and a sitting state, wherein the switching between the lying state and the sitting state depends on the key inflection points determined by the key points of the human body; Based on the input key point data and action attributes, the key inflection points are determined, and compliance judgments are made accordingly, and compliance judgment results and counting results are output.

6. The motion evaluation method according to claim 5, characterized in that: The key inflection points include a sitting-up inflection point and a lying-down inflection point, and the step of inputting them into a preset state machine to perform action compliance evaluation includes: When the subject sits up from a lying position until the moving speed is less than a preset threshold, the state machine counts the minimum body bending angle and determines the sitting-up inflection point based on this angle; When the subject lies flat from a sitting position to a position where the moving speed is less than a preset threshold, the state machine counts the maximum body bending angle and determines the lying flat inflection point based on this angle; Based on the determined sitting-up inflection point and lying-down inflection point, the state machine performs compliance judgment in combination with the action attributes, and outputs the compliance judgment result and the counting result; the compliance judgment result is used to confirm whether the subject's action meets the evaluation criteria, and the counting result is counted according to the compliant action switching to obtain the number of sit-ups performed by the subject.

7. The motion evaluation method according to claim 1, characterized in that: The method further includes identifying a subject in each test area, wherein the identifying step comprises: Input the video image of each subject entering the test area into a preset human posture estimation model to output a human body frame of each subject; For each test area, calculate the proportion of the overlap between each subject's body frame and its test area to the entire test area; If the ratio is greater than or equal to a preset threshold, the subject is assigned to the test area to identify the subject in each test area.

8. The motion evaluation method according to claim 7, characterized in that: The method also includes processing a situation where multiple people are assigned to a test area, and the processing steps include: When a test area is assigned multiple human frames, the Euclidean distance between the center point of each object and the center point of the test area is calculated in the horizontal direction; The subject closest to the center point of the test area is selected and confirmed as the subject of the test area.

9. The motion evaluation method according to claim 1, characterized in that: The key points of the human body include any one of the following or a combination thereof: Top of the head, nose, left ear, right ear, chin, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left palm, right palm, left middle finger, right middle finger, left hip, right hip, left knee, right knee, left ankle, right ankle, left heel, right heel, left toe, right toe, left eye, right eye, left thumb, right thumb.

10. A sports evaluation device, characterized in that: The device comprises: The identity detection module is used to analyze the video images of the subject entering the test area to achieve identity recognition, and determine its readiness status according to the preset evaluation mode to start the evaluation process; The compliance assessment module is used to continuously analyze the video images of the subjects during the evaluation process, extract the key points and action attributes of the human body, and input them into the preset state machine for action compliance assessment; The score output module is used to output the test scores of the subjects according to the results of the action compliance assessment.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the motion evaluation method according to any one of claims 1 to 10 are implemented.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the motion evaluation method according to any one of claims 1 to 10 are implemented.

Images