Exercise evaluation method and device and exercise evaluation model training method and device

Abstract

The invention provides an exercise evaluation method and device and an exercise evaluation model training method and device. The exercise evaluation method comprises the steps of obtaining a to-be-tested image; and based on the exercise evaluation model, simulating a decision-making thought of a referee for evaluating each to-be-tested behavior through the evaluation thinking chain, and reasoning by applying semantic information of the to-be-tested image to determine an exercise evaluation result. According to the method, the decision-making thought of a referee for evaluating various to-be-tested behaviors is simulated through the evaluation thinking chain, and the semantic information of the to-be-tested image is used for reasoning, so that intelligent identification and accurate penalty of complicated movement violation behaviors are realized, and the judgment level close to that of human referees is achieved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a motion assessment, a motion assessment model training method and apparatus. Background Technology

[0002] In throwing sports such as shot put and medicine ball, accurately identifying violations is crucial for improving training effectiveness and ensuring the fairness of refereeing decisions. Common violations include interference, leaving the throwing area, stepping on the line, jumping, and throwing with one or two hands.

[0003] Currently, motion assessment often employs a combination of single technologies such as keypoint recognition, target detection, facial recognition, and action recognition. This involves evaluating violations such as interference from personnel, leaving the throwing area, stepping on the line, jumping, and throwing technique one by one. However, this method is susceptible to complex factors such as short-term occlusion, motion blur, changes in lighting, and environmental interference, leading to reduced accuracy. Furthermore, the combination of multiple algorithms can easily result in error accumulation, reducing recognition efficiency. Summary of the Invention

[0004] This invention provides a method and apparatus for motion assessment, motion assessment model training, and other methods to address the deficiencies in existing technologies.

[0005] This invention provides a method for assessing physical activity, comprising the following steps: Acquire the image to be tested; Based on the motion assessment model, the decision-making process of the referee in assessing various behaviors is simulated through the assessment thinking chain, and the semantic information of the image to be tested is used for reasoning to determine the motion assessment result.

[0006] According to a motion assessment method provided by the present invention, the step of simulating the referee's decision-making process for assessing various behaviors by means of an assessment thought chain, and applying semantic information of the image to be tested for reasoning to determine the motion assessment result includes: Based on the decision-making approach for each behavior to be tested, the semantic information of the image to be tested is applied to perform motion evaluation on each behavior to be tested, and the evaluation results of each behavior to be tested are determined. The motion assessment results are determined by integrating the assessment results of various behaviors to be tested.

[0007] According to a motion assessment method provided by the present invention, the step of acquiring the image to be tested includes: Obtain the video to be tested; Select multiple key video frames from the video to be tested that contain both the preparatory and completion actions of the movement; Based on the semantic information of each key video frame, the start video frame and the end video frame are determined from each key video frame. The time corresponding to the start video frame is the start time of the motion preparation action, and the time corresponding to the end video frame is the end time of the motion completion action. The image to be tested is determined from the video to be tested based on the starting video frame and the ending video frame.

[0008] According to a motion assessment method provided by the present invention, the step of acquiring the image to be tested includes: If a command to start video recording is detected, video recording is initiated. Upon detecting an instruction to end video recording, the video recording is terminated; The video frames recorded between the start and end times of video recording are used as the image to be tested.

[0009] This invention provides a method for training a motion assessment model, comprising the following steps: Acquire sample images; Based on the model under the current iteration, the evaluation thinking chain is used to simulate the decision-making process of the judge in evaluating various behaviors to be tested, and the semantic information of the sample images is used to make inferences to determine the evaluation reasoning results. Based on the difference between the evaluation reasoning result and the actual evaluation result of the sample image, the reward value for the current iteration is determined; With the goal of maximizing the reward value in the current iteration, the model parameters in the current iteration are updated, and the inference steps are returned until the preset convergence condition is met. Then, the model in the current iteration with updated parameters is used as the motion evaluation model.

[0010] According to a method for training a motion assessment model provided by the present invention, determining the reward value for the current iteration based on the difference between the assessment inference result and the actual assessment result of the sample image includes: Based on the difference between the evaluation reasoning result and the actual evaluation result of the sample image, the result reward value under the current iteration is determined; Based on the process text generated by the model during the inference process in the current iteration, the process reward value in the current iteration is determined. The reward value for the current iteration is determined based on the result reward value and the process reward value for the current iteration.

[0011] According to a method for training a motion assessment model provided by the present invention, determining the reward value for the current iteration based on the result reward value and the process reward value for the current iteration includes: Based on the result reward value and the process reward value under the current iteration, the preliminary reward value under the current iteration is determined; Based on the initial reward value of the previous iteration and the initial reward value of the current iteration, the smoothed reward value of the current iteration is determined; The reward value for the current iteration is determined based on the initial reward value and the smoothed reward value for the current iteration.

[0012] According to a method for training a motion assessment model provided by the present invention, determining the reward value for the current iteration based on the initial reward value and the smoothed reward value for the current iteration includes: Based on the preliminary reward values ​​of each sample image in the current iteration, the average reward value in the current iteration is determined; The reward value for each sample image in the current iteration is determined based on the mean reward value in the current iteration, the preliminary reward value for each sample image in the current iteration, and the smoothed reward value in the current iteration.

[0013] The present invention also provides a motion assessment device, comprising the following modules: The first acquisition unit is used to acquire the image to be tested; The motion assessment unit is used to simulate the decision-making process of a referee in assessing various behaviors based on a motion assessment model and by applying the semantic information of the image to be tested to infer the motion assessment result.

[0014] The present invention also provides a motion assessment model training device, comprising the following modules: The second acquisition unit is used to acquire sample images; The reasoning unit is used to simulate the decision-making process of the judge in evaluating various behaviors under test based on the model in the current iteration through the evaluation thinking chain, and to use the semantic information of the sample image to perform reasoning to determine the evaluation reasoning result. The determining unit is used to determine the reward value in the current iteration based on the difference between the evaluation reasoning result and the actual evaluation result of the sample image; The training unit is used to update the model parameters in the current iteration with the goal of maximizing the reward value in the current iteration, and return to execute the inference steps until a preset convergence condition is met. Then, the model in the current iteration with updated parameters is used as the motion evaluation model.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the motion assessment methods or motion assessment model training methods described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the motion assessment method or motion assessment model training method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described motion assessment methods or motion assessment model training methods.

[0018] The present invention provides a sports assessment, a sports assessment model training method, and a device. By simulating the decision-making process of a referee in assessing various test behaviors through an assessment thought chain, and applying semantic information from the test images for reasoning, it achieves intelligent identification and accurate judgment of complex sports violations, reaching a level of judgment approaching that of human referees. Because the assessment thought chain integrates the experience, knowledge, and rule logic of human referees into the model in a structured manner, the model can perform multi-step, evidence-based reasoning, rather than simply feature matching. Therefore, when faced with complex violations that involve a degree of subjective judgment or visual ambiguity, the sports assessment model can conduct in-depth analysis and logical judgment based on the simulated decision path, yielding more accurate and interpretable assessment results. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the motion assessment method provided by the present invention.

[0021] Figure 2 This is a flowchart illustrating the implementation of step 120 in the motion assessment method provided by the present invention.

[0022] Figure 3 This is one of the flowcharts illustrating the implementation of step 110 in the motion assessment method provided by the present invention.

[0023] Figure 4 This is a second flowchart illustrating the implementation of step 110 in the sports assessment method provided by the present invention.

[0024] Figure 5 This is a flowchart illustrating the training method for the motion assessment model provided by the present invention.

[0025] Figure 6 This is a flowchart illustrating the implementation of step 530 in the motion assessment model training method provided by the present invention.

[0026] Figure 7 This is a schematic diagram of process supervision and result supervision in reinforcement learning provided by the present invention.

[0027] Figure 8 This is a schematic diagram of the motion assessment device provided by the present invention.

[0028] Figure 9 This is a schematic diagram of the structure of the motion assessment model training device provided by the present invention.

[0029] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] Currently, most sports assessments combine key point recognition, target detection, facial recognition, and motion recognition technologies, evaluating violations such as interference, leaving the throwing area, stepping on the line, jumping to throw, and throwing method.

[0032] For example, in assessing violations such as interference, leaving the throwing area, and stepping on the line, facial recognition algorithms are typically used first to accurately identify the thrower and potential interferers. After confirming their identities, human posture key point detection algorithms are used to accurately obtain the coordinate data of key points on the entire body of both the thrower and the interferer. Through in-depth analysis of this key point information, especially the comparison of the positional relationship between the foot key points and the boundary of the throwing area, it is determined whether the thrower has engaged in interference, whether they have left the designated area during the throw, or whether they have stepped on the line. For assessing violations related to jump throws and throwing techniques, target detection algorithms are typically used first to accurately locate the thrower in complex scenes and segment them from the background. Subsequently, motion recognition algorithms are used to meticulously analyze the thrower's continuous motion sequence during the throw, capturing the motion features of their arms, body, and legs to determine whether the thrower has engaged in violations such as one-handed throwing or jump throws.

[0033] However, the aforementioned methods require breaking down complex motion assessment tasks into multiple basic algorithms, such as object detection, keypoint detection, face recognition, and action recognition. These basic algorithms are susceptible to factors such as short-term occlusion, motion blur, changes in lighting, and environmental interference, leading to decreased recognition accuracy. Furthermore, since multiple basic algorithms need to be combined to make a comprehensive judgment on violations, errors may accumulate, further reducing recognition efficiency.

[0034] To address this, the present invention provides a method for assessing physical fitness (such as endurance running and standing long jump), strength training (such as pull-ups), and throwing events (such as shot put). For ease of understanding, the following embodiments are all illustrated using the application to assessing violations in throwing events.

[0035] in, Figure 1 This is a flowchart illustrating the motion assessment method provided by the present invention, as shown below. Figure 1 As shown, the method includes steps 110 and 120.

[0036] Step 110: Obtain the image to be tested.

[0037] Here, the image to be tested refers to visual data used for sports evaluation, which can clearly capture the athlete's movements, postures, and relative positional relationship with the field environment. It can be a single static image frame captured by fixed or mobile camera equipment (such as high-definition cameras, smartphones, drones, etc.), or a continuous video stream (such as from a real-time monitoring system or recording equipment), or fused image data combining multiple perspectives and multiple sensors (such as depth information provided by an RGB-D camera). This embodiment of the invention does not specifically limit this.

[0038] In order to improve analysis efficiency and focus on key behaviors, after acquiring continuous video streams, the start and end times of key movements of the target athlete can be determined by preprocessing algorithms such as motion detection and behavior recognition or by manual annotation. Based on these start and end times, a sequence of images to be tested is extracted from the video stream. This ensures that there is no redundant information in the image sequence that is not related to the behavior being tested, thus avoiding increasing the processing burden of subsequent motion assessment models and reducing recognition accuracy.

[0039] For shot put, after capturing the entire video stream from the athlete's preparation to the completion of the throw, a human posture key point detection algorithm can be used to monitor the amplitude or speed of changes in the thrower's arm, torso, or legs to identify the starting moment of the throw preparation. For example, when the athlete begins to bend their knees and squat or swings their arms backward to build up power (marking the formal start of the throwing motion), this can be identified as the starting moment of the completed throw. Alternatively, by setting up a monitoring area, the starting moment can be determined when a part of the athlete's body (such as the toes) first enters the throwing area or begins a clear preparatory movement.

[0040] Furthermore, target detection and action recognition algorithms can be combined to identify the end moment of the throw. For example, the moment the medicine ball leaves the athlete's hand (which can be determined by detecting whether the ball separates from the hand's target frame), or when the athlete's arm completes its maximum extension and begins the retraction motion, can be identified as the end moment of the throw. For fouls committed after the throw, such as stepping on the line, the end moment can be slightly extended to when the athlete has completed the throw but has not yet left the throwing area.

[0041] By using the above method, video clips containing only the key throwing actions of the thrower can be accurately extracted to form a sequence of images to be tested, such as a sequence of video frames of the throwing process with a duration of 2-5 seconds. This ensures that there is no redundant information such as the athlete walking, waiting, or background activities unrelated to the throwing in the image sequence, thus avoiding sending a large amount of irrelevant data into the subsequent evaluation model. This effectively reduces the computational burden of the model and avoids interference caused by redundant information, thereby improving the accuracy of the final determination of violations.

[0042] Step 120: Based on the motion assessment model, simulate the decision-making process of the referee in assessing various behaviors through the assessment thinking chain, and use the semantic information of the image to be tested to reason and determine the motion assessment result.

[0043] Specifically, the assessment mindset chain refers to a structured decision-making process used to simulate the referee's decision-making approach when assessing various behaviors to be tested. These behaviors refer to key actions or situations that may occur during throwing sports and require judgment and evaluation according to the rules. These behaviors may include personnel interference, personnel leaving the area, personnel stepping on the line, one-handed throwing, and jumping throwing, among others.

[0044] The decision-making process for each behavior to be tested can be understood as a complete logical chain for identifying, analyzing, and ultimately judging specific violations. The decision-making process can include the violation identification logic and the corresponding judgment criteria. The identification logic is used to indicate how to judge whether a specific behavior has occurred, its manner of occurrence, and its extent through visual information (such as posture, position, and trajectory). The judgment criteria are the rules for determining the nature of the identified behavior (such as whether it is a violation, the degree of violation, etc.) and giving the corresponding results based on the established rules of throwing motion.

[0045] Regarding interference from personnel, the identification logic in the corresponding decision-making approach could be as follows: Using target detection and personnel tracking algorithms, detect whether there are other personnel besides the thrower inside or outside the throwing area. Furthermore, use behavior recognition algorithms to analyze whether the behavior of these non-thrower personnel has a potential impact on the thrower's actions, throwing trajectory, or the field environment. For example, they might perform their own throwing actions during the thrower's throw (outside-bounds interference) or engage in undesignated activities within the throwing area (inside-bounds interference). The penalty criteria in the decision-making approach could be: based on the "personnel interference" clause in the sports assessment rules, determine whether the behavior of other personnel interferes with the thrower's normal assessment process or affects the fairness of the assessment results. For example, "when the thrower is throwing a shot put towards the throwing area, a student standing outside the boundary on their left also throws a shot put towards the throwing area almost simultaneously," this is outside-bounds interference; "another student within the throwing area moves away from the camera to retrieve a basketball from a distance," this is inside-bounds interference. Any external or internal personnel activity that may affect the accuracy, fairness, or raise suspicion of cheating will be penalized as personnel interference violation.

[0046] Regarding personnel leaving the throwing area, the identification logic in the corresponding decision-making approach could be: through key point detection of human posture and area judgment, real-time monitoring is conducted to ensure that any part of the thrower's body (especially the feet) completely exceeds the preset throwing area boundary (the parallelogram area formed by the two straight lines in front and behind and the four dots above and below). The penalty basis in the decision-making approach could be: according to the provisions of the solid ball throwing rules regarding "personnel leaving the throwing area," if any part of the thrower's body leaves the designated throwing area before the effective throw is completed or before the thrown ball lands, it is judged as a violation of personnel leaving the throwing area. For example, "during the evaluation process, the thrower never left the throwing area," then it is judged that there was no violation of personnel leaving the throwing area.

[0047] For instances of personnel stepping on the line, the identification logic in the corresponding decision-making approach could be: through the detection of key points in the human posture and the geometric relationship analysis with the boundary lines of the field, to accurately determine whether the thrower's feet (or other body parts) touched, stepped on, or crossed any boundary lines of the throwing area during or after the throw. The penalty criteria in this decision-making approach could be: according to the "stepping on the line" rule in the shot put rules, if the thrower's feet or other body parts step on or cross the boundary line of the throwing area during the throw or before the ball lands, it is considered a violation of the line-stepping rule. For example, if "the thrower's feet are both on the back line of the throwing area," it is considered a violation of the line-stepping rule.

[0048] For one-handed throws, the identification logic in the corresponding decision-making process could be: through motion recognition and hand target detection, at the critical moment when the thrower releases the ball from their hand, detect and analyze the number of hands in contact with the ball. Specifically, the monitoring point is the contact state between the hand and the ball before it leaves the body. The penalty criterion in the decision-making process could be: according to the "throwing method" rule in the shot put rules, the throw must be completed with both hands. If, at the critical moment when the ball leaves the body, only one hand contacts and throws the ball, it is penalized as a one-handed throw violation. For example, "using both hands to throw the ball during the throw" would be considered as not having engaged in a one-handed throw.

[0049] For jump throws, the identification logic in the corresponding decision-making process could be: through key point detection of human posture and analysis of ground contact status, monitor whether the thrower's feet simultaneously lose contact with the ground during the entire throwing process, i.e., whether a jumping motion occurs. The penalty criterion in the decision-making process could be: according to the "jump throw" rule in the shot put rules, both feet cannot leave the ground simultaneously during the throw. If the thrower's feet leave the ground simultaneously and complete the throwing motion during the throw, it is considered a jump throw violation. For example, if "both feet did not leave the ground simultaneously," it is judged that there is no violation of a two-foot jump throw.

[0050] Furthermore, the semantic information of the image under test refers to the high-level, abstract meaning and contextual information contained within it. This information is used to characterize entities (such as people, balls, and playing fields), attributes (such as posture, position, and speed), actions (such as throwing, jumping, and walking), and relationships between them (such as the relationship between a person and a ball, or between a person and the playing field) related to motion assessment. As an optional implementation, pre-trained deep learning models (such as object detection, pose estimation, and behavior recognition models) or multimodal learning methods can be used to extract the semantic information of the image under test.

[0051] It should be noted that traditional methods require breaking down complex motion assessment tasks into multiple basic algorithms, which are susceptible to factors such as short-term occlusion, motion blur, lighting changes, and environmental interference, leading to decreased recognition accuracy. Furthermore, the combination of multiple basic algorithms may result in error accumulation, reducing recognition efficiency. This invention, through an assessment thought chain, simulates the decision-making process of a referee evaluating various behaviors, enabling comprehensive judgment of complex violations, improving the accuracy and robustness of penalties, and effectively addressing the limitations of traditional methods. Simultaneously, by applying the semantic information of the image under test, a high-level, structured visual input is provided to the assessment thought chain, enabling it to understand the scene and perform logical reasoning like a human referee, achieving automated motion assessment that more closely approximates the judgment level of a human referee.

[0052] In other words, the embodiments of the present invention organically integrate the traditionally separate visual perception task with rule-based logical judgment, constructing a unified motion evaluation framework that simulates the cognitive process of human referees. It creatively introduces the concept of "evaluation thinking chain" and uses the semantic information of the image to be tested to support its reasoning, thereby achieving more intelligent and robust identification and punishment of violations.

[0053] Furthermore, in terms of hardware, the embodiments of the present invention can complete motion assessment by acquiring images to be tested using ordinary cameras (such as RGB cameras), which reduces hardware costs and is easy to deploy and promote, enabling motion assessment to be more widely applied to various scenarios without the need for expensive professional equipment.

[0054] The aforementioned motion assessment model can be built based on a large language model, or it can be based on a multimodal large model or a Transformer architecture model specifically designed for vision-language tasks. This embodiment of the invention does not specifically limit this. However, considering the powerful capabilities of large language models in understanding complex text instructions, performing logical reasoning, integrating multi-source information (such as visual semantics and rule text), and generating interpretable results, this embodiment of the invention preferably builds the motion assessment model based on a large language model. This allows for better simulation of the referee's decision-making process, enabling flexible understanding of rules and reasoning judgments in complex scenarios, and providing more interpretable assessment results.

[0055] The sports assessment method provided in this invention simulates the decision-making process of a referee in assessing various behaviors through an assessment thought chain. It also applies semantic information from the images to be assessed for reasoning, achieving intelligent identification and accurate judgment of complex sports violations, reaching a level of judgment close to that of human referees. Because the assessment thought chain integrates the experience, knowledge, and rule logic of human referees into the model in a structured manner, the model can perform multi-step, evidence-based reasoning, rather than simply feature matching. Therefore, when faced with complex violations that involve a degree of subjective judgment or visual ambiguity, the sports assessment model can conduct in-depth analysis and logical judgment based on the simulated decision path, yielding more accurate and interpretable assessment results. This avoids the misjudgments or inability to provide reasonable explanations that may occur with traditional black-box models or single algorithms when dealing with complex rules, abnormal situations, or violations involving multiple factors, thus improving the accuracy of the assessment results.

[0056] Based on the above embodiments, Figure 2 This is a flowchart illustrating the implementation of step 120 in the motion assessment method provided by the present invention, as shown below. Figure 2 As shown, step 120 simulates the decision-making process of a referee evaluating various behaviors through an assessment thinking chain, and uses the semantic information of the image to be tested for reasoning to determine the motion assessment results, including: Step 121: Based on the recognition logic of each behavior to be tested and combined with the semantic information of the image to be tested, determine the specific manifestation of each behavior to be tested from the image to be tested. Step 122: Based on the penalty criteria for each behavior to be tested, conduct compliance analysis and violation determination on the specific manifestation of each behavior to be tested, and determine the evaluation results for each behavior to be tested; Step 123: Integrate the assessment results of each behavior to be tested to determine the exercise assessment results.

[0057] Specifically, the decision-making approach for each test behavior refers to a complete set of identification logic and penalty criteria defined for each specific violation (such as interference, crossing the line, or one-handed throwing). This framework guides the sports assessment model in automatically judging and adjudicating individual test behaviors. When assessing each test behavior, following the decision-making approach and combining it with the semantic information of the test image, the model uses the identification logic defined in the decision-making approach (e.g., analyzing semantic information such as key points on the athlete's body and object positions) to determine the specific manifestation of the behavior. Based on the penalty criteria (e.g., determining a violation if specific conditions are met), it performs compliance analysis and violation judgment on the specific representation, arriving at the assessment result for that behavior. This allows for accurate identification and automated adjudication of each individual violation, ensuring the objectivity and consistency of the judgment. The specific manifestation refers to the specific action characteristics, positional relationships, and time sequence of the behavior to be tested in the image to be tested. For example, the configuration of the athlete's key body points, movement trajectory, relative position to the boundary of the throwing area, contact state between the hand and the ball, and other related behavioral characteristics. These manifestations reflect the occurrence of the behavior and its likelihood of violation.

[0058] The evaluation results of each test behavior are used to characterize the judgment conclusion and details (such as violation type, time of occurrence, and involved parties) of whether each specific test behavior (e.g., "personnel interference" or "personnel crossing the line") is a violation. The sports evaluation result is the final overall sports performance assessment or a summary report of violations, derived by integrating the evaluation results of all test behaviors. Therefore, after obtaining the evaluation results of each test behavior, the results of each test behavior are integrated to obtain the sports evaluation result.

[0059] As an optional implementation, the evaluation results of various behaviors to be tested can be integrated and sorted out through preset fusion rules (e.g., prioritizing according to the severity of violations, or summarizing and explaining multiple violations) or by utilizing the natural language generation capabilities of a large language model, to form a clear and easy-to-understand comprehensive evaluation report or directly give the final judgment conclusion, thereby obtaining the sports evaluation results.

[0060] Therefore, the embodiments of the present invention, by introducing an evaluation thinking chain to simulate the referee's decision-making process for each behavior to be tested, and by applying the semantic information of the image to be tested for reasoning, achieve accurate and comprehensive judgment of each behavior to be tested in complex motion scenarios, and can integrate the evaluation results of multiple behaviors to be tested to obtain the final motion evaluation result.

[0061] Based on any of the above embodiments Figure 3 This is one of the flowcharts illustrating the implementation of step 110 in the motion assessment method provided by the present invention, as shown below. Figure 3As shown, step 110 acquires the image to be tested, including: Step 111a: Obtain the video to be tested; Step 112a: Select multiple key video frames from the video to be tested that contain the preparatory movements and the completion movements of the movements; Step 113a: Based on the semantic information of each key video frame, determine the start video frame and the end video frame from each key video frame. The time corresponding to the start video frame is the start time of the motion preparation action, and the time corresponding to the end video frame is the end time of the motion completion action. Step 114a: Determine the image to be tested from the video to be tested based on the start video frame and the end video frame.

[0062] Specifically, the test video refers to a continuous image sequence that fully records the motion assessment process. It can be a full-process video recording captured by a fixed camera device, or a training video shot by a mobile camera device (such as a smartphone or drone) within a specific time period. After acquiring the test video, preprocessing can be performed, such as resolution adjustment, frame rate conversion, or noise reduction, to avoid excessively large video data or poor image quality affecting the efficiency and accuracy of subsequent processing.

[0063] The preparatory movements for exercise refer to a series of actions performed by athletes before they formally execute core movements, in order to accumulate energy, adjust posture, and adapt to the environment. Key video frames containing these preparatory movements are those that clearly show the athlete beginning these preparatory postures or movements. For shot put, the corresponding preparatory movements could be the athlete standing with both hands holding the ball, squatting down, or swinging their arms backward to build up momentum. Key video frames containing these preparatory movements can be frames that capture the athlete's body beginning to shift its center of gravity or the limbs beginning to move in the opposite direction of the throwing motion to accumulate kinetic energy.

[0064] The completion of a movement refers to the body's inertia or recovery of balance that usually accompanies an athlete after completing a core movement. Key video frames containing the completion of a movement are those that clearly show the athlete's body's follow-up and recovery posture after completing the core movement (such as the ball leaving the hand). For medicine ball throwing, the corresponding completion of the movement could be the arm's forward thrust after the medicine ball leaves the athlete's hand, the body leaning forward with the ball, or the cushioning action after the athlete has stabilized. Key video frames containing the completion of the movement could be those capturing the moment the medicine ball leaves the hand and the athlete's body's inertia and gradual recovery to stability after the throw.

[0065] Considering that the test video contains a large amount of redundant information unrelated to the core motion, such as scenes of athletes entering, waiting, resting, or leaving the field, directly using all video frames from the test video as the test image could significantly increase the computational load and processing time of the subsequent motion assessment model. Furthermore, it could introduce irrelevant background interference, leading to low assessment efficiency and even affecting the accuracy of recognition. For example, a 5-minute training video might only contain a few seconds to tens of seconds of actual throwing motion; feeding all 5 minutes of frames into the model would be a huge waste of computational resources.

[0066] Based on this, after acquiring the video to be tested, the embodiments of the present invention filter out multiple key video frames containing the preparatory movements and the completion movements of the movement from the video to be tested, thereby effectively focusing on the core movement process of the athlete, significantly reducing the amount of data to be processed, improving the efficiency of data processing and the targeting of subsequent model inference.

[0067] As an optional embodiment, motion detection algorithms can be used to identify regions or time periods with significant motion changes in the video, or a pre-trained behavior recognition model can be used to perform preliminary classification of the video content and filter out video segments related to "motion preparation" and "motion completion" in order to obtain multiple key video frames containing motion preparation and motion completion actions from the video to be tested.

[0068] It should be noted that the key video frames selected here may still contain non-critical moments in the preparation or completion phase of the movement, or there may be temporal redundancy. For example, the "preparation" phase may start from the moment the athlete walks to the throwing line, but the actual "preparation action" is only a few seconds before the throw; the "completion" phase may include a long buffer period after the ball lands, but the "completion action" is limited to the moment the ball leaves the hand and the brief follow-up motion thereafter.

[0069] To further pinpoint the precise start and end points of the core motion process and eliminate irrelevant frames to the maximum extent, this embodiment of the invention uses semantic information from each key video frame to select and determine the start and end video frames. The start video frame corresponds to the beginning of the motion preparation action, and the end video frame corresponds to the end of the motion completion action.

[0070] As an optional embodiment, a fine-grained motion recognition model or key point trajectory analysis can be used, combined with pre-defined semantic rules (such as "the rate of change of the distance between the hand and the ball reaches a threshold" to determine the moment the ball leaves the hand), to accurately identify the frames representing the start of the motion preparation action and the end of the motion completion action from each key video frame, and mark them as the start video frame and the end video frame, and select the start video frame and the end video frame from each key video frame.

[0071] After determining the start and end video frames, the segment consisting of all video frames between the start and end video frames in the video under test precisely contains the key process of the athlete's core movement. Therefore, this sequence of video frames between the start and end video frames in the video under test can be used as the image under test.

[0072] Based on any of the above embodiments Figure 4 This is a second flowchart illustrating the implementation of step 110 in the motion assessment method provided by this invention, as shown below. Figure 4 As shown, step 110 acquires the image to be tested, including: Step 111b: Upon detecting a command to start video recording, initiate video recording; Step 112b: If a command to end video recording is detected, end video recording; Step 113b: Use the video frames recorded between the start and end times of video recording as the image to be tested.

[0073] Specifically, the instruction to start video recording refers to a signal or operation that triggers the camera device to begin recording a video stream. Upon detecting this instruction, video recording is triggered, i.e., video recording begins. This instruction can be generated by the user pressing a physical or virtual record button, or it can be triggered by voice commands, gesture recognition, a preset timer, or a specific sensor signal (such as a motion sensor detecting an athlete entering a designated area). This embodiment of the invention does not specifically limit the specific type of instruction.

[0074] A command to end video recording refers to a signal or operation that triggers the camera device to stop recording the video stream. Upon detecting this command, the video is saved or the recording process is stopped, thus ending the video recording. This command can be generated by the user pressing a physical or virtual stop recording button, or it can be transmitted via voice command, reaching a preset recording duration, the athlete completing a specific action, or leaving a designated area. This embodiment of the invention does not impose specific limitations on these methods.

[0075] The video recording start time refers to the precise point in time when the recording start command is detected and video data capture actually begins. The video recording end time refers to the precise point in time when the recording end command is detected and video data capture stops. The video frames recorded between these two points completely record all visual information from the start to the end of the recording, representing the entire process that the user considers to contain the target motion event. They can be directly used as a motion image sequence to be analyzed and evaluated without additional cropping or filtering steps. Therefore, the video frames recorded between these two points are used as the images to be tested.

[0076] Based on any of the above embodiments Figure 5 This is a flowchart illustrating the training method for the motion assessment model provided by the present invention, as shown below. Figure 5 As shown, the method includes: Step 510: Obtain sample images.

[0077] Specifically, sample images refer to the dataset used to train the motion assessment model. These images or video clips contain a variety of motion scenarios and behaviors, and these sample images have been labeled with the actual assessment results (i.e., true labels or ground truth values) for each behavior to be tested. They comprehensively reflect the various compliant and non-compliant actions to be tested and their contextual information.

[0078] The sample images can be obtained in the following ways: representative frames or segments are extracted from a large number of real motion video recordings and manually labeled frame by frame; or, data augmentation techniques (such as rotation, scaling, brightness adjustment, adding noise, etc.) are used to expand the existing dataset to increase the model's generalization ability and robustness.

[0079] Considering the potential class imbalance in the sample data (e.g., violations are usually less common than compliance) and the interference factors such as occlusion, blurring, and lighting changes that may exist in complex real-world environments, the sample images can be preprocessed after acquisition. This includes operations such as unifying resolution, color correction, and background removal. Data balancing can also be performed (e.g., oversampling rare classes or undersampling common classes) to ensure that the model can fully learn the features of various behaviors during training, avoiding overfitting or underfitting, thereby improving the model's recognition accuracy and generalization ability.

[0080] Step 520: Based on the model in the current iteration, simulate the decision-making process of the judge in evaluating each behavior to be tested through the evaluation thinking chain, and use the semantic information of the sample images to make inferences to determine the evaluation reasoning results.

[0081] Specifically, the model in the current iteration refers to the version of the model currently used for inference and evaluation after one or more rounds of parameter updates during training. The evaluation thought process refers to a structured decision-making process used to simulate the decision-making approach of a referee in evaluating various behaviors to be tested. The decision-making approach for each behavior to be tested can be understood as a complete logical chain for identifying, analyzing, and ultimately issuing a penalty for a specific violation.

[0082] The current iteration of the model simulates the decision-making process of a referee in evaluating various test behaviors through an assessment thought chain. This allows the model to systematically analyze and evaluate complex and diverse sports behaviors according to preset logical rules and judgment criteria, rather than simply performing pattern recognition. This improves the model's logicality and interpretability in judging behaviors. Furthermore, by combining semantic information from sample images for reasoning, the model can incorporate high-level abstract features extracted from images (such as the athlete's posture, the position of objects, and the trajectory of movement) into the decision-making logic of the assessment thought chain for in-depth analysis and judgment, achieving a leap from perception to cognition and obtaining assessment reasoning results.

[0083] Among them, the evaluation inference result is the automatic judgment output of the current model on each test behavior in the sample image. It details the model's identification conclusion for each potential violation behavior, including whether it is a violation, the type of violation, and the possible degree of violation or related evidence information (for example, for the behavior of stepping on the line, it may indicate which frame the foot crossed the line).

[0084] Step 530: Determine the reward value for the current iteration based on the difference between the evaluation reasoning result and the actual evaluation result of the sample image.

[0085] Specifically, the difference between the evaluation inference result and the actual evaluation result of the sample image is used to characterize the accuracy or error of the model's judgment on a specific motion behavior in the current iteration. The larger the difference, the greater the deviation of the model's current judgment from the reality, the worse its performance, and the more adjustments are needed. The reward value in the current iteration is a quantified feedback signal used to evaluate the model's performance on the sample data in the current iteration and guide the update direction of the model parameters, optimizing it towards more accurate judgments. It can be determined by comparing the consistency between the evaluation inference result and the actual evaluation result. For example, a positive reward can be given when the judgment is correct, and a negative reward or no reward can be given when the judgment is incorrect. For each behavior to be tested (such as human interference, crossing the line, etc.), if the model's inference result is completely consistent with the actual evaluation result of the sample image, a positive reward (such as +1 point) is given to the model; if the inference result is inconsistent with the actual evaluation result (i.e., a missed judgment or misjudgment occurred), a negative reward (such as -1 point) is given. The final reward value can be the sum of the reward values ​​obtained for all the tested behaviors, or a weighted sum based on the importance of each violation, thereby comprehensively evaluating the overall performance of the model in this iteration.

[0086] Step 540: With the goal of maximizing the reward value in the current iteration, update the model parameters in the current iteration and return to execute the inference steps until the preset convergence condition is met. Then, use the model in the current iteration with updated parameters as the motion evaluation model.

[0087] Specifically, a larger reward value indicates a closer match between the model's evaluation and inference results in the current iteration and the actual evaluation results of the sample image, meaning the model performs better and its judgments are more accurate. When aiming to maximize the reward value in the current iteration, algorithms such as policy gradient methods, group relative policy optimization (GRPO), and proximal policy optimization (PPO) in reinforcement learning (RL) can be used. These algorithms enable the model to adjust its decision-making strategy and internal parameters based on the reward signal, gradually learning how to more accurately simulate the referee's decision-making process and extracting correct judgment criteria from the image's semantic information. Specifically, when aiming to maximize the reward value in the current iteration, the policy gradient (i.e., the derivative of the reward value with respect to the model parameters) can be calculated and combined with optimization algorithms such as gradient descent (e.g., Adam, SGD) to update the model parameters in the current iteration.

[0088] After updating the model parameters for the current iteration, the model with updated parameters can be used as the model for the current iteration, and the process returns to step 520 until a preset convergence condition is met. Then, the model with updated parameters for the current iteration is used as the motion evaluation model. The preset convergence condition may include reaching the maximum number of training iterations, the model's performance metrics on the validation set (such as accuracy and F1 score) no longer improving for multiple consecutive rounds (i.e., early stopping mechanism), and the reward value converging to a certain threshold or its rate of change being less than a preset value.

[0089] The sports assessment model training method provided in this invention deeply integrates a multimodal large model with a reinforcement learning paradigm, aiming to maximize reward values ​​and iteratively optimize the model's ability to simulate referee decision-making through the assessment thought chain. Because this method overcomes the limitations of traditional single algorithms, the model can fully utilize multimodal data and contextual information in sample images for deep understanding and reasoning, thereby achieving accurate understanding and dynamic analysis of the entire throwing motion scenario and significantly improving the accuracy and efficiency of violation identification.

[0090] The model in the current iteration can be a multimodal large model built on Intern3VL, capable of acquiring 10,000 sample images. 5,000 sample images are used for model training to obtain the motion assessment model. The remaining 5,000 sample images are used as test images to score the model. For each test image, the model, based on its internal assessment thought chain and reasoning about the semantic information of the image, outputs assessment inference results for various possible behaviors to be tested in that image (such as interference from people, people leaving, people stepping on lines, one-handed throwing, jumping throwing, etc.). Subsequently, these inference results are precisely compared with the pre-labeled "actual assessment results" (i.e., ground truth) of the 5,000 test images. If the model's judgment on a violation (e.g., whether "people stepping on lines" exists) is completely consistent with the actual result, then that judgment receives one point; if the judgment is inconsistent, no points are awarded. Finally, by summing up the correct judgment scores for each behavior on all test images and dividing by the total number of judgments, the overall accuracy of the model can be calculated. For example, the model achieves an accuracy of 98% in recognizing "person stepping on the line", 95% in recognizing "jumping and throwing", and a comprehensive recognition accuracy of 96% for all violations. This quantitatively evaluates the model's performance.

[0091] Based on any of the above embodiments Figure 6 This is a flowchart illustrating the implementation of step 530 in the motion assessment model training method provided by the present invention, as shown below. Figure 6 As shown, step 530 determines the reward value for the current iteration based on the difference between the evaluation reasoning result and the actual evaluation result of the sample image, including: Step 531: Based on the difference between the evaluation reasoning result and the actual evaluation result of the sample image, determine the result reward value for the current iteration; Step 532: Based on the process text generated by the model during inference in the current iteration, determine the process reward value for the current iteration; Step 533: Determine the reward value for the current iteration based on the result reward value and the process reward value for the current iteration.

[0092] Specifically, the reward value for the current iteration refers to the numerical value used to quantify the degree of consistency between the model's final judgment (i.e., the evaluation reasoning result) and the true label (i.e., the actual evaluation result of the sample image) for each test behavior in the current iteration. It is used to characterize the accuracy of the model at the final decision level.

[0093] The reward value for the current iteration is determined based on the difference between the evaluation inference result and the actual evaluation result of the sample image. In other words, the reward value guides the model to directly optimize its final judgment, making it as consistent as possible with reality. This can be achieved by setting a binary or multi-class reward function. For example, a positive reward (e.g., +1) is given when the model's inference result for a certain behavior perfectly matches the actual evaluation result; otherwise, a negative reward (e.g., -1) or zero reward is given. For multiple behaviors to be tested, the reward value for each behavior can be accumulated or weighted to determine the reward value for the current iteration.

[0094] The process reward value in the current iteration refers to the quality of the process text (i.e., the reasoning steps of the thought chain) generated by the model during the reasoning process, including its logic, completeness, consistency with the rules, and the degree of effective use of image semantic information. It is used to characterize the rationality and interpretability of the model in the decision-making process and reasoning path.

[0095] The process reward value in the current iteration is determined based on the process text generated by the model during inference. In other words, the process reward value guides the model to generate more persuasive reasoning processes that align with human judges' thought processes, thereby improving the model's reliability and transparency. This can be achieved by designing a reward model based on natural language processing. This model can evaluate the similarity between the generated process text and pre-defined "ideal" or "expert" process text (e.g., using ROUGE or BLEU scores), or assess its logical coherence, the degree of mention of key rule terms, and whether it correctly references image semantic information to support judgment. Alternatively, the process text can be quality-scored through manual annotation or crowdsourcing, and the score can be used as a reward signal to determine the process reward value in the current iteration.

[0096] The process text refers to the natural language description of a series of intermediate reasoning steps, logical judgments, and supporting evidence generated by the model during its reasoning process based on the assessment thought chain in order to arrive at the final assessment result. For example, when assessing the behavior of "person interference," the process text could be: "Two non-throwing personnel were detected in the image. One of them simultaneously made a throwing motion outside the boundary while the throwing personnel were throwing the ball, constituting interference from outside the boundary. The other person was retrieving the ball within the throwing area, affecting the assessment order and constituting interference from inside the boundary." In this embodiment of the invention, the final reward value is determined by combining the result reward value and the process reward value, thereby simultaneously optimizing the model's final judgment accuracy (result) and the rationality of its reasoning process (process). In other words, the model's process and result are subject to dual constraints and incentives, ensuring that the motion assessment model can not only give the correct judgment result, but also provide clear reasoning basis that conforms to human cognition, which greatly improves the model's robustness, interpretability and credibility in practical applications.

[0097] in, Figure 7 This is a schematic diagram of process supervision and result supervision in reinforcement learning provided by the present invention, as shown below. Figure 7 As shown, by combining result supervision (such as...) Figure 7 The accuracy assessment of the model's final penalty result is indicated by "×" and "√" to mark whether the result is correct or not, and process supervision is also included. Figure 7 The constraints on the rationality of each step of the model's reasoning process (marked with "√") not only ensure the accuracy of the final evaluation result but also enhance the rationality of the model's reasoning process, making the model output more credible, reducing misjudgments and unreasonable reasoning, thereby achieving efficient and accurate automation of motion evaluation.

[0098] Based on any of the above embodiments, the reward value for the current iteration is determined based on the result reward value and the process reward value for the current iteration, including: Based on the result reward value and the process reward value of the current iteration, determine the preliminary reward value for the current iteration; Based on the initial reward value of the previous iteration and the initial reward value of the current iteration, determine the smoothed reward value of the current iteration; The reward value for the current iteration is determined based on the initial reward value and the smoothed reward value for the current iteration.

[0099] Specifically, the preliminary reward value refers to the original comprehensive reward signal calculated by the model for each sample image in the current training iteration, based on the difference between its evaluation inference result and the actual evaluation result (reflected in the result reward value) and the quality of the inference process (reflected in the process reward value). It is used to measure the model's current performance. It is determined based on the result reward value and the process reward value in the current iteration. That is, the preliminary reward value is an immediate feedback after the model evaluates the current sample from two dimensions: "what it did" (final judgment) and "how it did it" (inference process). It is usually a weighted sum or other combination of the result reward value and the process reward value, aiming to comprehensively reflect the model's comprehensive performance on a specific sample in this iteration.

[0100] The initial reward value for the current iteration can be determined based on the following formula: in, Indicates the first The initial reward value for each sample image. Represents a sample image. This represents the evaluation reasoning results output by the model. This indicates the actual evaluation result. Represents process text, This indicates a question entered by the user (e.g., the user enters "Is there human interference?"). Indicates model output and The accuracy score of the conclusion (i.e., the result reward value). Indicates model output and process text The reasoning process score (i.e., process reward value).

[0101] For the accuracy score of the conclusion, if the output is... and If it does not match, then a score of 0 can be given; if the output is... and If the results match, a score of 1 is awarded. For the reasoning process, if the process text contains reasoning errors, a score of 0 is awarded; if the process text contains no reasoning errors, a score of 1 is awarded. Furthermore, if all reasoning in the process text is correct and the output... and If the inference matches, the initial reward is 10 points; if all reasoning in the process text is incorrect and the output is... and If it does not match, the initial reward value is 0 points.

[0102] The smoothed reward value in the current iteration refers to the reward signal calculated by combining the model performance of the previous and current iterations. This signal is used to smooth reward fluctuations and stabilize the training process. It characterizes the performance trend and consistency of the model in continuous iterations, helping to reduce reward fluctuations caused by data noise or occasional errors in a single iteration. The smoothed reward value in the current iteration is determined based on the preliminary reward value of the previous iteration and the preliminary reward value of the current iteration. For example, the KL divergence can be determined based on the preliminary reward value of the previous and current iterations, and then used as the smoothed reward value in the current iteration.

[0103] After obtaining the smoothed reward value for the current iteration, the reward value for the current iteration is determined by combining the initial reward value and the smoothed reward value for the current iteration. This not only effectively alleviates the drastic fluctuations in the reward signal and improves the stability of the training process, but also guides the model to improve its immediate performance while taking into account the continuous optimization of long-term performance and generalization ability. This ensures that the trained motion evaluation model has higher robustness, accuracy and inference consistency, and is more suitable for complex and ever-changing real-world motion evaluation scenarios.

[0104] Based on any of the above embodiments, the reward value for the current iteration is determined based on the initial reward value and the smoothed reward value for the current iteration, including: Based on the preliminary reward values ​​of each sample image in the current iteration, determine the average reward value in the current iteration; The reward value for each sample image in the current iteration is determined based on the mean reward value in the current iteration, the preliminary reward value for each sample image in the current iteration, and the smoothed reward value in the current iteration.

[0105] Specifically, the preliminary reward value for the next sample image in the current iteration refers to the raw, unsmoothed, and unweighted reward value obtained by directly comparing the model's evaluation inference result with the actual evaluation result of that sample image for each independent sample image in the current training iteration. It is used to characterize the model's immediate performance and accuracy on a single sample. The mean reward in the current iteration refers to the arithmetic mean of the preliminary reward values ​​for all sample images in the current training iteration. It is used to characterize the overall performance level or recognition trend of the model in the current batch or iteration.

[0106] If the initial reward value of any sample image is higher than the average reward, it indicates that the model's recognition performance of that sample image is better than the average level of the current batch, and its judgment is more accurate and reliable. At this time, the weight of the corresponding sample image can be increased so that when the model parameters are updated, the correct features and decision logic learned by the model on these well-performing samples can be emphasized and consolidated, thereby further improving the overall generalization ability and accuracy of the model, or prompting the model to learn these patterns that are "easy" to obtain high rewards more actively.

[0107] If the initial reward value of any sample image is lower than the average reward value, it indicates that the model's recognition performance for that sample image is lower than the average level of the current batch. The model may have difficulty in judgment, bias, or the sample may have high noise. In this case, the weight of the corresponding sample image can be reduced to reduce the impact of these "difficult samples" or "abnormal samples" that may mislead the model or cause unstable training on the overall reward signal. This will prevent the model from overfitting to individual bad samples or noise, thereby making the training process more stable and improving the robustness of the model.

[0108] The average reward in the current iteration can be determined based on the following formula: in, Indicates the first The mean reward for each sample image. Indicates the first The initial reward value for each sample image. Indicates the number of samples. This represents the average value. It represents the standard deviation.

[0109] The mean reward in the current iteration provides an overall reference benchmark or average performance level for the initial reward values ​​of all samples in the current training batch. The initial reward value of each sample image measures the model's immediate, raw performance on a single sample, reflecting the accuracy and rationality of its judgment process on a specific sample. The smoothed reward value in the current iteration provides a more stable and less noisy reward signal. By combining past and current initial reward information, it effectively reduces the impact of fluctuations in the reward value of a single sample on model updates, ensuring training stability and helping the model learn longer-term optimization directions. Combining these three factors to determine the reward value of each sample image in the current iteration enables an adaptive and dynamic sample weight adjustment mechanism and a reward signal smoothing strategy.

[0110] Specifically, by comparing the reward with the mean, samples that perform "well" or "poorly" can be identified. For samples that perform "well" (with initial reward values ​​higher than the mean), their influence in training can be appropriately increased to solidify the correct patterns learned by the model. For samples that perform "poorly" (with initial reward values ​​lower than the mean), their influence can be reduced to prevent the model from being misled by the noise of these difficult or abnormal samples. Introducing a smoothed reward value further ensures the stability of this weight adjustment and the overall reward signal. It prevents the model from overreacting to short-term, drastic reward fluctuations in each training round, instead guiding the model towards more stable, long-term improvement.

[0111] Wherein, the reward value of the next sample image in the current iteration. Determined based on the following formula: in, This represents the initial reward value from the previous iteration. This represents the initial reward value in the current iteration. Denotes KL divergence, It is a hyperparameter that controls the cutting range. It is a hyperparameter of the KL divergence penalty strength, used to constrain the magnitude of model updates.

[0112] The following describes the motion assessment device provided by the present invention. The motion assessment device described below can be referred to in correspondence with the motion assessment method described above.

[0113] Based on any of the above embodiments Figure 8 This is a schematic diagram of the structure of the motion assessment device provided by the present invention, as shown below. Figure 8 As shown, the device includes: The first acquisition unit 810 is used to acquire the image to be tested; The motion assessment unit 820 is used to simulate the decision-making process of the referee in assessing various behaviors based on the motion assessment model and through the assessment thinking chain, and to use the semantic information of the image to be tested to reason and determine the motion assessment result.

[0114] Based on any of the above embodiments Figure 9 This is a schematic diagram of the structure of the motion assessment model training device provided by the present invention, as shown below. Figure 9 As shown, the device includes: The second acquisition unit 910 is used to acquire sample images; The reasoning unit 920 is used to simulate the decision-making process of the judge in evaluating various behaviors based on the model under the current iteration through the evaluation thinking chain, and to use the semantic information of the sample image to perform reasoning to determine the evaluation reasoning result. Unit 930 is used to determine the reward value in the current iteration based on the difference between the evaluation reasoning result and the actual evaluation result of the sample image. Training unit 940 is used to update the model parameters in the current iteration with the goal of maximizing the reward value in the current iteration, and then return to execute the inference steps until the preset convergence condition is met. After that, the model in the current iteration with updated parameters is used as the motion evaluation model.

[0115] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 10 As shown, the electronic device may include a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040. The processor 1010, communications interface 1020, and memory 1030 communicate with each other via the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute a motion assessment method. This method includes: acquiring a test image; simulating the decision-making process of a referee evaluating various test behaviors based on a motion assessment model and using the semantic information of the test image for reasoning to determine the motion assessment result.

[0116] Alternatively, a method for training a motion assessment model can be implemented, comprising: acquiring sample images; based on the model in the current iteration, simulating the decision-making process of a referee in assessing various behaviors through an assessment thought chain, and applying the semantic information of the sample images to perform reasoning to determine the assessment reasoning result; determining the reward value in the current iteration based on the difference between the assessment reasoning result and the actual assessment result of the sample images; updating the model parameters in the current iteration with the goal of maximizing the reward value in the current iteration, and returning to the inference step until a preset convergence condition is reached, and then using the model in the current iteration with updated parameters as the motion assessment model.

[0117] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the motion assessment method provided by the above methods. The method includes: acquiring an image to be tested; based on a motion assessment model, simulating the decision-making process of a referee to assess various behaviors to be tested through an assessment thought chain, and applying the semantic information of the image to be tested for reasoning to determine the motion assessment result.

[0119] Alternatively, a method for training a motion assessment model can be implemented, comprising: acquiring sample images; based on the model in the current iteration, simulating the decision-making process of a referee in assessing various behaviors through an assessment thought chain, and applying the semantic information of the sample images to perform reasoning to determine the assessment reasoning result; determining the reward value in the current iteration based on the difference between the assessment reasoning result and the actual assessment result of the sample images; updating the model parameters in the current iteration with the goal of maximizing the reward value in the current iteration, and returning to the inference step until a preset convergence condition is reached, and then using the model in the current iteration with updated parameters as the motion assessment model.

[0120] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the motion assessment method provided by the above methods. The method includes: acquiring an image to be tested; simulating the decision-making process of a referee to assess various behaviors to be tested through an assessment thought chain based on a motion assessment model, and applying the semantic information of the image to be tested for reasoning to determine the motion assessment result.

[0121] Alternatively, a method for training a motion assessment model can be implemented, comprising: acquiring sample images; based on the model in the current iteration, simulating the decision-making process of a referee in assessing various behaviors through an assessment thought chain, and applying the semantic information of the sample images to perform reasoning to determine the assessment reasoning result; determining the reward value in the current iteration based on the difference between the assessment reasoning result and the actual assessment result of the sample images; updating the model parameters in the current iteration with the goal of maximizing the reward value in the current iteration, and returning to the inference step until a preset convergence condition is reached, and then using the model in the current iteration with updated parameters as the motion assessment model.

[0122] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of motion assessment, characterized by, The method comprises the following steps: acquiring a to-be-tested image; based on a sports evaluation model, simulating the decision-making thought of the referee in evaluating each to-be-tested behavior through an evaluation thought chain, and applying the semantic information of the to-be-tested image to reasoning, to determine the sports evaluation result; the decision-making thought of the referee in evaluating each to-be-tested behavior through an evaluation thought chain, and applying the semantic information of the to-be-tested image to reasoning, to determine the sports evaluation result, comprises: determining the specific performance mode of each to-be-tested behavior from the to-be-tested image according to the identification logic of each to-be-tested behavior in the decision-making thought and combining the semantic information of the to-be-tested image; performing compliance analysis and rule violation determination on the specific performance mode of each to-be-tested behavior according to the penalty basis of each to-be-tested behavior in the decision-making thought, to determine the evaluation result of each to-be-tested behavior; fusing the evaluation results of each to-be-tested behavior to determine the sports evaluation result.

2. The method of claim 1, wherein, The acquiring a to-be-tested image comprises: acquiring a to-be-tested video; screening a plurality of key video frames containing sports preparation actions and sports completion actions from the to-be-tested video; based on the semantic information of each key video frame, determining a starting video frame and an ending video frame from each key video frame, wherein the starting video frame corresponds to the starting time of the sports preparation action, and the ending video frame corresponds to the ending time of the sports completion action; based on the starting video frame and the ending video frame, determining the to-be-tested image from the to-be-tested video.

3. The method of claim 1, wherein, The acquiring a to-be-tested image comprises: starting video recording upon detecting an instruction for starting video recording; ending the video recording upon detecting an instruction for ending video recording; recording the video frames between the starting time of video recording and the ending time of video recording as the to-be-tested image.

4. A method for training a motion assessment model, characterized in that, The method comprises the following steps: acquiring a sample image; based on the model under the current iteration, simulating the decision-making thought of the referee in evaluating each to-be-tested behavior through an evaluation thought chain, and applying the semantic information of the sample image to reasoning, to determine the evaluation reasoning result; based on the difference between the evaluation reasoning result and the actual evaluation result of the sample image, determining the reward value under the current iteration; updating the model parameters under the current iteration with the goal of maximizing the reward value under the current iteration, and returning to the reasoning step until the updated parameters under the current iteration are used as the sports evaluation model after the preset convergence condition is reached.

5. The motion assessment model training method of claim 4, wherein, based on the difference between the evaluation reasoning result and the actual evaluation result of the sample image, determining the reward value under the current iteration, comprises: based on the difference between the evaluation reasoning result and the actual evaluation result of the sample image, determining the result reward value under the current iteration; based on the process text generated by the model under the current iteration during the reasoning process, determining the process reward value under the current iteration; based on the result reward value under the current iteration and the process reward value under the current iteration, determining the reward value under the current iteration.

6. The motion assessment model training method of claim 5, wherein, based on the result reward value under the current iteration and the process reward value under the current iteration, determining the reward value under the current iteration, comprises: determine a preliminary reward value of the current iteration based on the result reward value of the current iteration and the process reward value of the current iteration; determine a smoothed reward value of the current iteration based on the preliminary reward value of the last iteration and the preliminary reward value of the current iteration; determine the reward value of the current iteration based on the preliminary reward value of the current iteration and the smoothed reward value of the current iteration.

7. The motion assessment model training method of claim 6, wherein, The determining the reward value of the current iteration based on the preliminary reward value of the current iteration and the smoothed reward value of the current iteration comprises: determine a reward mean value of the current iteration based on the preliminary reward value of each sample image of the current iteration; determine the reward value of each sample image of the current iteration based on the reward mean value of the current iteration, the preliminary reward value of each sample image of the current iteration and the smoothed reward value of the current iteration.

8. A motion assessment device, comprising: comprise: a first obtaining unit, configured to obtain a to-be-tested image; a movement evaluation unit, configured to simulate a decision-making thought of a referee in evaluating each to-be-tested behavior based on a movement evaluation model and an evaluation thought chain, and to determine a movement evaluation result by reasoning based on semantic information of the to-be-tested image; The simulating the decision-making thought of the referee in evaluating each to-be-tested behavior based on the movement evaluation model and the evaluation thought chain, and the reasoning based on the semantic information of the to-be-tested image to determine the movement evaluation result comprise: determining specific performance modes of each to-be-tested behavior from the to-be-tested image according to an identification logic of each to-be-tested behavior in the decision-making thought and in combination with the semantic information of the to-be-tested image; performing compliance analysis and rule violation determination on the specific performance modes of each to-be-tested behavior according to a penalty basis of each to-be-tested behavior in the decision-making thought, and determining evaluation results of each to-be-tested behavior; fusing the evaluation results of each to-be-tested behavior to determine the movement evaluation result.

9. A training device for a motion assessment model, characterized in that, comprise: a second obtaining unit, configured to obtain a sample image; a reasoning unit, configured to simulate a decision-making thought of a referee in evaluating each to-be-tested behavior based on a model of a current iteration and an evaluation thought chain, and to determine an evaluation reasoning result by reasoning based on semantic information of the sample image; a determining unit, configured to determine a reward value of the current iteration based on a difference between the evaluation reasoning result and an actual evaluation result of the sample image; a training unit, configured to update model parameters of the current iteration with a target of maximizing the reward value of the current iteration, and to return to performing the reasoning step until a model of the current iteration after the model parameters are updated is used as the movement evaluation model after a preset convergence condition is reached.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor implements the movement evaluation method according to any one of claims 1 to 3 or the movement evaluation model training method according to any one of claims 4 to 7 when the processor executes the computer program.

11. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the movement evaluation method according to any one of claims 1 to 3 or the movement evaluation model training method according to any one of claims 4 to 7 when the computer program is executed by the processor.

12. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the motion evaluation method according to any one of claims 1 to 3, or implements the motion evaluation model training method according to any one of claims 4 to 7.

Images