Physical exercise real-time evaluation method, device and equipment based on computer vision and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a physical exercise real-time evaluation method and device based on computer vision, equipment and a storage medium. The method comprises the following steps: acquiring a target video frame, wherein the target video frame comprises a testee in a sports process; human body posture estimation is carried out on the target video frame, posture estimation information is obtained, and the posture estimation information comprises position information of multiple human body key points of the testee and included angle information among the multiple human body key points; according to the posture estimation information, a trained motion analysis model is called for motion analysis, a motion analysis result of the testee is output and obtained, and the motion analysis result is used for indicating the stage of the current motion of the testee. According to the embodiment of the invention, the position information and the included angle information of the key points of the human body are introduced as model input, and the model can more accurately capture details of actions, so that the accuracy of motion analysis is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a computer vision-based real-time sports evaluation method, device, equipment and storage medium. Background Art

[0002] In the field of sports evaluation, traditional methods mainly rely on manual observation or infrared measurement technology to capture the position changes of the human body during exercise to obtain evaluation data. Although these methods can provide evaluation results to a certain extent, they are often costly and inefficient due to the limitations of equipment and human resources, and are difficult to meet the needs of large-scale or real-time evaluation.

[0003] With the development of technology, especially the progress of computer vision and artificial intelligence technology, new possibilities have been brought to sports evaluation. Through computer vision technology, human key point information can be extracted from video frames to achieve automatic recognition and analysis of human movements. However, related technologies still have certain limitations when dealing with the counting and analysis of periodic movements, such as insufficient accuracy in capturing movement details or insufficient robustness in complex scenes. Summary of the invention

[0004] In view of this, the present disclosure proposes a computer vision-based real-time sports evaluation method, device, equipment and storage medium.

[0005] According to one aspect of the present disclosure, a method for real-time sports evaluation based on computer vision is provided, the method comprising:

[0006] Acquire a target video frame, wherein the target video frame includes a test person in a physical exercise process;

[0007] Performing human posture estimation on the target video frame to obtain posture estimation information, wherein the posture estimation information includes position information of multiple human body key points of the tester and angle information between the multiple human body key points;

[0008] According to the posture estimation information, the trained motion analysis model is called to perform motion analysis, and the motion analysis result of the tester is output, where the motion analysis result is used to indicate the stage of the tester's current action.

[0009] In a possible implementation, the step of performing human posture estimation on the target video frame to obtain posture estimation information includes:

[0010] According to the target video frame, calling the trained posture estimation model to perform human posture estimation, and outputting the position information of the multiple human body key points of the tester;

[0011] According to the position information of the multiple human body key points, the angle information between the multiple human body key points is determined, the angle information includes the angles corresponding to multiple groups of the human body key points, each group of the human body key points includes at least three human body key points.

[0012] In another possible implementation, the motion analysis model is a motion analysis model corresponding to the sports type of the tester, and before calling the trained motion analysis model to perform motion analysis based on the posture estimation information and outputting the motion analysis result of the tester, the method further includes:

[0013] Acquire training sample sets corresponding to multiple sports types, wherein the training sample sets include multiple groups of sample data, and each group of sample data includes sample posture estimation information and corresponding pre-labeled correct evaluation results;

[0014] For each of the sports types, the original neural network model is trained separately according to the corresponding training sample set to obtain a sports analysis model corresponding to the sports type.

[0015] In another possible implementation, the motion analysis result includes a predicted value, and the stage of the tester's current action includes one of the start of motion, in progress, and end of motion. When the predicted value is less than or equal to a first preset threshold, it is used to indicate the start of motion; when the predicted value is greater than the first preset threshold and less than a second preset threshold, it is used to indicate that motion is in progress; when the predicted value is greater than or equal to the second preset threshold, it is used to indicate the end of motion, and the second preset threshold is greater than the first preset threshold.

[0016] In another possible implementation, the method further includes:

[0017] Acquire first video frames corresponding to a plurality of sports project areas, wherein the first video frames include the target video frames of one or more testers, and each of the sports project areas is configured with a corresponding cache queue and scheduler;

[0018] For each first video frame of the sports item area, cache the first video frame through the corresponding cache queue;

[0019] The cached first video frame is processed by the corresponding scheduler to obtain a motion analysis result corresponding to the first video frame.

[0020] In another possible implementation, the cache queue is used to continuously receive the first video frame from the receiving end, and distribute the cached first video frame from the sending end at preset time intervals.

[0021] In another possible implementation, the first video frame includes the target video frames of multiple testers, and the processing of the cached first video frame by the corresponding scheduler to obtain the motion analysis result corresponding to the first video frame includes:

[0022] Cropping the first video frame to obtain the target video frames corresponding to the multiple testers, wherein the target video frame is an image including only a single tester;

[0023] The corresponding multiple threads of the scheduler are called to process the multiple target video frames in parallel, and the motion analysis results corresponding to the multiple testers are obtained.

[0024] According to another aspect of the present disclosure, a real-time sports evaluation device based on computer vision is provided, the device comprising:

[0025] An acquisition module, used for acquiring a target video frame, wherein the target video frame includes a tester in a sports process;

[0026] An estimation module, used for performing human posture estimation on the target video frame to obtain posture estimation information, wherein the posture estimation information includes position information of multiple human body key points of the tester and angle information between the multiple human body key points;

[0027] The analysis module is used to call the trained motion analysis model to perform motion analysis according to the posture estimation information, and output the motion analysis result of the tester, wherein the motion analysis result is used to indicate the stage of the tester's current action.

[0028] In a possible implementation, the analysis module is further used to:

[0029] According to the target video frame, calling the trained posture estimation model to perform human posture estimation, and outputting the position information of the multiple human body key points of the tester;

[0030] According to the position information of the multiple human body key points, the angle information between the multiple human body key points is determined, the angle information includes the angles corresponding to multiple groups of the human body key points, each group of the human body key points includes at least three human body key points.

[0031] In another possible implementation, the motion analysis model is a motion analysis model corresponding to the sports type of the tester, and the device further includes: a training module, which is used to:

[0032] Acquire training sample sets corresponding to multiple sports types, wherein the training sample sets include multiple groups of sample data, and each group of sample data includes sample posture estimation information and corresponding pre-labeled correct evaluation results;

[0033] For each of the sports types, the original neural network model is trained separately according to the corresponding training sample set to obtain a sports analysis model corresponding to the sports type.

[0034] In another possible implementation, the motion analysis result includes a predicted value, and the stage of the tester's current action includes one of the start of motion, in progress, and end of motion. When the predicted value is less than or equal to a first preset threshold, it is used to indicate the start of motion; when the predicted value is greater than the first preset threshold and less than a second preset threshold, it is used to indicate that motion is in progress; when the predicted value is greater than or equal to the second preset threshold, it is used to indicate the end of motion, and the second preset threshold is greater than the first preset threshold.

[0035] In another possible implementation, the device further includes: a processing module, configured to:

[0036] Acquire first video frames corresponding to a plurality of sports project areas, wherein the first video frames include the target video frames of one or more testers, and each of the sports project areas is configured with a corresponding cache queue and scheduler;

[0037] For each first video frame of the sports item area, cache the first video frame through the corresponding cache queue;

[0038] The cached first video frame is processed by the corresponding scheduler to obtain a motion analysis result corresponding to the first video frame.

[0039] In another possible implementation, the cache queue is used to continuously receive the first video frame from the receiving end, and distribute the cached first video frame from the sending end at preset time intervals.

[0040] In another possible implementation, the first video frame includes the target video frames of multiple testers, and the processing module is further configured to:

[0041] Cropping the first video frame to obtain the target video frames corresponding to the multiple testers, wherein the target video frame is an image including only a single tester;

[0042] The corresponding multiple threads of the scheduler are called to process the multiple target video frames in parallel, and the motion analysis results corresponding to the multiple testers are obtained.

[0043] According to another aspect of the present disclosure, there is provided a computing device, the device comprising:

[0044] processor;

[0045] a memory for storing processor-executable instructions;

[0046] Wherein, the processor is configured to implement the above method when executing the instructions stored in the memory.

[0047] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0048] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of a computing device, the processor in the computing device executes the above method.

[0049] The disclosed embodiment obtains the target video frame of the tester during the sports process, uses the human posture estimation technology to extract the position information of multiple key points of the human body and the angle information between multiple key points of the human body, and further uses the trained motion analysis model to conduct an in-depth analysis of the tester's movements, and outputs the motion analysis results, thereby indicating the stage of the tester's current movement. In particular, in the counting of periodic reciprocating motion, this method introduces the position information and angle information of the key points of the human body as model input, and the motion analysis model can more accurately capture the details of the movement, thereby providing more accurate motion analysis results, effectively improving the accuracy of motion analysis.

[0050] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0052] Figure 1 A schematic diagram of a sports evaluation system provided by an exemplary embodiment of the present disclosure is shown.

[0053] Figure 2 A flowchart of a real-time sports evaluation method based on computer vision provided by an exemplary embodiment of the present disclosure is shown.

[0054] Figure 3A flowchart of a real-time sports evaluation method based on computer vision provided by another exemplary embodiment of the present disclosure is shown.

[0055] Figure 4 A flowchart of a real-time sports evaluation method based on computer vision provided by another exemplary embodiment of the present disclosure is shown.

[0056] Figure 5 A flowchart of a sub-item algorithm module in a scheduler provided by an exemplary embodiment of the present disclosure is shown.

[0057] Figure 6 A flowchart of a real-time sports evaluation method based on computer vision provided by another exemplary embodiment of the present disclosure is shown.

[0058] Figure 7 A flowchart of a method for optimizing a lightweight reasoning library provided by an exemplary embodiment of the present disclosure is shown.

[0059] Figure 8 A flow chart of a method for analyzing time-series key point positions of a human body to determine measurement results provided by an exemplary embodiment of the present disclosure is shown.

[0060] Fig. 9 A schematic structural diagram of a real-time sports evaluation device based on computer vision provided by an exemplary embodiment of the present disclosure is shown.

[0061] Fig.10 It is a block diagram of a device for executing a real-time sports evaluation method according to an exemplary embodiment. DETAILED DESCRIPTION

[0062] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0063] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0064] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0065] Please refer to Figure 1, which shows a schematic diagram of a sports evaluation system provided by an exemplary embodiment of the present disclosure. The system may include: a video acquisition module 12, a motion analysis module 14 and a result display module 16.

[0066] The video acquisition module 12 includes a video acquisition device arranged at the sports project area of ​​the playground, and the video acquisition device is used to acquire video frames in the sports project area. In terms of design, the playground can be divided into several sports project areas, and each sports project area is equipped with at least one video acquisition device to achieve comprehensive monitoring of the sports process. For example, the video acquisition device is a camera, and the camera includes a monocular camera, a binocular camera (stereo camera), a multi-camera system, etc.

[0067] To improve flexibility and adaptability, this technical solution allows for a correspondence between multiple video acquisition devices and sports project areas. Specifically, a one-to-one configuration can be implemented, that is, each sports project area corresponds to a video acquisition device; a one-to-many layout can also be adopted, with a single video acquisition device covering multiple sports project areas; or a many-to-one strategy can be implemented, with multiple video acquisition devices centrally serving one sports project area. This solution does not specifically limit these different configuration methods to adapt to sports scenes of different scales and needs.

[0068] The sports in the present technical solution are not limited to specific types, and can be counting sports, such as rope skipping, sit-ups, pull-ups and other periodic repetitive movements; or ranging sports, such as standing long jump, which requires measuring the horizontal or vertical distance of the movement execution result. Through the efficient operation of the video acquisition module 12, accurate motion capture and data analysis can be provided for these sports, thereby supporting more intelligent and automated sports evaluation.

[0069] The motion analysis module 14 is the core component of the counting solution, which is responsible for receiving and processing the video frames of each sports area captured by the video acquisition module 12. The module is equipped with advanced motion analysis models, which have been professionally trained to perform in-depth analysis of the motion performance in the video and provide accurate motion analysis results.

[0070] In the disclosed embodiment, when the user starts the sports evaluation system program through the computing device and establishes a connection with the camera, the tester can enter the designated sports project area to start the sports evaluation. At this time, the computing device receives the video stream from the camera in real time through the sports evaluation system program, and reads it frame by frame. Using the algorithm provided by the present technical solution, human posture estimation is performed on each video frame to capture the position information of multiple human key points of the tester and the angle information between the human key points. These posture estimation information is then input into the motion analysis model, which performs in-depth analysis based on the input data and finally outputs the motion analysis results of the tester. These results indicate in detail the stage of the tester's current action, providing an important reference for sports evaluation.

[0071] The result display module 16 is responsible for displaying the obtained motion analysis results in an intuitive way. It integrates the motion analysis results of the evaluator in multiple target video frames and generates the final evaluation results based on them, such as the counting measurement results in counting items such as skipping or sit-ups. These evaluation results can be displayed on various terminal devices, such as computers, tablets or smart phones, so that users can view and analyze them anytime and anywhere.

[0072] The following uses several exemplary embodiments to introduce the computer vision-based real-time sports evaluation method provided by the embodiments of the present disclosure.

[0073] Please refer to Figure 2 , which shows a flowchart of a real-time sports evaluation method based on computer vision provided by an exemplary embodiment of the present disclosure. This embodiment uses the method for Figure 1 The method is illustrated by using the computing device shown in FIG.

[0074] Step 201, obtaining a target video frame, where the target video frame includes a tester in a sports activity.

[0075] The target video frame is a video picture captured by a video acquisition device and selected for analysis during a sports activity. The target video frame includes a tester performing sports. The tester refers to an object recorded and analyzed by a video acquisition device during a sports activity, i.e., an individual performing sports. There are one or more testers.

[0076] Step 202 , performing human body posture estimation on the target video frame to obtain posture estimation information, wherein the posture estimation information includes position information of multiple human body key points of the tester and angle information between the multiple human body key points.

[0077] In the process of performing human posture estimation of the target video frame, advanced computer vision technology is used to capture and analyze the dynamics of the test subject. This process not only accurately identifies multiple key points of the test subject in the video, such as the head, wrist, shoulder, knee, etc., but also records the coordinate positions of these multiple key points in the target video frame in detail.

[0078] In addition, this technical solution goes a step further and not only focuses on the position of a single key point, but also provides richer spatial information for understanding the tester's movements by calculating the angles between the key points of the human body. This angle information reveals the relative position relationship between the various parts of the body, so that the dynamic posture and movement details of the tester can be fully captured. Combining the position information and angle information of these key points, a comprehensive posture estimation information is obtained.

[0079] Human pose estimation refers to identifying and tracking human key points from the target video frame, and inferring the position information and angle information of these human key points, so as to estimate the human pose and movement. Pose estimation information refers to a set of data obtained through human pose estimation, including the position information of multiple human key points and the angle information between multiple human key points.

[0080] Optionally, based on the target video frame, a trained posture estimation model is called to perform human posture estimation, and the position information of multiple human body key points of the test subject is output; based on the position information of the multiple human body key points, the angle information between the multiple human body key points is determined.

[0081] The posture estimation model is a computer vision model, which is a model that uses computer vision technology to estimate the position information of key points of the human body from the target video frame. The posture estimation model can be a neural network model. The posture estimation model can be a general type, applicable to multiple sports types, or it can be a dedicated model specific to a certain sports type to meet specific analysis needs. In some embodiments, according to the type of sports participated by the tester, the posture estimation model corresponding to the sports type can be selected to perform accurate human posture analysis.

[0082] The key points of the human body refer to specific parts of the human body, such as the head, wrist, shoulder, knee, etc. The position information of the key points of the human body is the coordinate position of the key points of the human body in the target video frame.

[0083] The angle information between multiple human key points is used to indicate the relative position relationship between multiple human key points. The angle information can be a set of all possible angles formed by multiple human key points, or a set of angles corresponding to specific groups of human key points. For example, the angle information includes 24 angles formed by different human key points.

[0084] Optionally, the angle information includes angles corresponding to multiple groups of human key points, each group of human key points includes at least three human key points. For example, in a person's arm, the shoulder, elbow and wrist may form an angle. Similarly, in a leg, the hip, knee and ankle may also form another angle.

[0085] In human posture estimation, the angle usually refers to the angle between two vectors formed by three human key points. The two vectors are respectively the vectors formed by key point 1 pointing to key point 2, and the vectors formed by key point 2 pointing to key point 3. The angle between two vectors can be calculated by the direction and length of the two vectors, usually expressed in angles or radians. If more complex motion analysis or specific application scenarios are considered, sometimes four or more human key points may be involved to define. This is not limited in the embodiments of the present disclosure.

[0086] In some cases, the posture estimation model can directly output these two sets of information, namely the position information of multiple key points of the human body and the angle information between them. This model usually integrates additional algorithms or neural network layers specifically used to calculate the angle information from the position information of the key points of the human body. In other cases, the posture estimation model only outputs the position information of multiple key points of the human body, and then another independent program or algorithm calculates the angle information based on this position information. This provides greater flexibility because users can choose different algorithms to calculate the angle information as needed. The embodiments of the present disclosure are not limited to this.

[0087] Step 203, based on the posture estimation information, calling the trained motion analysis model to perform motion analysis, and outputting the motion analysis result of the tester, the motion analysis result is used to indicate the stage of the tester's current action.

[0088] The motion analysis model can be a neural network model based on deep learning technology, which can analyze and identify the various stages of the test subject in sports according to the input posture estimation information. In particular, when processing periodic or repetitive movements (such as skipping, sit-ups, pull-ups), the motion analysis model adopts a neural network architecture based on the self-attention mechanism (English: Transformer), which not only considers the position information of the key points of the human body, but also introduces the angle information formed by these key points of the human body, thereby providing the model with richer input data and significantly improving the accuracy of counting.

[0089] The motion analysis results are the data output after being processed by the motion analysis model. These results reflect the stage of the tester's current action in the sport, which can include one of the start of the movement, the ongoing movement, and the end of the movement.

[0090] When the target video frame includes one tester, the tester's motion analysis result is the motion analysis result of one tester; when the target video frame includes multiple testers, the tester's motion analysis result is the motion analysis result corresponding to each of the multiple testers. Since it takes a long time to detect multiple people in one video frame at the same time, it cannot meet the real-time requirements. Therefore, the embodiment of the present disclosure only takes the target video frame including one tester, and the motion analysis model is introduced according to the example of motion analysis of a single tester.

[0091] The motion analysis model can be a general type, applicable to multiple sports types, or a dedicated model specific to a certain sports type to meet specific analysis needs. In some embodiments, according to the posture estimation information, the motion analysis model corresponding to the tester's sports type is called for motion analysis. That is to say, in the model training stage, a motion analysis model can be trained separately for each of the multiple sports types, that is, each motion analysis model is used to perform motion analysis specifically for the data of the corresponding sports type. In order to build these models, training sample sets corresponding to multiple sports types can be collected in advance, each training sample set includes multiple groups of sample data, each group of sample data not only includes sample posture estimation information, but also includes pre-labeled correct evaluation results. By training the original neural network model in a targeted manner, a dedicated motion analysis model is customized for each sports type. In other words, the model training process may include: obtaining training sample sets corresponding to multiple sports types, the training sample sets include multiple groups of sample data, each group of sample data includes sample posture estimation information and corresponding pre-labeled correct evaluation results; for each sports type, the original neural network model is trained separately according to the corresponding training sample set to obtain the motion analysis model corresponding to the sports type.

[0092] The motion analysis model can include an encoder for extracting features, a multi-layer perceptron for classifying the types of sports, and a fully connected layer for outputting the motion analysis results. The following describes each of these components: Encoder: It is used to extract visual features from the input pose estimation information. The encoder uses deep learning techniques such as convolutional neural networks to identify and extract visual features that are critical for motion recognition. 2. Multi-layer perceptron: After feature extraction, the multi-layer perceptron is used to further process these visual features to identify and classify different types of sports. The multi-layer perceptron is a feed-forward neural network composed of multiple layers, each containing multiple neurons that can learn complex nonlinear relationships to achieve accurate classification of sports types. 3. Fully connected layer: After the multi-layer perceptron, the fully connected layer will process the classified data and output the motion analysis results, such as the motion analysis results including prediction values, which are the motion analysis model's assessment of the motion stage, such as the start, progress, or end of the action.

[0093] In some embodiments, the predicted value determines the stage of the action according to a preset threshold. When the predicted value is less than or equal to the first preset threshold, it is used to indicate the start of the action; when the predicted value is greater than the first preset threshold and less than the second preset threshold, it is used to indicate that the action is in progress; when the predicted value is greater than or equal to the second preset threshold, it is used to indicate the end of the action, and the second preset threshold is greater than the first preset threshold.

[0094] Optionally, the sports item is a counting sports item, such as rope skipping, sit-ups, pull-ups and other periodic repetitive actions. When the predicted value is less than or equal to the first preset threshold, this usually means the start of the movement, or can be regarded as the initial stage of the action, or can be understood as the reset of the action. When the predicted value is greater than the first preset threshold and less than the second preset threshold, this indicates that the movement is in progress. When the predicted value is greater than or equal to the second preset threshold, the movement is considered to have ended, which marks the completion of an action, and the counting measurement result will increase by one.

[0095] In addition, the computing device can perform motion analysis on multiple target video frames of the tester in a continuous test time period to obtain the final evaluation result, and the evaluation result may include count measurement results and / or normative results or other relevant indicators, and the count measurement results include the total number of times the tester completes the action in the test time period. For example, in a rope skipping test, the count measurement result is the number of rope skipping completed by the tester within the specified time. The normative result is used to indicate the accuracy of the tester's action and the degree of compliance with the standard. The normative result may include aspects such as the amplitude, speed, rhythm and posture of the action. For example, in a sit-up test, the normative result will evaluate whether the tester's actions of getting up and lying down meet the standard requirements, such as whether the elbow touches the knee, whether the back is completely in contact with the ground, etc. In addition to the count measurement results and the normative results, the evaluation results may also include other relevant indicators, such as coordination, balance, strength and endurance of the action. These indicators can help to more comprehensively evaluate the tester's sports performance and physical condition. The disclosed embodiment is not limited to this. The evaluation results can be displayed on a variety of terminal devices, including computers, tablets or smartphones, allowing users to conveniently view and analyze their athletic performance anytime and anywhere.

[0096] In summary, compared with the simple repeated counting method in traditional technology, the motion analysis method provided by the embodiment of the present disclosure has been improved in two aspects, which effectively improves the accuracy of motion evaluation. First, in terms of input data, in addition to the traditional position information of the key points of the human body, this method also introduces the angle information between the key points of the human body, which increases the model's ability to capture the details of the movement. Secondly, in terms of model training strategy, this method abandons the method of mixed training of all motion type data, and instead trains each type of motion data separately. Such customized training helps the model to learn and distinguish the characteristics of different sports more accurately. These two improvements not only improve the model's counting accuracy of test data, but also significantly increase the proportion of counting errors less than or equal to 1, at least by 15%.

[0097] Please refer to Figure 3 , which shows a flowchart of a real-time sports evaluation method based on computer vision provided by another exemplary embodiment of the present disclosure. In this embodiment, the method is used for Figure 1 The method is illustrated by using the computing device shown in FIG.

[0098] Step 301 : acquiring first video frames corresponding to respective multiple sports project areas, wherein the first video frames include one or more target video frames of a tester, and each sports project area is configured with a corresponding cache queue and scheduler.

[0099] A first video frame corresponding to each of a plurality of sports project areas is obtained, wherein the first video frame includes a tester in a sports process, and each sports project area is configured with a corresponding cache queue and a scheduler.

[0100] In order to improve the accuracy of video analysis, a series of calibration work can be done in advance, including determining the scope of the sports area, the type of sports, and the height and angle of the video acquisition device. These presets ensure that the video screen can accurately cover and face the sports area. In addition, using the image annotation tool, a rectangular box is annotated on a single frame, providing an accurate reference for subsequent processing.

[0101] Next, the computing device acquires a plurality of third video frames captured by the video acquisition device, which frames contain one or more sports venues. Each third video frame is preprocessed to obtain a first video frame of each sports venue, and the preprocessing includes cropping according to a pre-calibrated rectangular frame and reducing the clarity of the third video frame to a preset level. Cropping ensures that the video frame only contains the relevant sports venue, and reducing the clarity helps to reduce the amount of data and optimize subsequent processing steps.

[0102] In order to achieve efficient video data processing, a corresponding cache queue and scheduler are configured for each sports project area, that is, there is a one-to-one correspondence between multiple sports project areas, multiple cache queues, and multiple schedulers. The cache queue is used to cache the first video frame of each sports project area, and the scheduler is responsible for timely processing of these cached video frames. Through this one-to-one correspondence setting, it is ensured that the video frames of multiple sports project areas can be processed efficiently simultaneously and asynchronously.

[0103] Step 302: Process the cached first video frame through a corresponding scheduler to obtain a motion analysis result corresponding to the first video frame.

[0104] Refined cache management is implemented for the first video frame of each sports project area. That is, the first video frame of each sports project area is received and temporarily stored by its corresponding cache queue. This cache queue plays a core role in data flow. The cache queue is used to continuously receive the first video frame from the receiving end and distribute the cached first video frame from the sending end at preset time intervals. This mechanism ensures the continuity and stability of data processing.

[0105] Furthermore, when the number of first video frames in the cache queue reaches a preset upper limit, the reception of new video frames will be automatically suspended to avoid data overflow. This process will continue until a video frame in the cache queue is successfully distributed. In addition, when the cache queue is emptied, the distribution operation will be suspended until a new video frame is received, thereby ensuring the efficiency and orderliness of data processing.

[0106] Step 303: Process the cached first video frame through the corresponding scheduler to obtain a motion analysis result corresponding to the first video frame.

[0107] The first video frame in each cache queue will be further processed by its corresponding scheduler. The scheduler analyzes the first video frame according to the human posture estimation algorithm and the motion analysis algorithm to obtain the motion analysis result corresponding to the tester included in the first video frame.

[0108] In an illustrative example, a flowchart of a real-time sports evaluation method based on computer vision is provided in another exemplary embodiment of the present disclosure. Figure 4 The method comprises the following steps.

[0109] 1. Monocular camera calibration and sports field calibration: This is the starting point of the entire system. The monocular camera is accurately calibrated to ensure that the captured image data is accurate and provides high-quality input for subsequent processing: the process of accurately measuring and calibrating the sports area to ensure that the content in the video frame accurately corresponds to the size and layout of the actual venue, providing accurate spatial reference for motion analysis.

[0110] 2. Preprocessing: Perform preliminary processing on the captured video frames, such as denoising, contrast adjustment, and scaling and reducing the clarity of the images as needed to optimize performance and resource usage.

[0111] 3. Video Cache Queue: Also known as the cache queue, a bidirectional cache queue is used to cache pre-processed video frames. This mechanism allows video frames to be cached before being processed, and also supports on-demand distribution of video frames from the cache. It allows the system to run smoothly under high load conditions and can flexibly manage the inflow and outflow of video frames.

[0112] 4. Main control program: As the brain of the sports evaluation system, the main control program is responsible for coordinating and managing the entire evaluation process. It controls the input, processing and output of the video, ensuring that all modules run according to the predetermined logic and operation commands.

[0113] 5. Sub-project scheduler: Also called a scheduler, the system initializes an independent scheduler for each sub-project algorithm module. These schedulers are the core of the system and are responsible for managing and scheduling the tasks of their respective modules. Each scheduler receives the distributed video frames and cyclically calls the sub-project algorithm module to process each transmitted video frame, allowing multiple sports projects to be asynchronously evaluated at the same time, realizing multi-process scheduling between algorithm modules, that is, asynchronous evaluation of multiple projects at the same time.

[0114] When the first video frame is a target video frame of a tester, it means that there is a tester in the sports project area for testing, and the computing device can call a thread of the corresponding scheduler to process the target video frame to obtain the motion analysis result of the tester. When the first video frame includes target video frames of multiple testers, it means that there are multiple testers in the sports project area for testing at the same time, and the computing device can crop the first video frame to obtain target video frames corresponding to the multiple testers, and the target video frame is an image that only includes a single tester; call multiple threads of the corresponding scheduler to process multiple target video frames in parallel to obtain the motion analysis results corresponding to the multiple testers.

[0115] Among them, for each of the multiple threads of the scheduler, calling the thread to process the corresponding target video frame includes: calling the thread to perform human posture estimation on the target video frame to obtain posture estimation information, the posture estimation information includes the position information of multiple human key points of the tester and the angle information between multiple human key points; according to the posture estimation information, calling the trained motion analysis model to perform motion analysis, outputting the motion analysis result of the tester, and the motion analysis result is used to indicate the stage of the tester's current action. For relevant details, please refer to the relevant description in the above embodiment, which will not be repeated here.

[0116] In an illustrative example, a flowchart of a sub-item algorithm module in a scheduler provided by an exemplary embodiment of the present disclosure is as follows: Figure 5 As shown. When multiple testers are tested simultaneously in the same sports area, the computing device can evaluate the video frames of multiple testers in the same sports area in real time. The method includes the following steps. 1. Multi-person video frame: This is the original video input, which contains multiple testers in the sports area. The video frame is captured and transmitted to the computing device for further processing. 2. Identification and annotation: In this step, the position of each person in the video frame is identified and annotated with a rectangular box. These rectangular boxes correspond to the outline of each tester, providing a basis for subsequent video frame cropping. 3. Video frame cropping: According to the identified rectangular boxes, the video frame is cropped into smaller pictures, each of which contains only one tester. This step ensures that the data of each tester can be processed independently, improving the accuracy and efficiency of the algorithm. The figure only schematically shows three testers, namely tester 1, tester 2, and tester 3, which represent individual testers identified in the video frame. The computing device will identify and analyze each tester. 4. Multi-threaded parallel processing: The computing device uses multi-threading technology to allow multiple algorithm threads to run simultaneously, so as to process the data of multiple testers in parallel within the interval of two frames. The cropped images are assigned to the corresponding algorithm threads. Each thread calls the sub-project algorithm module for processing, such as human posture assessment, motion analysis, etc. The figure only schematically shows three algorithm threads, namely algorithm thread 1, algorithm thread 2, and algorithm thread 3. Each algorithm thread is responsible for processing the video frames of the corresponding tester to achieve multi-threaded parallel processing. 5. Output evaluation results: Finally, after each algorithm thread is processed, the corresponding motion analysis results will be output. Through the above process, the computing device can evaluate multiple sports items for multiple testers in real time, achieving efficient parallel processing.

[0117] In some embodiments, the disclosed embodiments propose a real-time sports evaluation method based on computer vision, which can run efficiently on a central processing unit (CPU). The general computer vision method for human motion analysis relies on the high computing power provided by the graphics processing unit (GPU) on the edge to meet the real-time requirements. The programming language and the computing structure that is not optimized for the CPU restrict the performance of using CPU reasoning.

[0118] This method uses C++ programming language instead of Python to deeply explore the underlying potential of model reasoning. By performing motion analysis directly on the basis of the computational graph output stream (i.e., posture estimation information) of the posture estimation model, this method realizes the tight integration and static binding of the posture estimation model and the motion analysis algorithm. Joint compilation is contained in a single algorithm thread. The posture estimation model can provide a Python version of the API interface and a C++ version of the underlying source code. In the traditional mode of application programming interface (API) call, the user inputs the picture into the model to obtain the key point information of the human posture, and then uses these key points for subsequent motion analysis. In this method, the joint compilation method is adopted, that is, the changes are made directly at the C++ source code level, and the motion analysis algorithm can be directly embedded in the source code of the posture estimation model. Although this method puts higher requirements on the learning, modification and environment configuration of the source code, it avoids the overhead of API calls, thereby significantly improving the overall operation efficiency and reducing the reasoning delay.

[0119] In an illustrative example, a flowchart of a real-time sports evaluation method based on computer vision is provided in another exemplary embodiment of the present disclosure. Figure 6 As shown. The method includes the following steps. 1. Input video frames to provide the system with a data source to be analyzed; 2. Output the evaluation results through the dynamic link library (DLL) of the sub-project algorithm module. The dynamic link library integrates the computer vision model (i.e., the posture estimation model) and the motion analysis algorithm to achieve efficient execution of the algorithm. Through this method, it is possible to use the CPU to achieve fast and accurate real-time evaluation of sports without relying on the high computing power of the GPU, which provides new possibilities for the application of computer vision in resource-constrained environments.

[0120] In some embodiments, the model inference process is optimized for pose estimation models and motion analysis models. Traditional model inference involves loading a pre-trained model to disk, inputting data into a computational graph, and then performing inference operations through CPU operators to output the results. However, this method uses a lightweight inference library, which significantly improves the efficiency of the model running on the CPU. Further, the advantages of a lightweight inference library include:

[0121] 1. Model conversion and file serialization: Convert the pre-trained model into a lightweight model file and perform file serialization, which greatly reduces disk usage and speeds up model loading. File serialization is used to convert the objects of the pose estimation model into binary data blocks and establish their own data indexes for the binary data blocks, so that the data can be accessed in situ like a pointer-based data structure.

[0122] 2. Parameter quantization: Through quantization technology, the floating-point parameters in the model are converted to unsigned integer parameters for calculation, and then converted back to floating-point type when output. This process involves storing the maximum and minimum values ​​of each layer and linearly dividing the interval of these values ​​into 256 discrete values. In this way, each floating-point number can be approximately represented as the closest discrete value in the interval, represented by an 8-bit integer, thereby reducing the model size and speeding up the calculation process. Integer calculations are performed first during inference, and then converted back to floating-point type when output.

[0123] 3. Optimized inference operator library: Model inference also uses a highly optimized inference operator library, which further improves the computing speed of CPU operators and ensures the realization of real-time motion analysis.

[0124] In an illustrative example, a flowchart of a method for optimizing a lightweight reasoning library provided by an exemplary embodiment of the present disclosure is as follows: Figure 7 As shown. Convert the pre-trained model into a lightweight model file and perform file serialization. This step involves converting the pre-trained model into a lightweight model file and converting the model data into an efficient binary format through serialization to reduce disk usage and speed up the loading of the model; and perform parameter quantization. In this step, the parameters of the model are quantized to convert the original floating-point parameters into unsigned integer parameters. This process involves recording the parameter range of each layer and linearly mapping it to a smaller set of integer values, thereby reducing the model size and accelerating the calculation. Finally, perform model inference on the CPU. Using a highly optimized inference operator library, the CPU operator is accelerated so that the model can run efficiently on the CPU and achieve real-time motion analysis.

[0125] Through these optimization measures, the method provided by the embodiment of the present disclosure not only improves the inference speed of the model on the CPU, but also maintains the accuracy of the model, thereby achieving efficient and real-time sports evaluation.

[0126] In some embodiments, the evaluation results can be obtained based on the analysis of the position information of the key points of the human body in time sequence. This method can divide the sports evaluation into three stages: the beginning of the movement, the ongoing movement, and the end of the movement. For distance-measuring sports, such as standing long jump, the change in the position information of the key points of the human body specific to the tester is used as the criterion for judging the beginning and end of the movement. For example, the coordinate change rate of the key points of the tester's ankle can be used to determine the moment of take-off and landing. For counting sports, such as rope skipping, the start and end times of the movement are specified by system instructions.

[0127] In counting sports, the evaluation results include counting measurement results, and can also include normative results and other related indicators. The counting measurement results can be obtained in two ways: one is through the above-mentioned motion analysis model, and the other is through statistical analysis methods. In the statistical analysis method, the motion data within a motion cycle is first extracted, and the motion data is used to indicate the position changes of key points of the human body. Then, from the perspective of signal analysis, the motion data features such as the peak-to-valley difference, rise-and-fall difference, duration, and rise-and-fall time ratio of the waveform are statistically analyzed. In order to establish a reliable evaluation standard, a large amount of test data was deeply statistically analyzed. Based on these analyses, a normalized data feature range of standard actions is defined for each type of sports. By comparing the motion data features obtained by real-time analysis with the data feature range of standard actions, the counting of actions can be completed accurately, and the irregularity of any action can be identified. That is, by calculating the extracted motion data features and comparing them with the data feature range of standard actions. If the motion data features fall within the data feature range of standard actions, they are counted as valid actions; if the motion data features exceed this range, they are marked as irregular actions. The information of these irregular actions, corresponding to the motion data features with large deviations from the standard data, provides a basis for further analysis and improvement. This approach not only provides accurate action counts, but also provides deep insights into the quality of action execution.

[0128] In an illustrative example, a flowchart of a method for analyzing the time series of key point positions of a human body to determine a measurement result is provided by an exemplary embodiment of the present disclosure. Figure 8 As shown. 1. Extraction of coordinate sequence of key points in test video: responsible for extracting coordinate sequence of key points of human body from test video (i.e., multiple continuous video frames). These coordinate sequences are the basis for analyzing the tester's movements and can be used for further movement analysis and evaluation. 2. Single action reciprocating cycle interception: intercept the reciprocating characteristics of a single action from the coordinate sequence of key points of human body, for example, in periodic movements (such as skipping or sit-ups), analyze the changes of key points of human body in each movement cycle. 3. Data waveform characteristics: waveform analysis is performed on the coordinate sequence of key points of human body to obtain real-time coordinate point analysis data to identify the periodic characteristics of the movement. By analyzing the peak-to-valley difference, rise and fall difference, etc. of the waveform, the dynamic characteristics of the movement can be understood more deeply. 4. Obtain the data feature range of real-time coordinate point analysis data and standard movement. 5. Compare the motion data features of real-time coordinate point analysis data with the data feature range of standard movement. 6. If the motion data features fall within the data feature range of standard movement, the counting measurement result is increased by one. 7. If the motion data feature is not within the data feature range of the standard action, it will be marked as an irregular action and classified accordingly to understand the problem in the execution of the action.

[0129] In distance measurement sports, the evaluation results include distance measurement results, and may also include normative results and other related indicators. This method proposes that after obtaining the posture estimation information, the time domain and spatial domain smoothing technology can be used to remove the noise in the human key point detection process, so as to accurately locate the start and end time of the movement, and the smoothed results will be used for further motion analysis. In terms of time domain smoothing, the position information of the previous few video frames of the same tester can be retained and averaged with the position information of the current video frame of the tester to obtain more stable and accurate smoothed position information. This method effectively reduces the random fluctuations between video frames and provides more reliable basic data for motion analysis. In terms of spatial domain smoothing, multiple human key points that are symmetrical on the left and right or in the same area can be selected. The same area is the same part of the human body, such as the trunk, palm, face, and foot, and each part includes one or more human key points. These human key points show a high degree of consistency in position between video frames. By spatially averaging the displacements of these human key points, the deviation of the position estimation of a single human key point can be eliminated, further improving the accuracy and reliability of the data. Through this comprehensive temporal and spatial smoothing process, more accurate and reliable motion analysis results can be obtained.

[0130] In some embodiments, the computing device can reflect the evaluation results of each sports area in real time on the system management interface, so that users can monitor the progress and effect of sports evaluation in real time. After the evaluation is completed, these evaluation results can be easily imported into a target database, such as a MySQL database, to ensure the safe storage of data and subsequent in-depth analysis.

[0131] In order to further enhance the user experience, the computing device can also display irregular and illegal actions captured during the evaluation process on the evaluation result analysis interface. By pre-classifying and numbering the video frames and identifying them according to the type of illegal actions, users only need to select the corresponding number to quickly locate and view the corresponding video frame, greatly improving the efficiency of problem diagnosis and feedback. Through this intelligent display method, users can not only obtain evaluation feedback in real time, but also quickly access detailed evaluation results after the evaluation is completed. This technical solution not only improves the scientificity and accuracy of sports evaluation, but also makes the entire evaluation process more transparent, efficient and easy to manage.

[0132] In an illustrative example, the solution provided by the embodiment of the present disclosure is integrated into the system in the form of an application. The workflow is as follows: 1. The user starts the sports evaluation system program and connects to the camera. 2. The tester enters the test site and starts the sports evaluation. 3. The program will read frame by frame from the video stream provided by the camera, and through the solution provided by the embodiment of the present disclosure, the human body movement posture is analyzed in real time. The evaluation results are synchronized to the large screen display. The teacher can view the evaluation results and status of each sports area in real time on the system management interface. After the evaluation is completed, the evaluation results are imported into the MySQL database. 4. Enter the evaluation result analysis interface to view the irregular actions and illegal action captures during the evaluation process. The video frames are pre-classified with different numbers according to the type of illegal actions on the evaluation result analysis interface. Select the corresponding number to view.

[0133] In summary, the disclosed embodiment proposes a real-time sports evaluation method based on computer vision, which can realize automatic evaluation and performance recording of sports events by using only ordinary computers and cameras above the sports field. On the one hand, the target video frame of the tester in the process of sports is obtained, and the position information of multiple human key points and the angle information between multiple human key points are extracted by using human posture estimation technology. The motion analysis model completed by training is further used to analyze the tester's action in depth, and the motion analysis result is output, thereby indicating the stage of the tester's current action. Especially in the counting of periodic reciprocating motion, this method effectively improves the accuracy of the motion analysis result by introducing the angle information formed by the human key points as the model input, and training the model separately for the data of each sports type. On the other hand, the disclosed embodiment proposes a method for real-time evaluation of multiple people and multiple projects. This method adopts a general multimedia video encoding and decoding framework at the video input end, designs a cache mechanism of a two-way queue, and distributes the video frame to the scheduler after reducing the clarity after scaling the video frame. On the other hand, the disclosed embodiment initializes a corresponding scheduler for each sub-project algorithm module, cyclically calls the algorithm to process each frame of video frame transmitted, and realizes multi-process scheduling between operator project algorithm modules, that is, asynchronous evaluation of multiple projects at the same time. On the other hand, the posture estimation model optimized on the CPU is used to analyze the human body motion posture in real time. In order to optimize the running rate, the posture estimation model optimized by the operator under the CPU is jointly compiled with the algorithm program, and the position information and angle information of the human body key points of interest are directly obtained from the output stream of the calculation graph at the bottom layer, instead of being obtained from the encapsulated API before. On the other hand, from the time series human body key point position data, the position change of the human body key points is extracted to determine the start time and the end time, and the information of the periodic transformation of the height and angle is extracted for motion counting. The correlation between the time series and the spatial key point position is used to filter out erroneous interference and noise, further ensuring the accuracy of the final evaluation results.

[0134] The following is an apparatus embodiment of the present disclosure. For parts not described in detail in the apparatus embodiment, reference may be made to the technical details disclosed in the above method embodiment.

[0135] Please refer to Fig. 9 , which shows a schematic diagram of the structure of a real-time sports evaluation device based on computer vision provided by an exemplary embodiment of the present disclosure. The device can realize all or part of a computing device through software, hardware, or a combination of the two. The device includes: an acquisition module 92, a conversion module 93, and a training module 94.

[0136] An acquisition module 92, for acquiring a target video frame, wherein the target video frame includes a tester in a sports process;

[0137] An estimation module 93 is used to estimate the human body posture of the target video frame to obtain posture estimation information, where the posture estimation information includes position information of multiple human body key points of the tester and angle information between multiple human body key points;

[0138] The analysis module 94 is used to call the trained motion analysis model to perform motion analysis based on the posture estimation information, and output the motion analysis result of the tester. The motion analysis result is used to indicate the stage of the tester's current action.

[0139] In a possible implementation, the analysis module 94 is further configured to:

[0140] According to the target video frame, the trained posture estimation model is called to estimate the human posture, and the position information of multiple key points of the test subject is output;

[0141] According to the position information of the multiple human body key points, the angle information between the multiple human body key points is determined, the angle information includes the angles corresponding to the multiple groups of human body key points, and each group of human body key points includes at least three human body key points.

[0142] In another possible implementation, the motion analysis model is a motion analysis model corresponding to the tester's sports type, and the device further includes: a training module for:

[0143] Acquire training sample sets corresponding to multiple sports types, the training sample sets including multiple groups of sample data, each group of sample data including sample posture estimation information and corresponding pre-labeled correct evaluation results;

[0144] For each sports type, the original neural network model is trained separately according to the corresponding training sample set to obtain the sports analysis model corresponding to the sports type.

[0145] In another possible implementation, the motion analysis result includes a predicted value, and the stage of the tester's current action includes one of the start of motion, in progress, and end of motion. When the predicted value is less than or equal to a first preset threshold, it is used to indicate the start of motion; when the predicted value is greater than the first preset threshold and less than the second preset threshold, it is used to indicate the progress of motion; when the predicted value is greater than or equal to the second preset threshold, it is used to indicate the end of motion, and the second preset threshold is greater than the first preset threshold.

[0146] In another possible implementation, the apparatus further includes: a processing module, configured to:

[0147] Acquire first video frames corresponding to each of a plurality of sports project areas, the first video frames including one or more target video frames of a tester, and each sports project area is configured with a corresponding cache queue and scheduler;

[0148] For a first video frame of each motion item area, cache the first video frame through a corresponding cache queue;

[0149] The cached first video frame is processed by a corresponding scheduler to obtain a motion analysis result corresponding to the first video frame.

[0150] In another possible implementation, the cache queue is used to continuously receive the first video frame from the receiving end, and distribute the cached first video frame from the sending end at preset time intervals.

[0151] In another possible implementation, the first video frame includes a plurality of target video frames of the tester, and the processing module is further configured to:

[0152] The first video frame is cropped to obtain target video frames corresponding to the multiple testers, where the target video frame is an image including only a single tester;

[0153] Multiple threads of the corresponding scheduler are called to process multiple target video frames in parallel, and motion analysis results corresponding to multiple testers are obtained.

[0154] It should be noted that the device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example to implement its functions. In practical applications, the above-mentioned functions can be assigned to different functional modules according to actual needs, that is, the content structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0155] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0156] An embodiment of the present disclosure also provides a real-time sports evaluation device based on computer vision, the device comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0157] The embodiment of the present disclosure further provides a non-volatile computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above method is implemented.

[0158] The embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of a computing device, the processor in the computing device executes the above method.

[0159] Fig.10 1 is a block diagram of a device 1900 for executing a real-time sports evaluation method according to an exemplary embodiment. For example, the device 1900 can be provided as a server or a terminal device. Fig.10 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0160] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0161] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0162] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0163] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0164] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0165] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0166] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0167] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0168] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0169] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0170] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A real-time sports evaluation method based on computer vision, characterized in that: The method comprises: Acquire a target video frame, wherein the target video frame includes a test person in a physical exercise process; Performing human posture estimation on the target video frame to obtain posture estimation information, wherein the posture estimation information includes position information of multiple human body key points of the tester and angle information between the multiple human body key points; According to the posture estimation information, the trained motion analysis model is called to perform motion analysis, and the motion analysis result of the tester is output, where the motion analysis result is used to indicate the stage of the tester's current action.

2. The method according to claim 1, characterized in that: The step of performing human posture estimation on the target video frame to obtain posture estimation information includes: According to the target video frame, calling the trained posture estimation model to perform human posture estimation, and outputting the position information of the multiple human body key points of the tester; According to the position information of the multiple human body key points, the angle information between the multiple human body key points is determined, the angle information includes the angles corresponding to multiple groups of the human body key points, each group of the human body key points includes at least three human body key points.

3. The method according to claim 1 or 2, characterized in that: The motion analysis model is a motion analysis model corresponding to the sports type of the tester, and before calling the trained motion analysis model to perform motion analysis based on the posture estimation information and outputting the motion analysis result of the tester, the method further includes: Acquire training sample sets corresponding to multiple sports types, wherein the training sample sets include multiple groups of sample data, and each group of sample data includes sample posture estimation information and corresponding pre-labeled correct evaluation results; For each of the sports types, the original neural network model is trained separately according to the corresponding training sample set to obtain a sports analysis model corresponding to the sports type.

4. The method according to any one of claims 1 to 3, characterized in that The motion analysis result includes a predicted value, and the stage of the tester's current action includes one of the start of motion, ongoing motion and end of motion. When the predicted value is less than or equal to a first preset threshold, it is used to indicate the start of motion; when the predicted value is greater than the first preset threshold and less than a second preset threshold, it is used to indicate the ongoing motion; when the predicted value is greater than or equal to the second preset threshold, it is used to indicate the end of motion, and the second preset threshold is greater than the first preset threshold.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Acquire first video frames corresponding to a plurality of sports project areas, wherein the first video frames include the target video frames of one or more testers, and each of the sports project areas is configured with a corresponding cache queue and scheduler; For each first video frame of the sports item area, cache the first video frame through the corresponding cache queue; The cached first video frame is processed by the corresponding scheduler to obtain a motion analysis result corresponding to the first video frame.

6. The method according to claim 5, characterized in that The cache queue is used to continuously receive the first video frame from the receiving end, and distribute the cached first video frame from the sending end at preset time intervals.

7. The method according to claim 5, characterized in that The first video frame includes the target video frames of multiple testers, and the processing of the cached first video frame by the corresponding scheduler to obtain the motion analysis result corresponding to the first video frame includes: Cropping the first video frame to obtain the target video frames corresponding to the multiple testers, wherein the target video frame is an image including only a single tester; The corresponding multiple threads of the scheduler are called to process the multiple target video frames in parallel, and the motion analysis results corresponding to the multiple testers are obtained.

8. A real-time sports evaluation device based on computer vision, characterized in that: The device comprises: An acquisition module, used for acquiring a target video frame, wherein the target video frame includes a tester in a sports process; An estimation module, used for performing human posture estimation on the target video frame to obtain posture estimation information, wherein the posture estimation information includes position information of multiple human body key points of the tester and angle information between the multiple human body key points; The analysis module is used to call the trained motion analysis model to perform motion analysis according to the posture estimation information, and output the motion analysis result of the tester, wherein the motion analysis result is used to indicate the stage of the tester's current action.

9. A computing device, characterized in that The device comprises: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 7 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images