Methods, devices, equipment and media for evaluating broadcast gymnastics performance

Abstract

The present invention discloses a method, device, equipment, and medium for evaluating broadcasting gymnastics performance. The method features the following: responding to a broadcasting gymnastics evaluation request from a user to be evaluated, loading a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request; sequentially capturing at least one video capture frame corresponding to each second of the broadcasting gymnastics video of the user to be evaluated based on the playback progress of the target broadcasting gymnastics video; using the evaluation metadata set to score each second of the broadcasting gymnastics video based on the skeleton point coordinate set and millisecond offset value of each video capture frame, thereby determining the broadcasting gymnastics score corresponding to each second of the broadcasting gymnastics video; aggregating each second of the broadcasting gymnastics score of the target broadcasting gymnastics video to obtain a total video score, and performing a performance evaluation based on the total video score and the total broadcasting gymnastics score corresponding to the target broadcasting gymnastics video to determine the broadcasting gymnastics score of the user to be evaluated. The method can automatically monitor and evaluate the accuracy and completion of the user's broadcasting gymnastics movements, thereby improving the efficiency and quality of broadcasting gymnastics instruction.

Description

Technical Field

[0001] The present invention relates to the technical field of online broadcasting gymnastics, and in particular to a broadcasting gymnastics performance evaluation method, device, equipment and medium. Background Art

[0002] As a highly popular form of physical exercise, radio gymnastics plays an important role in improving people's physical health and quality of life. However, traditional radio gymnastics is conducted offline by radio gymnastics teachers. Radio gymnastics teachers need to spend a lot of time to check whether students' movements are standard and evaluate the completion and performance of students' radio gymnastics. Traditional radio gymnastics acceptance is very inconvenient. With the development of the Internet, the development of online radio gymnastics has enabled one-to-many video or live broadcast teaching. Although students can submit videos for acceptance and evaluation of radio gymnastics, radio gymnastics teachers are still required to check and evaluate the videos one by one. Although existing technologies can detect radio gymnastics videos through image recognition algorithms, the detection accuracy is very low, making it impossible to accurately evaluate the quality of radio gymnastics videos, let alone the performance of radio gymnastics videos. Summary of the Invention

[0003] The present invention provides a method, device, equipment and medium for evaluating the performance of a broadcast gymnastics exercise, so as to solve the technical problem in the prior art that the performance of a broadcast gymnastics exercise video cannot be automatically evaluated.

[0004] According to one aspect of the present invention, a method for evaluating broadcast gymnastics performance is provided, comprising:

[0005] In response to a broadcasting gymnastics evaluation request from a user to be evaluated, loading a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request;

[0006] At least one video capture frame corresponding to each second of the radio gymnastics video of the user to be evaluated is captured in sequence according to the playback progress of the target radio gymnastics video; wherein the video capture frame includes a skeleton point coordinate set and a millisecond offset value;

[0007] For the per-second broadcast calisthenics video, score the video according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame using the evaluation metadata set to determine the per-second broadcast calisthenics score corresponding to the per-second broadcast calisthenics video;

[0008] The radio gymnastics scores of each second of the target radio gymnastics video are summarized to obtain the total score of the video, and a performance evaluation is performed based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated.

[0009] According to another aspect of the present invention, there is provided a device for evaluating broadcast gymnastics performance, comprising:

[0010] A loading module, configured to respond to a broadcasting exercise evaluation request from a user to be evaluated and load a target broadcasting exercise video and an evaluation metadata set corresponding to the broadcasting exercise evaluation request;

[0011] An acquisition module is configured to sequentially acquire at least one video acquisition frame corresponding to each second of the broadcasting calisthenics video of the user to be evaluated according to the playback progress of the target broadcasting calisthenics video; wherein the video acquisition frame includes a skeleton point coordinate set and a millisecond offset value;

[0012] A scoring module is used to score the per-second broadcast calisthenics video according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame using the evaluation metadata set, and determine a per-second broadcast calisthenics score corresponding to the per-second broadcast calisthenics video;

[0013] An evaluation module is used to summarize the radio gymnastics scores of each second of the target radio gymnastics video to obtain a total video score, perform performance evaluation based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video, and determine the radio gymnastics score of the user to be evaluated.

[0014] According to another aspect of the present invention, an electronic device is provided, comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the broadcast gymnastics performance evaluation method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the broadcast gymnastics performance evaluation method described in any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention is to load the target radio gymnastics video and the evaluation metadata set corresponding to the radio gymnastics evaluation request in response to the radio gymnastics evaluation request of the user to be evaluated. When the user selects the target radio gymnastics video for evaluation, the target radio gymnastics video and the evaluation metadata set can be loaded synchronously, thereby realizing the synchronous evaluation of the target radio gymnastics video and effectively improving the efficiency of the radio gymnastics video evaluation; at least one video acquisition frame corresponding to the radio gymnastics video of the user to be evaluated per second is collected in sequence according to the playback progress of the target radio gymnastics video, and the video frames of the radio gymnastics video are collected in real time to realize the recognition of each second of the radio gymnastics video, thereby effectively improving the accuracy of the evaluation; for the radio gymnastics video per second, the evaluation metadata set is used to collect the video frames of the radio gymnastics video in real time ... The system uses a data set to score the performance of each frame of the video capture based on the skeletal point coordinates and millisecond offset values, and determines the per-second radio gymnastics score corresponding to the per-second radio gymnastics video. By scoring each per-second radio gymnastics video, accurate scoring of the radio gymnastics video is achieved, which can improve the accuracy of the evaluation. The per-second radio gymnastics scores of the target radio gymnastics video are aggregated to obtain a total video score. The performance evaluation is performed based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video, and the radio gymnastics score of the user to be evaluated is determined. By aggregating the scores of each per-second radio gymnastics video, the performance evaluation of the entire video is completed, solving the technical problem of the inability to automatically evaluate the performance of radio gymnastics videos in the existing technology. The system can automatically monitor and evaluate the accuracy and completion of the user's radio gymnastics movements, thereby improving the efficiency and quality of radio gymnastics teaching.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of a method for evaluating broadcast gymnastics performance is provided for an embodiment of the present invention;

[0023] Figure 2 A flowchart of another method for evaluating broadcast gymnastics scores provided by an embodiment of the present invention;

[0024] Figure 3 A flowchart of another method for evaluating broadcast gymnastics scores provided by an embodiment of the present invention;

[0025] Figure 4 A schematic structural diagram of a broadcast gymnastics performance evaluation device provided by an embodiment of the present invention;

[0026] Figure 5 A schematic structural diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Figure 1 The present invention provides a flowchart of a method for evaluating the performance of a broadcasting gymnastics exercise. This embodiment is applicable to the situation where the performance of a user's broadcasting gymnastics exercise is evaluated online in real time. The method can be executed by a broadcasting gymnastics exercise evaluation device. The broadcasting gymnastics exercise evaluation device can be implemented in the form of hardware and / or software. The broadcasting gymnastics exercise evaluation device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0030] S110 : In response to a broadcasting gymnastics evaluation request from a user to be evaluated, loading a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request.

[0031] The user to be evaluated may be a user who needs to have his or her radio gymnastics performance evaluated.

[0032] The broadcasting gymnastics evaluation request may be a request message from a user to be evaluated requesting evaluation of a broadcasting gymnastics performance. It should be noted that, in the broadcasting gymnastics evaluation request, the user to be evaluated selects a broadcasting gymnastics performance evaluation.

[0033] Optionally, in an embodiment of the present invention, the electronic device can provide an evaluation function for evaluating the performance of a radio gymnastics exercise of the user to be evaluated. The evaluation function displays at least one radio gymnastics exercise for which the user to be evaluated can perform performance evaluation. The user can select a radio gymnastics exercise in the evaluation function by operating the electronic device, thereby generating a radio gymnastics evaluation request. Exemplarily, the electronic device can be a mobile terminal, and the evaluation function can be a function provided by an application in the mobile terminal. In the application, radio gymnastics A, radio gymnastics B, and radio gymnastics C associated with the user to be evaluated are provided. After the user to be evaluated selects radio gymnastics A, a radio gymnastics evaluation request corresponding to radio gymnastics A is generated.

[0034] The target broadcasting exercise video can be a video file corresponding to the broadcasting exercise that the user to be evaluated needs to have their performance evaluated. It should be noted that the embodiment of the present invention supports the user to be evaluated to conduct online broadcasting exercise performance evaluation, requiring the user to be evaluated to provide a broadcasting exercise video for performance evaluation. Then, after the user to be evaluated issues a broadcasting exercise evaluation request, the user can respond to the broadcasting exercise evaluation request and load the target broadcasting exercise corresponding to the broadcasting exercise evaluation request for evaluation.

[0035] Optionally, in an embodiment of the present invention, when the user to be evaluated is evaluating his / her radio gymnastics performance, he / she can choose between two evaluation modes of the evaluation function, namely, real-time video evaluation and upload or download video evaluation; when the user to be evaluated chooses real-time video evaluation, he / she can download a standard radio gymnastics video and an evaluation metadata set, play the standard radio gymnastics video in an electronic device, and record the radio gymnastics of the user to be evaluated in real time through the camera function as a target radio gymnastics video; when the user to be evaluated chooses upload video evaluation, a video upload function can be provided, and the user to be evaluated can select a video file for upload through the video upload function, and after the upload is completed, the uploaded video file is loaded, and the video file is used as the target radio gymnastics video; when the user to be evaluated chooses download video evaluation, a video download function can be provided, and the user to be evaluated can select a video file for download through the video download function, and after the download is completed, the uploaded video file is loaded, and the video file is used as the target radio gymnastics video.

[0036] The evaluation metadata set may be a pre-set dataset for evaluating the performance of a broadcast calisthenics video. The evaluation metadata set is typically stored on a server. The server pre-sets a one-to-one corresponding evaluation metadata set for each type of broadcast calisthenics video. When a user to be evaluated evaluates the performance of one of the broadcast calisthenics videos, the electronic device downloads the evaluation metadata set corresponding to the broadcast calisthenics video from the server.

[0037] Optionally, the evaluation metadata set includes at least one metadata video frame, and the metadata video frame can be a video frame obtained by the server collecting video frames of the standard broadcast exercise video corresponding to each broadcast exercise video; it should be noted that the embodiment of the present invention can pre-record the standard broadcast exercise video of each broadcast exercise, and the server collects the metadata video frame of the standard broadcast exercise video to obtain the evaluation metadata set.

[0038] Optionally, the evaluation metadata set may be stored on the server in the form of a JSON file, multiple evaluation metadata sets may be stored in one JSON file, and multiple evaluation metadata sets may each be stored as a JSON file.

[0039] Optionally, the metadata video frame includes a set of bone point coordinates, the millisecond offset value and a similarity confidence, the bone point coordinate set includes at least one bone point coordinate, the bone point coordinates include 14 bone point coordinates of the head, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle in the metadata video frame, each bone point coordinate has a horizontal coordinate, a vertical coordinate and a coordinate accuracy score, the coordinate accuracy score is used to evaluate the accuracy of the bone point coordinates, the millisecond offset value can be the millisecond offset of the time of the metadata video frame in the entire standard radio gymnastics video, and the similarity confidence can be used to evaluate the degree of similarity between video frames.

[0040] Specifically, in an embodiment of the present invention, in response to a radio gymnastics evaluation request from a user to be evaluated, the target radio gymnastics video corresponding to the radio gymnastics evaluation request is identified, the standard radio gymnastics video corresponding to the target radio gymnastics video is determined, and the evaluation metadata set corresponding to the target radio gymnastics video is obtained from the server, the target radio gymnastics video and the evaluation metadata set corresponding to the radio gymnastics evaluation request are loaded, and the target radio gymnastics video is played.

[0041] S120 , sequentially collecting at least one video collection frame corresponding to each second of the radio gymnastics video of the user to be evaluated according to the playback progress of the target radio gymnastics video.

[0042] Optionally, the playback progress of the target radio gymnastics video can be understood as follows: if the target radio gymnastics video is a real-time recording of the radio gymnastics of the user to be evaluated, then the playback progress of the target radio gymnastics video is the progress of the real-time recording of the user to be evaluated; if the target radio gymnastics video is an uploaded video file, then the playback progress of the uploaded video file will be used as the playback progress of the target radio gymnastics video.

[0043] Optionally, when the target radio gymnastics video is an uploaded video file, video frames in the video file are identified, and the initial frame of the video file is adjusted to the initial frame of the standard radio gymnastics video.

[0044] Optionally, the broadcasting exercise video per second can be every second of the target broadcasting exercise video. Exemplarily, the total duration of the target broadcasting exercise video is 5 seconds, and the broadcasting exercise video per second can be the 1st second broadcasting exercise video, the 2nd second broadcasting exercise video, the 3rd second broadcasting exercise video, the 4th second broadcasting exercise video and the 5th second broadcasting exercise video.

[0045] The video capture frame may be a video frame in the broadcast gymnastics video per second. It should be noted that the broadcast gymnastics video per second may be composed of multiple video frames, and capturing the broadcast gymnastics video per second may capture multiple video capture frames. The video capture frames captured in this embodiment of the present invention include a set of skeletal point coordinates and a millisecond offset value; illustratively, 30 video capture frames may be captured per second of the broadcast gymnastics video.

[0046] Optionally, when performing video capture on the target radio gymnastics video, the capture can be performed by calling the interface for frame capture, and sequential capture can be performed according to the playback progress of the target radio gymnastics video, and the radio gymnastics video can be captured every second in turn to obtain at least one video capture frame corresponding to the radio gymnastics video per second.

[0047] S130. For the per-second broadcast calisthenics video, score the video according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame through the evaluation metadata set to determine the per-second broadcast calisthenics score corresponding to the per-second broadcast calisthenics video.

[0048] The per-second broadcasting exercise score can be a score of the broadcasting exercise movements of the user to be evaluated in the per-second broadcasting exercise video. It should be noted that, in the embodiment of the present invention, the per-second broadcasting exercise video is scored based on the video capture frames corresponding to the per-second broadcasting exercise video, thereby obtaining the per-second broadcasting exercise score corresponding to the per-second broadcasting exercise video.

[0049] Optionally, when scoring the performance of the radio gymnastics video per second, the evaluation metadata set corresponding to the target radio gymnastics video and the skeleton point coordinate set and millisecond offset value of each video acquisition frame are scored to determine the radio gymnastics score of the radio gymnastics video per second.

[0050] Specifically, each second of the broadcasting gymnastics video is scored, and the performance score is scored according to the skeleton point coordinate set and millisecond offset value of each video acquisition frame through the evaluation metadata data set to determine the second-by-second broadcasting gymnastics score corresponding to the second-by-second broadcasting gymnastics video.

[0051] S140, summarizing the radio gymnastics scores of each second of the target radio gymnastics video to obtain a total video score, performing a performance evaluation based on the total video score and the radio gymnastics total score corresponding to the target radio gymnastics video, and determining the radio gymnastics performance of the user to be evaluated.

[0052] The total video score may be the sum of the radio gymnastics scores for each second of the target radio gymnastics video.

[0053] Optionally, after obtaining the score of each second of the radio gymnastics, all the second-by-second radio gymnastics scores are aggregated and summed to obtain the total video score of the target radio gymnastics video.

[0054] The total score of the radio gymnastics may be a pre-set total score of a standard radio gymnastics video corresponding to the target radio gymnastics video.

[0055] The radio gymnastics score may be a performance evaluation score of a target radio gymnastics video of the user to be evaluated.

[0056] Specifically, the radio gymnastics scores of each second of the target radio gymnastics video are summarized to obtain the total score of the video, and the performance evaluation is performed based on the total score of the video and the total score of the radio gymnastics corresponding to the target radio gymnastics video to determine the radio gymnastics performance of the user to be evaluated.

[0057] The technical solution of the embodiment of the present invention is to load the target radio gymnastics video and the evaluation metadata set corresponding to the radio gymnastics evaluation request in response to the radio gymnastics evaluation request of the user to be evaluated. When the user selects the target radio gymnastics video for evaluation, the target radio gymnastics video and the evaluation metadata set can be loaded synchronously, thereby realizing the synchronous evaluation of the target radio gymnastics video and effectively improving the efficiency of the radio gymnastics video evaluation; at least one video acquisition frame corresponding to the radio gymnastics video of the user to be evaluated per second is collected in sequence according to the playback progress of the target radio gymnastics video, and the video frames of the radio gymnastics video are collected in real time to realize the recognition of each second of the radio gymnastics video, thereby effectively improving the accuracy of the evaluation; for the radio gymnastics video per second, the evaluation metadata set is used to collect the video frames of the radio gymnastics video in real time ... The system uses a data set to score the performance of each frame of the video capture based on the skeletal point coordinates and millisecond offset values, and determines the per-second radio gymnastics score corresponding to the per-second radio gymnastics video. By scoring each per-second radio gymnastics video, accurate scoring of the radio gymnastics video is achieved, which can improve the accuracy of the evaluation. The per-second radio gymnastics scores of the target radio gymnastics video are aggregated to obtain a total video score. The performance evaluation is performed based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video, and the radio gymnastics score of the user to be evaluated is determined. By aggregating the scores of each per-second radio gymnastics video, the performance evaluation of the entire video is completed, solving the technical problem of the inability to automatically evaluate the performance of radio gymnastics videos in the existing technology. The system can automatically monitor and evaluate the accuracy and completion of the user's radio gymnastics movements, thereby improving the efficiency and quality of radio gymnastics teaching.

[0058] Figure 2 This is a flowchart of another method for evaluating the performance of broadcasting exercises provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is a specific method for scoring the performance of broadcasting exercises video per second. Figure 2As shown, the method includes:

[0059] S210 : In response to a broadcasting gymnastics evaluation request from a user to be evaluated, loading a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request.

[0060] S220 , sequentially collecting at least one video capture frame corresponding to each second of the broadcasting exercise video of the user to be evaluated according to the playback progress of the target broadcasting exercise video.

[0061] S230. For each second of the broadcast exercise video, the millisecond offset value and the skeleton point coordinate set of the video capture frame are sequentially matched with the millisecond offset value and the skeleton point coordinate set of each metadata video frame. If there is a video capture frame that successfully matches a metadata video frame in similarity, the video capture frame is determined as the target scoring frame, and the metadata video frame is determined as the similar video frame.

[0062] The target scoring frame can be a video capture frame used to score the performance of the broadcast calisthenics video per second. It should be noted that when scoring the broadcast calisthenics video per second, a reference frame for scoring a video capture frame of the broadcast calisthenics video per second is selected, i.e., the target scoring frame. The target scoring frame is a video capture frame among all video capture frames that has a similarity to the metadata video frame.

[0063] The similar video frame may be a metadata video frame in the evaluation metadata set that matches the similarity of a video capture frame in the broadcast exercise video per second.

[0064] Optionally, in the implementation of the present invention, for multiple video capture frames captured per second of the broadcast gymnastics video, each time a video capture frame is captured, the millisecond offset value and the skeleton point coordinate set of the video capture frame are similarly matched with the millisecond offset value and the skeleton point coordinate set of each video capture frame in the evaluation metadata set. If the video capture frame successfully matches the similarity with a metadata video frame, the other video capture frames corresponding to the per second broadcast gymnastics video will no longer be similarly matched, and the successfully matched video capture frame will be directly used as the target scoring frame, and the metadata video frame will be determined as the similar video frame.

[0065] Optionally, if the video capture frame cannot be successfully matched with any metadata video frame, it is considered that the video capture frame cannot be matched. Then, according to the capture order of the broadcast gymnastics video per second, the millisecond offset value and skeleton point coordinate set of the next video capture frame are selected to perform similarity matching with the millisecond offset value and skeleton point coordinate set of each video capture frame in the evaluation metadata set until the target scoring frame and similar video frame corresponding to the broadcast gymnastics video per second are found.

[0066] Optionally, if all video acquisition frames corresponding to the broadcast gymnastics video per second cannot be successfully matched with any metadata video frame, then the broadcast gymnastics video per second does not have a target scoring frame and a similar video frame, and the score of the broadcast gymnastics video per second is set to 0.

[0067] S240 , scoring is performed according to the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the per-second broadcasting exercise score.

[0068] Optionally, for the per-second radio gymnastics video, after obtaining the target scoring frame of the per-second radio gymnastics video, the per-second radio gymnastics video is scored using the target scoring frame, and the score is scored using the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to obtain the per-second radio gymnastics score corresponding to the per-second radio gymnastics score.

[0069] Optionally, in another optional embodiment of the present invention, the step of performing a score based on the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the per-second broadcast gymnastics score includes:

[0070] Obtaining a second millisecond offset range corresponding to the similar video frame; if the millisecond offset value of the target scoring frame is not within the second millisecond offset range, evaluating the target scoring frame as the first score;

[0071] If the millisecond offset value of the target scoring frame is within the second millisecond offset range, the target scoring frame is evaluated as a second score value.

[0072] Among them, the second millisecond offset range can be a millisecond offset range for assigning scores to the target scoring frame. It should be noted that the second millisecond offset range is used to distinguish whether the target scoring frame and the similar video frame are perfect hits or ordinary hits. If the target scoring frame is an ordinary hit, the score is the first score. If the target scoring frame is a perfect hit, the score is the second score. In an embodiment of the present invention, the second score is greater than the first score, and the second score is twice the first score. Exemplarily, the first score can be set to 1 point and the second score is set to 2 points.

[0073] Optionally, in an embodiment of the present invention, a second millisecond offset range is set based on the millisecond offset of the similar video frames, and the second millisecond offset range is selected from 50 milliseconds before the millisecond offset of the similar video frames to 100 milliseconds after the millisecond offset of the similar video frames. Exemplarily, the millisecond offset of the similar video frames is represented by k1.offset, and the second millisecond offset range is set to: [k1.offset - 50, k1.offset + 100].

[0074] Optionally, in an embodiment of the present invention, a second millisecond offset range of the similar video frame is obtained based on the millisecond offset of the similar video frame, and the relationship between the target scoring frame and the second millisecond offset range is identified. If the millisecond offset value of the target scoring frame is identified as not within the second millisecond offset range, the target scoring frame is considered to be an ordinary hit, and the score of the target scoring frame is evaluated as the first score; if the millisecond offset value of the target scoring frame is identified as within the second millisecond offset range, the target scoring frame is considered to be a perfect hit, and the score of the target scoring frame is evaluated as the second score.

[0075] S250, summarizing the radio gymnastics scores of each second of the target radio gymnastics video to obtain a total video score, performing a performance evaluation based on the total video score and the radio gymnastics total score corresponding to the target radio gymnastics video, and determining the radio gymnastics performance of the user to be evaluated.

[0076] Optionally, in an embodiment of the present invention, performing a performance evaluation based on the total score of the video and the total score of the radio gymnastics corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated includes:

[0077] Calculate the score of the broadcasting gymnastics for the user to be evaluated based on the total score of the video and the total score of the broadcasting gymnastics;

[0078] If the broadcasting exercise score is not less than the preset broadcasting exercise passing score, the broadcasting exercise score of the user to be evaluated is considered valid;

[0079] If the broadcasting gymnastics score is less than the preset broadcasting gymnastics passing score, the broadcasting gymnastics score of the user to be evaluated is considered invalid.

[0080] The radio gymnastics score may be a percentage score of the total video score relative to the total radio gymnastics score under the percentage scoring rule.

[0081] Optionally, in an embodiment of the present invention, a total score for radio gymnastics is set for each type of standard radio gymnastics video through the server. When evaluating the performance, the server is requested to obtain the total score for the radio gymnastics corresponding to the target radio gymnastics video, and the radio gymnastics score per second of the target radio gymnastics video is summarized to obtain the total score for the video. Under the percentage scoring rule, a percentage calculation is performed based on the total score of the video and the total score of the radio gymnastics, and the total score of the video is converted into a radio gymnastics score. Exemplarily, the total score for radio gymnastics of the target radio gymnastics video is set to total_score in the background of the server. For similarity matching of the radio gymnastics videos per second, the radio gymnastics videos per second that do not have the target scoring frames are set to not be scored. For the radio gymnastics videos per second that do have the target scoring frames, the first score for ordinary hits is set to single_score, and the second score for perfect hits is set to perfect_score, perfect_score = single_score 2. Summarize the scores of all the broadcasting exercises videos per second and get the total score of the video as sum_score. Under the percentage scoring rule, set the broadcasting exercise score as percent_score. The specific calculation process is: floor(sum_score / total_score 100); where floor means rounding down.

[0082] Among them, the preset passing score for the broadcasting exercises can be a passing score pre-set for evaluating the broadcasting exercises performance of the user to be evaluated. It should be noted that when the broadcasting exercises score of the user to be evaluated is greater than or equal to the passing score for the broadcasting exercises, the broadcasting exercises performance of the user to be evaluated in the target broadcasting exercises video is considered to be valid; when the broadcasting exercises score of the user to be evaluated is less than the passing score for the broadcasting exercises, the broadcasting exercises performance of the user to be evaluated in the target broadcasting exercises video is considered to be invalid. For example, the passing score for the broadcasting exercises is set to 60 points, and the score of the user to be evaluated for completing the score evaluation of the target broadcasting exercises video is 59 points, the performance of the user to be evaluated in the target broadcasting exercises video is displayed as invalid, and the user's score is displayed as 59 points; if the score of the user to be evaluated for completing the score evaluation of the target broadcasting exercises video is 69 points, the performance of the user to be evaluated in the target broadcasting exercises video is displayed as a valid score, and the user's score is displayed as 69 points.

[0083] Optionally, in an embodiment of the present invention, the radio gymnastics score of each second of the target radio gymnastics video is summarized to obtain the total score of the video, and the score is calculated based on the total video score and the total radio gymnastics score to determine the radio gymnastics score of the user to be evaluated. If the radio gymnastics score is not less than the preset radio gymnastics passing score, the radio gymnastics score of the user to be evaluated is considered to be valid; if the radio gymnastics score is less than the preset radio gymnastics passing score, the radio gymnastics score of the user to be evaluated is considered to be invalid.

[0084] In an embodiment of the present invention, for each second of the broadcast calisthenics video, similarity matching is performed through the millisecond offset value of the video acquisition frame and the skeleton point coordinate set and the evaluation metadata set, and the target scoring frame of each second of the broadcast calisthenics video is determined, so that each second of the broadcast calisthenics video can be scored separately, effectively improving the efficiency of the score scoring; by classifying and scoring the target scoring frames, the accuracy of the scoring is further improved, and by aggregating the scores of each second of the broadcast calisthenics video, the score evaluation of the entire video is completed, further improving the quality and efficiency of the broadcast calisthenics video score evaluation.

[0085] Figure 3 This is a flowchart of another method for evaluating the performance of broadcasting exercises provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is a specific method for matching the target scoring frames per second of broadcasting exercises video. Figure 3 As shown, the method includes:

[0086] S310 : In response to a broadcasting gymnastics evaluation request from a user to be evaluated, loading a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request.

[0087] S320 , sequentially collecting at least one video capture frame corresponding to each second of the broadcasting exercise video of the user to be evaluated according to the playback progress of the target broadcasting exercise video.

[0088] S330. For the broadcast exercise video per second, the millisecond offset value of each video acquisition frame is matched with the millisecond offset value of each metadata video frame in turn. If there is a millisecond offset value of the video acquisition frame that successfully matches the millisecond offset value of the metadata video frame, the metadata video frame is determined as the matching video frame corresponding to the video acquisition frame.

[0089] The matching video frame may be a metadata video frame whose millisecond offset matches the video capture frame in the evaluation metadata set.

[0090] Optionally, in an embodiment of the present invention, for multiple video capture frames captured per second of the broadcast gymnastics video, each time a video capture frame is captured, the millisecond offset value of the video capture frame is similarly matched with the millisecond offset value of each video capture frame in the evaluation metadata set. If the video capture frame successfully matches the millisecond offset value of a metadata video frame, the millisecond offset value matching will no longer be performed on other video capture frames corresponding to the broadcast gymnastics video per second, and the successfully matched metadata video frame will be directly determined as the matching video frame.

[0091] Optionally, in an embodiment of the present invention, after the first matching video frame of the broadcast exercise video per second is identified, the matching of other video acquisition frames with the evaluation metadata set may be stopped directly.

[0092] Optionally, if the video capture frame cannot be successfully matched with any metadata video frame, it is considered that the video capture frame cannot be matched. Then, according to the capture order of the broadcast gymnastics video per second, the millisecond offset value of the next video capture frame is selected to match the millisecond offset value of each video capture frame in the evaluation metadata set until a matching video frame corresponding to the broadcast gymnastics video per second is found.

[0093] Optionally, if each video capture frame of the per-second broadcast calisthenics video cannot be successfully matched with any metadata video frame, it is considered that the per-second broadcast calisthenics video cannot be successfully matched with the evaluation metadata set, and the score of the per-second broadcast calisthenics video can be directly set to 0.

[0094] Specifically, for each second of broadcast exercise video, the millisecond offset value of each video capture frame is matched with the millisecond offset value of each metadata video frame in turn. If there is a millisecond offset value of a video capture frame that successfully matches the millisecond offset value of a metadata video frame, the metadata video frame is determined as the matching video frame corresponding to the video capture frame.

[0095] Optionally, in another optional embodiment of the present invention, sequentially matching the millisecond offset value of each video capture frame with the millisecond offset value of each metadata video frame, and if there is a millisecond offset value of the video capture frame that successfully matches the millisecond offset value of the metadata video frame, determining the metadata video frame as the matching video frame corresponding to the video capture frame includes:

[0096] For each metadata video frame, determining a first millisecond offset range of the metadata video frame based on the millisecond offset value of the metadata video frame;

[0097] The millisecond offset value of each video acquisition frame is identified with the first millisecond offset range of each metadata video frame in turn. If there is a video acquisition frame whose millisecond offset value is within the first millisecond offset range of a metadata video frame, it is considered that the millisecond offset value of the video acquisition frame successfully matches the millisecond offset value of a metadata video frame, and the metadata video frame is determined as the matching video frame.

[0098] The first millisecond offset range may be a millisecond offset range for matching the metadata video frame with the video acquisition frame. It should be noted that the first millisecond offset range is used to determine whether the video acquisition frame and the metadata video frame are successfully matched. The first millisecond offset range is set based on the millisecond offset of the metadata video frame. The minimum and maximum millisecond values for setting the millisecond offset of the metadata video frame are selected. For each metadata video frame, the millisecond range between the minimum and maximum millisecond values of the metadata video frame is used as the first millisecond offset range. Exemplarily, the minimum and maximum millisecond values of the millisecond offset of the metadata video frame are represented by minOffset and maxOffset, respectively. k1.offset is set to represent the millisecond offset value of the metadata video frame. minOffset = k1.offset - 200, maxOffset = k1.offset + 600 are set. The first millisecond offset range is from 200 milliseconds before the metadata video frame to 600 milliseconds after the millisecond offset of the metadata video frame. The first millisecond offset range is set to: [k1.offset - 200, k1.offset + 600].

[0099] Optionally, for the broadcast gymnastics video per second, the millisecond offset value of each video capture frame of the broadcast gymnastics video per second is compared with the first millisecond offset range of each metadata video frame in turn, and the comparison process is as follows: for a video capture frame, if the millisecond offset value of the video capture frame is not within the first millisecond offset range of a metadata video frame, then compare it with the first millisecond offset range of the next metadata video frame; if the millisecond offset value of the video capture frame is within the first millisecond offset range of the metadata video frame, then use the metadata video frame as a matching video frame, and no longer compare other video capture frames with the evaluation metadata set; if the millisecond offset value of the video capture frame is not within the first millisecond offset range of any metadata video frame, then select the next video capture frame according to the acquisition order of the video capture frames to repeat the comparison process until the matching video frame corresponding to the broadcast gymnastics video per second is found.

[0100] Specifically, for each second of radio exercise video, the millisecond offset value of each video capture frame is identified with the first millisecond offset range of each metadata video frame in turn. If there is a video capture frame whose millisecond offset value is within the first millisecond offset range of a metadata video frame, it is considered that the millisecond offset value of the video capture frame successfully matches the millisecond offset value of a metadata video frame, and the metadata video frame is determined to be a matching video frame.

[0101] S340 , performing similarity calculation based on the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score.

[0102] The similarity score may be a score of the similarity between the broadcasting calisthenics action in the matching video frame and the broadcasting calisthenics action in the video capture frame corresponding to the matching video frame.

[0103] Optionally, for the broadcasting calisthenics movements that match the video frame and the video acquisition frame, the similarity of the broadcasting calisthenics movements in the video frame can be directly evaluated by image comparison to provide a similarity score.

[0104] Optionally, in an embodiment of the present invention, similarity calculation can be performed between the matching video frame and the video acquisition frame through the skeleton point coordinate set. The specific similarity calculation process is as follows: based on the same coordinate system constructed by the matching video frame and the video acquisition frame with the video image, the similarity of the skeleton point coordinates of each skeleton part in the matching video frame and the video acquisition frame is calculated to obtain a similarity score.

[0105] Optionally, in an embodiment of the present invention, the coordinates of the skeletal points of each skeletal part in the matching video frame and the video acquisition frame can be compared one by one, and the distance calculation can be performed on the coordinates of the skeletal points of each skeletal part to obtain a distance calculation result. For the distance calculation result, a distance scoring table can be set, and the similarity score of each skeletal point coordinate can be determined based on the distance calculation result through the distance scoring table, and finally the similarity scores of all skeletal point coordinates can be summarized to obtain a similarity score. Similarly, the distance calculation results of the skeletal point coordinates of each skeletal part can be summarized, and the similarity score can be performed based on the summarized distance calculation results through the distance scoring table to obtain a similarity score.

[0106] Specifically, similarity calculation is performed based on the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score.

[0107] Optionally, in another optional embodiment of the present invention, the similarity calculation is performed based on the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score, including:

[0108] The skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame are normalized to determine the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame; similarity is calculated based on the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame using a weighted Euclidean distance formula to determine the similarity score.

[0109] Among them, the skeleton comparison coordinate set of the video acquisition frame can be a data coordinate set obtained by normalizing the coordinates of each skeleton point in the skeleton point coordinate set of the video acquisition frame; the skeleton comparison coordinate set of the matching video frame can be a data coordinate set obtained by normalizing the coordinates of each skeleton point in the skeleton point coordinate set of the matching video frame.

[0110] Optionally, for each second-by-second broadcast calisthenics video, separate data normalization is performed based on the matching video frame of the second-by-second broadcast calisthenics video and the video acquisition frame corresponding to the matching video frame. The specific process is as follows: identify the maximum horizontal coordinate, minimum horizontal coordinate, maximum vertical coordinate and minimum vertical coordinate of all bone point coordinates in the matching video frame and the video acquisition frame, and perform normalization calculation on each bone point coordinate in the matching video frame and the video acquisition frame, that is, the horizontal coordinate of each bone point coordinate is divided by the difference between the maximum horizontal coordinate and the minimum horizontal coordinate, and the vertical coordinate is divided by the difference between the maximum vertical coordinate and the minimum vertical coordinate, to obtain the normalized bone point coordinates. Then, the normalized bone point coordinates corresponding to the video acquisition frame form the bone comparison coordinate set of the video acquisition frame, and the normalized bone point coordinates corresponding to the matching video frame form the bone comparison coordinate set of the matching video frame.

[0111] Exemplarily, k1 represents the matching video frame, k2 represents the video acquisition frame corresponding to the matching video frame, the coordinates of each bone point in k1 and k2 are expressed as (abscissa x, ordinate y, coordinate accuracy score), and the maximum abscissa, minimum abscissa, maximum ordinate and minimum ordinate in k1 and k2 are expressed as: maxX, minX, maxY and minY respectively; the normalized calculation of the coordinates of each bone point can be expressed as: abscissa x: (x - minX) / (maxX - minX), ordinate y: (y - minY) / (maxY - minY), and the normalized comparison coordinate set k2c of the skeleton of the video acquisition frame is obtained, and the normalized comparison coordinate set k1c of the skeleton of the matching video frame is obtained. Since the coordinate accuracy score of each bone point coordinate is not included in the calculation, assuming that k1 and k2 only include the head and neck, the bone point coordinates of k1 and k2 are expressed as k1 head (100, 100), k1 neck (200, 250), k2 head (101, 102), k2 neck (201, 251), then minX = min(100, 200, 101, 201) = 100, maxX = max(100, 200, 101, 201) = 201, minY = min(100, 250, 102, 251) = 100, maxY=max(100, 250, 102, 251) = 251. Then the normalized coordinates of k1's head are calculated to get the coordinates of k1c's head: ((100 - 100) / (201 - 100), (100 - 100) / (251 -100)) = (0, 0), the normalized coordinates of k2's head are calculated to get the coordinates of k2c's head: ((101 -100) / (201 - 100), (102 - 100) / (251 - 100)) = (0.009, 0.013), and so on. Based on the above calculation process, when the skeletal points of k1 and k2 are the head, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle, the coordinates of k1c and k2c of each part can be calculated using the same calculation method.

[0112] Optionally, a weighted Euclidean distance formula can be used to evaluate the similarity between the matching video frame and the video acquisition frame; the weighted Euclidean distance formula is used to calculate the similarity based on the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame to obtain a similarity score. The specific calculation process is as follows:

[0113] Since each bone point coordinate of the matching video frame k1 will have a coordinate accuracy score, for example, the bone point coordinates of k1's head are expressed as (head horizontal coordinate x, head vertical coordinate y, coordinate accuracy score), and the bone point coordinates of k1's neck are expressed as (neck horizontal coordinate x, neck vertical coordinate y, coordinate accuracy score), and so on, the bone point coordinates of k1's left shoulder, k1's right shoulder, k1's left elbow, k1's right elbow, k1's left wrist, k1's right wrist, k1's left hip, k1's right hip, k1's left knee, k1's right knee, k1's left ankle, and k1's right ankle also have corresponding coordinate accuracy scores; the coordinate accuracy scores of each bone point of k1 are summed to obtain the first coordinate score k1_score, and then the weighted Euclidean scores of k2 and k1 are calculated, and the weighted Euclidean scores of k2 and k1 are represented by weight_score, and the weighted Euclidean scores of k2 and k1 are represented by weight_score. represents the coordinate accuracy score of the i-th bone point in the matching video frame k1, where i is used to represent the i-th bone point in the order of k1 head, k1 neck, k1 left shoulder, k1 right shoulder, k1 left elbow, k1 right elbow, k1 left wrist, k1 right wrist, k1 left hip, k1 right hip, k1 left knee, k1 right knee, k1 left ankle and k1 right ankle; The x-axis coordinate of the i-th bone point in the same arrangement order as K1 is used to represent the normalized comparison coordinate set k1c of the matched video frame; The y-axis coordinate of the i-th bone point in the same arrangement order as K1 is used to represent the normalized comparison coordinate set k1c of the matched video frame; The normalized contrast coordinate set k2c of the skeleton used to represent the video capture frame is the x-axis coordinate of the i-th skeleton point in the same arrangement order as K1; The normalized comparison coordinate set k2c of the skeleton used to represent the matching video frame is the y-axis coordinate of the i-th skeleton point in the same arrangement order as K1. The calculation formula of the weighted Euclidean score of k2 and k1 is as follows:

[0114]

[0115] Then calculate the similarity score, which is represented by resultSim. The similarity score is calculated as follows: resultSim = weight_score / k1_score.

[0116] S350. If the similarity score is less than the similarity confidence of the matching video frame, it is considered that there is a successful similarity match between the video acquisition frame and the metadata video frame, and the matching video frame is determined as the similar video frame, and the video acquisition frame is determined as the target scoring frame.

[0117] Optionally, obtain the similarity confidence of the matching video frame, and judge the similarity confidence of the matching video frame and the similarity score. If the similarity score is less than the similarity confidence of the matching video frame, it is considered that the similarity between a video capture frame and a metadata video frame is successfully matched. The matching video frame is determined as the similar video frame, and the video capture frame is determined as the target scoring frame. Exemplarily, the similarity confidence of the matching video frame can be expressed as k1_sim. If the similarity score resultSim is less than the similarity confidence k1_sim, that is, resultSim < k1_sim, it is considered that k2 and k1 are successfully matched.

[0118] Optionally, if the similarity score is not less than the similarity confidence of the matching video frame, it is considered that the similarity between a video capture frame and a metadata video frame fails to match, and the score of the broadcast exercise per second of the video is directly set to 0.

[0119] S360. Perform score evaluation according to the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame, and determine the score of the broadcast exercise per second.

[0120] S370. Aggregate the scores of the broadcast exercise per second of each target broadcast exercise video to obtain the total video score, and perform score evaluation according to the total video score and the total score of the broadcast exercise corresponding to the target broadcast exercise video, and determine the broadcast exercise score of the user to be evaluated.

[0121] In the embodiment of the present invention, for the broadcast exercise video per second, a matching video frame is found in the evaluation metadata set through the millisecond offset value of the video capture frame. After the matching video frame is determined, the matching process of the matching video frame is stopped, which effectively improves the evaluation efficiency. After the matching video frame is determined, the matching video frame and the video capture frame are normalized based on the skeletal point coordinate set, and similarity matching is performed to determine the target scoring frame of each broadcast exercise video per second, realizing accurate and individual scoring of each second of the broadcast exercise video, effectively improving the accuracy of the score evaluation, and further improving the quality and efficiency of the broadcast exercise video score evaluation.

[0122] Figure 4 It is a schematic structural diagram of a broadcast exercise score evaluation device provided by an embodiment of the present invention. As Figure 4 shown, the device includes: a loading module 410, a collection module 420, a scoring module 430, and an evaluation module 440; wherein,

[0123] The loading module 410 is configured to load the target broadcast exercise video and the evaluation metadata set corresponding to the broadcast exercise evaluation request in response to the broadcast exercise evaluation request of the user to be evaluated;

[0124] The acquisition module 420 is configured to sequentially acquire at least one video acquisition frame corresponding to each second of the broadcasting exercise video of the user to be evaluated according to the playback progress of the target broadcasting exercise video; wherein the video acquisition frame includes a skeleton point coordinate set and a millisecond offset value;

[0125] Scoring module 430, configured to score the per-second broadcast calisthenics video based on the skeleton point coordinate set and the millisecond offset value of each video capture frame using the evaluation metadata set, and determine a per-second broadcast calisthenics score corresponding to the per-second broadcast calisthenics video;

[0126] The evaluation module 440 is used to summarize the radio gymnastics scores of each second of the target radio gymnastics video to obtain the total score of the video, perform performance evaluation based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video, and determine the radio gymnastics score of the user to be evaluated.

[0127] The technical solution of the embodiment of the present invention is to load the target radio gymnastics video and the evaluation metadata set corresponding to the radio gymnastics evaluation request in response to the radio gymnastics evaluation request of the user to be evaluated. When the user selects the target radio gymnastics video for evaluation, the target radio gymnastics video and the evaluation metadata set can be loaded synchronously, thereby realizing the synchronous evaluation of the target radio gymnastics video and effectively improving the efficiency of the radio gymnastics video evaluation; at least one video acquisition frame corresponding to the radio gymnastics video of the user to be evaluated per second is collected in sequence according to the playback progress of the target radio gymnastics video, and the video frames of the radio gymnastics video are collected in real time to realize the recognition of each second of the radio gymnastics video, thereby effectively improving the accuracy of the evaluation; for the radio gymnastics video per second, the evaluation metadata set is used to collect the video frames of the radio gymnastics video in real time ... The system uses a data set to score the performance of each frame of the video capture based on the skeletal point coordinates and millisecond offset values, and determines the per-second radio gymnastics score corresponding to the per-second radio gymnastics video. By scoring each per-second radio gymnastics video, accurate scoring of the radio gymnastics video is achieved, which can improve the accuracy of the evaluation. The per-second radio gymnastics scores of the target radio gymnastics video are aggregated to obtain a total video score. The performance evaluation is performed based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video, and the radio gymnastics score of the user to be evaluated is determined. By aggregating the scores of each per-second radio gymnastics video, the performance evaluation of the entire video is completed, solving the technical problem of the inability to automatically evaluate the performance of radio gymnastics videos in the existing technology. The system can automatically monitor and evaluate the accuracy and completion of the user's radio gymnastics movements, thereby improving the efficiency and quality of radio gymnastics teaching.

[0128] Optionally, the evaluation metadata set includes at least one metadata video frame; the metadata video frame includes the skeleton point coordinate set, the millisecond offset value and the similarity confidence;

[0129] Optionally, the scoring module 430 is specifically configured to:

[0130] Sequentially performing similarity matching on the millisecond offset value and the skeleton point coordinate set of the video capture frame and the millisecond offset value and the skeleton point coordinate set of each metadata video frame; if there is a video capture frame that successfully matches a metadata video frame in similarity, then determining the video capture frame as a target scoring frame and determining the metadata video frame as a similar video frame;

[0131] The performance score is scored according to the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the per-second broadcasting exercise score.

[0132] Optionally, the scoring module 430 is further configured to:

[0133] Matching the millisecond offset value of each video capture frame with the millisecond offset value of each metadata video frame in sequence; if there is a millisecond offset value of the video capture frame that successfully matches the millisecond offset value of the metadata video frame, determining the metadata video frame as the matching video frame corresponding to the video capture frame;

[0134] Calculating similarity between the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score;

[0135] If the similarity score is less than the similarity confidence of the matching video frame, it is considered that there is a successful similarity match between the video acquisition frame and the metadata video frame, the matching video frame is determined as the similar video frame, and the video acquisition frame is determined as the target scoring frame.

[0136] Optionally, the scoring module 430 is further configured to:

[0137] For each metadata video frame, determining a first millisecond offset range of the metadata video frame based on the millisecond offset value of the metadata video frame;

[0138] The millisecond offset value of each video acquisition frame is identified with the first millisecond offset range of each metadata video frame in turn. If there is a video acquisition frame whose millisecond offset value is within the first millisecond offset range of a metadata video frame, it is considered that the millisecond offset value of the video acquisition frame successfully matches the millisecond offset value of a metadata video frame, and the metadata video frame is determined as the matching video frame.

[0139] Optionally, the skeleton point coordinate set includes at least one skeleton point coordinate;

[0140] Optionally, the scoring module 430 is further configured to:

[0141] Normalizing the coordinates of each of the skeleton points in the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to determine the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame;

[0142] A similarity calculation is performed based on the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame by using a weighted Euclidean distance formula to determine the similarity score.

[0143] Optionally, the scoring module 430 is further configured to:

[0144] Obtaining a second millisecond offset range corresponding to the similar video frame;

[0145] If the millisecond offset value of the target scoring frame is not within the second millisecond offset range, evaluating the target scoring frame as a first score value;

[0146] If the millisecond offset value of the target scoring frame is within the second millisecond offset range, the target scoring frame is evaluated as a second score value.

[0147] Optionally, the evaluation module 440 is specifically configured to:

[0148] Calculate the score of the broadcasting gymnastics for the user to be evaluated based on the total score of the video and the total score of the broadcasting gymnastics;

[0149] If the broadcasting exercise score is not less than the preset broadcasting exercise passing score, the broadcasting exercise score of the user to be evaluated is considered valid;

[0150] If the broadcasting gymnastics score is less than the preset broadcasting gymnastics passing score, the broadcasting gymnastics score of the user to be evaluated is considered invalid.

[0151] The broadcast gymnastics performance evaluation device provided in the embodiment of the present invention can execute the broadcast gymnastics performance evaluation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0152] Figure 5A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their modes are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0153] like Figure 5 As shown, electronic device 10 includes at least one processor 11 and memory, such as read-only memory (ROM) 12 and random access memory (RAM) 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by the at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer programs stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for the operation of electronic device 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. An input / output (I / O) interface 15 is also connected to bus 14.

[0154] Various components in electronic device 10 are connected to an input / output (I / O) interface 15, including an input unit 16, such as a keyboard and mouse; an output unit 17, such as various types of displays and speakers; a storage unit 18, such as a magnetic disk and optical disk; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0155] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as the method for evaluating broadcast calisthenics performance.

[0156] In some embodiments, the broadcasting gymnastics performance evaluation method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the read-only memory (ROM) 12 and / or the communication unit 19. When the computer program is loaded into the random access memory (RAM) 13 and executed by the processor 11, one or more steps of the broadcasting gymnastics performance evaluation method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the broadcasting gymnastics performance evaluation method in any other appropriate manner (for example, by means of firmware).

[0157] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0158] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the modes / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0159] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0160] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0161] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0162] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0163] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0164] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the broadcast gymnastics performance evaluation method provided in any embodiment of the present invention are implemented. The method includes:

[0165] In response to a broadcasting gymnastics evaluation request from a user to be evaluated, loading a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request;

[0166] At least one video capture frame corresponding to each second of the radio gymnastics video of the user to be evaluated is captured in sequence according to the playback progress of the target radio gymnastics video; wherein the video capture frame includes a skeleton point coordinate set and a millisecond offset value;

[0167] For the per-second broadcast calisthenics video, score the video according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame using the evaluation metadata set to determine the per-second broadcast calisthenics score corresponding to the per-second broadcast calisthenics video;

[0168] The radio gymnastics scores of each second of the target radio gymnastics video are summarized to obtain the total score of the video, and a performance evaluation is performed based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated.

[0169] The computer storage medium of the embodiments of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0170] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0171] Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0172] The computer program code for performing the operations of the present invention can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0173] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computing device. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they can be implemented using program code executable by a computer device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

[0174] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0175] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for evaluating the performance of broadcasting exercises, characterized in that: include: In response to a broadcasting gymnastics evaluation request from a user to be evaluated, loading a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request; At least one video capture frame corresponding to each second of the radio gymnastics video of the user to be evaluated is captured in sequence according to the playback progress of the target radio gymnastics video; wherein the video capture frame includes a skeleton point coordinate set and a millisecond offset value; For the per-second broadcast calisthenics video, a score is performed based on the skeleton point coordinate set and the millisecond offset value of each video capture frame using the evaluation metadata set to determine a per-second broadcast calisthenics score corresponding to the per-second broadcast calisthenics video; wherein the per-second broadcast calisthenics score is a score of the broadcast calisthenics movements of the user to be evaluated in the per-second broadcast calisthenics video; Summarizing the radio gymnastics scores of each second of the target radio gymnastics video to obtain a total video score, performing a performance evaluation based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video, and determining the radio gymnastics score of the user to be evaluated; The evaluation metadata set includes at least one metadata video frame; the metadata video frame includes the skeleton point coordinate set, the millisecond offset value and the similarity confidence; The step of performing a score based on the skeleton point coordinate set and the millisecond offset value of each video acquisition frame through the evaluation metadata set to determine a per-second broadcasting exercise score corresponding to the per-second broadcasting exercise video includes: Sequentially performing similarity matching on the millisecond offset value and the skeleton point coordinate set of the video capture frame and the millisecond offset value and the skeleton point coordinate set of each metadata video frame; if there is a video capture frame that successfully matches a metadata video frame in similarity, then determining the video capture frame as a target scoring frame and determining the metadata video frame as a similar video frame; Scoring is performed based on the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the per-second broadcasting exercise score; The method sequentially performing similarity matching on the millisecond offset value and the skeleton point coordinate set of the video acquisition frame with the millisecond offset value and the skeleton point coordinate set of each metadata video frame, and if there is a video acquisition frame that successfully matches the similarity with a metadata video frame, determining the video acquisition frame as a target scoring frame and determining the metadata video frame as a similar video frame, includes: Sequentially matching the millisecond offset value of each video capture frame with the millisecond offset value of each metadata video frame; if there is a successful match between the millisecond offset value of the video capture frame and the millisecond offset value of the metadata video frame, determining the metadata video frame as the matching video frame corresponding to the video capture frame; wherein the matching video frame is the metadata video frame whose millisecond offset matches the video capture frame in the evaluation metadata set; Calculating similarity between the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score; If the similarity score is less than the similarity confidence of the matching video frame, it is considered that there is a successful similarity match between the video acquisition frame and the metadata video frame, the matching video frame is determined as the similar video frame, and the video acquisition frame is determined as the target scoring frame.

2. The method according to claim 1, characterized in that The step of sequentially matching the millisecond offset value of each video capture frame with the millisecond offset value of each metadata video frame, and if there is a millisecond offset value of the video capture frame that successfully matches the millisecond offset value of the metadata video frame, determining the metadata video frame as a matching video frame corresponding to the video capture frame, includes: For each metadata video frame, determining a first millisecond offset range of the metadata video frame based on the millisecond offset value of the metadata video frame; The millisecond offset value of each video acquisition frame is identified with the first millisecond offset range of each metadata video frame in turn. If there is a video acquisition frame whose millisecond offset value is within the first millisecond offset range of a metadata video frame, it is considered that the millisecond offset value of the video acquisition frame successfully matches the millisecond offset value of a metadata video frame, and the metadata video frame is determined as the matching video frame.

3. The method according to claim 2, characterized in that The skeleton point coordinate set includes at least one skeleton point coordinate; and performing similarity calculation based on the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score includes: Normalizing the coordinates of each of the skeleton points in the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to determine the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame; A similarity calculation is performed based on the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame by using a weighted Euclidean distance formula to determine the similarity score.

4. The method according to claim 1, wherein The step of scoring the performance according to the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the per-second broadcasting exercise score includes: Obtaining a second millisecond offset range corresponding to the similar video frame; If the millisecond offset value of the target scoring frame is not within the second millisecond offset range, evaluating the target scoring frame as a first score value; If the millisecond offset value of the target scoring frame is within the second millisecond offset range, the target scoring frame is evaluated as a second score value.

5. The method according to claim 1, wherein The step of performing a performance evaluation based on the total score of the video and the total score of the radio gymnastics corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated includes: Calculate the score of the broadcasting gymnastics for the user to be evaluated based on the total score of the video and the total score of the broadcasting gymnastics; If the broadcasting exercise score is not less than the preset broadcasting exercise passing score, the broadcasting exercise score of the user to be evaluated is considered valid; If the broadcasting gymnastics score is less than the preset broadcasting gymnastics passing score, the broadcasting gymnastics score of the user to be evaluated is considered invalid.

6. A radio gymnastics performance evaluation device, characterized in that: include: A loading module, configured to respond to a broadcasting exercise evaluation request from a user to be evaluated and load a target broadcasting exercise video and an evaluation metadata set corresponding to the broadcasting exercise evaluation request; An acquisition module is configured to sequentially acquire at least one video acquisition frame corresponding to each second of the broadcasting calisthenics video of the user to be evaluated according to the playback progress of the target broadcasting calisthenics video; wherein the video acquisition frame includes a skeleton point coordinate set and a millisecond offset value; A scoring module is configured to score the per-second broadcast calisthenics video based on the skeleton point coordinate set and the millisecond offset value of each video capture frame using the evaluation metadata set, and determine a per-second broadcast calisthenics score corresponding to the per-second broadcast calisthenics video; wherein the per-second broadcast calisthenics score is a score of the broadcast calisthenics movements of the user to be evaluated in the per-second broadcast calisthenics video; An evaluation module is configured to summarize the radio gymnastics scores of each second of the target radio gymnastics video to obtain a total video score, perform a performance evaluation based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video, and determine the radio gymnastics score of the user to be evaluated; The evaluation metadata set includes at least one metadata video frame; the metadata video frame includes the skeleton point coordinate set, the millisecond offset value and the similarity confidence; The scoring module is specifically used to: Sequentially performing similarity matching on the millisecond offset value and the skeleton point coordinate set of the video capture frame and the millisecond offset value and the skeleton point coordinate set of each metadata video frame; if there is a video capture frame that successfully matches a metadata video frame in similarity, then determining the video capture frame as a target scoring frame and determining the metadata video frame as a similar video frame; Scoring is performed based on the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the per-second broadcasting exercise score; The scoring module is also used to: Sequentially matching the millisecond offset value of each video capture frame with the millisecond offset value of each metadata video frame; if there is a successful match between the millisecond offset value of the video capture frame and the millisecond offset value of the metadata video frame, determining the metadata video frame as the matching video frame corresponding to the video capture frame; wherein the matching video frame is the metadata video frame whose millisecond offset matches the video capture frame in the evaluation metadata set; Calculating similarity between the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score; If the similarity score is less than the similarity confidence of the matching video frame, it is considered that there is a successful similarity match between the video acquisition frame and the metadata video frame, the matching video frame is determined as the similar video frame, and the video acquisition frame is determined as the target scoring frame.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the broadcast gymnastics performance evaluation method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the broadcast gymnastics performance evaluation method according to any one of claims 1 to 5 when executed.

Images