Broadcast operation score evaluation method, device, equipment and medium

Abstract

The invention discloses a broadcasting operation score evaluation method, device and equipment and a medium. The method is characterized by comprising the following steps: in response to a broadcast operation evaluation request of a to-be-evaluated user, loading a target broadcast operation video and an evaluation metadata set corresponding to the broadcast operation evaluation request; acquiring at least one video acquisition frame corresponding to the broadcast video per second of the user to be evaluated according to the playing progress sequence of the target broadcast video; for the broadcast operation video per second, score scoring is carried out according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame through the evaluation metadata set, and a broadcast operation score per second corresponding to the broadcast operation video per second is determined; and summarizing each second broadcast operation score of the target broadcast operation video to obtain a video total score, performing score assessment according to the video total score and the broadcast operation total score corresponding to the target broadcast operation video, and determining the broadcast operation score of the user to be assessed. The motion accuracy and completeness of user broadcasting gymnastics can be automatically monitored and evaluated, and the efficiency and quality of broadcasting gymnastics teaching are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of online broadcasting exercises, and in particular to a method, device, equipment and medium for evaluating broadcasting exercises performance. Background Art

[0002] As a form of physical exercise with high popularity, radio gymnastics is of great significance to improving people's physical health and quality of life. However, traditional radio gymnastics is accepted by radio gymnastics teachers offline. Radio gymnastics teachers need to spend a lot of time to check whether students' movements are standard one by one, and evaluate the completion and completion results of students' radio gymnastics. Traditional radio gymnastics acceptance is very inconvenient. With the development of the Internet, the development of online radio gymnastics has enabled teaching to be conducted through one-to-many video or live broadcast teaching. Although students can submit the acceptance and evaluation of radio gymnastics in the form of videos, radio gymnastics teachers are still required to check and evaluate the videos one by one. Although the existing technology can detect radio gymnastics videos through image recognition algorithms, the detection accuracy is very low, and it is impossible to accurately evaluate the quality of radio gymnastics videos, let alone the performance of radio gymnastics videos. Summary of the invention

[0003] The present invention provides a method, device, equipment and medium for evaluating the performance of a broadcasting gymnastics, so as to solve the technical problem in the prior art that the performance of a broadcasting gymnastics video cannot be automatically evaluated.

[0004] According to one aspect of the present invention, a method for evaluating broadcasting gymnastics performance is provided, comprising:

[0005] In response to a broadcasting gymnastics evaluation request from a user to be evaluated, loading a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request;

[0006] At least one video acquisition frame corresponding to each second of the broadcasting exercise video of the user to be evaluated is collected in sequence according to the playback progress of the target broadcasting exercise video; wherein the video acquisition frame includes a skeleton point coordinate set and a millisecond offset value;

[0007] For the per-second broadcasting gymnastics video, score the per-second broadcasting gymnastics video according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame through the evaluation metadata set, and determine the per-second broadcasting gymnastics score corresponding to the per-second broadcasting gymnastics video;

[0008] The radio gymnastics scores of each second of the target radio gymnastics video are summarized to obtain the total score of the video, and a performance evaluation is performed based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated.

[0009] According to another aspect of the present invention, there is provided a broadcast gymnastics performance evaluation device, comprising:

[0010] A loading module, configured to respond to a radio gymnastics evaluation request from a user to be evaluated and load a target radio gymnastics video and an evaluation metadata set corresponding to the radio gymnastics evaluation request;

[0011] A collection module, used to collect at least one video collection frame corresponding to each second of the broadcasting exercise video of the user to be evaluated according to the playback progress sequence of the target broadcasting exercise video; wherein the video collection frame includes a skeleton point coordinate set and a millisecond offset value;

[0012] A scoring module is used to score the per-second broadcasting exercise video according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame through the evaluation metadata set, and determine the per-second broadcasting exercise score corresponding to the per-second broadcasting exercise video;

[0013] An evaluation module is used to summarize the radio gymnastics scores of each second of the target radio gymnastics video to obtain a total video score, and to perform a performance evaluation based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the broadcast gymnastics performance evaluation method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the broadcast gymnastics performance evaluation method described in any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention, by responding to the broadcasting exercise evaluation request of the user to be evaluated, loading the target broadcasting exercise video and the evaluation metadata set corresponding to the broadcasting exercise evaluation request, when the user selects the target broadcasting exercise video for evaluation, the target broadcasting exercise video and the evaluation metadata set can be loaded synchronously, so as to realize the synchronous evaluation of the target broadcasting exercise video, and effectively improve the efficiency of the broadcasting exercise video evaluation; according to the playback progress sequence of the target broadcasting exercise video, at least one video acquisition frame corresponding to the broadcasting exercise video of the user to be evaluated per second is collected, and the video frames of the broadcasting exercise video are collected in real time to realize the recognition of each second of the broadcasting exercise video, so as to effectively improve the accuracy of the evaluation; for the broadcasting exercise video per second, the evaluation metadata set is used to identify the broadcasting exercise video of the user to be evaluated, and the evaluation metadata set is used to identify the broadcasting exercise video of the user to be evaluated ... The data set is used to score the score according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame, and the score of the radio gymnastics per second corresponding to the radio gymnastics video per second is determined. By scoring the score of each second of the radio gymnastics video, the accurate scoring of the radio gymnastics video can be achieved, which can improve the accuracy of the evaluation; the score of each second of the radio gymnastics of the target radio gymnastics video is summarized to obtain the total score of the video, and the score evaluation is performed according to the total score of the video and the total score of the radio gymnastics corresponding to the target radio gymnastics video, and the radio gymnastics score of the user to be evaluated is determined. By summarizing the score of each second of the radio gymnastics video, the score evaluation of the entire video is completed, which solves the technical problem that the radio gymnastics video cannot be automatically evaluated in the prior art. It can automatically monitor and evaluate the accuracy and completion of the user's radio gymnastics movements, and improve the efficiency and quality of radio gymnastics teaching.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of a method for evaluating broadcast gymnastics performance is provided for an embodiment of the present invention;

[0023] Figure 2 A flowchart of another method for evaluating broadcast gymnastics scores provided by an embodiment of the present invention;

[0024] Figure 3 A flowchart of another method for evaluating broadcast gymnastics scores provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of the structure of a broadcast gymnastics performance evaluation device provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of the structure of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Figure 1 A flowchart of a method for evaluating the performance of a broadcasting gymnastics is provided for an embodiment of the present invention. This embodiment is applicable to the situation where the performance of a user's broadcasting gymnastics is evaluated online in real time. The method can be executed by a broadcasting gymnastics performance evaluation device. The broadcasting gymnastics performance evaluation device can be implemented in the form of hardware and / or software. The broadcasting gymnastics performance evaluation device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0030] S110 . In response to a broadcasting gymnastics evaluation request from a user to be evaluated, a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request are loaded.

[0031] Among them, the user to be evaluated may be a user who needs to have his / her radio gymnastics performance evaluated.

[0032] The broadcasting exercise evaluation request may be request information for the user to be evaluated to evaluate the performance of the broadcasting exercise. It should be noted that in the broadcasting exercise evaluation request, the user to be evaluated selects the broadcasting exercise for which the performance is to be evaluated.

[0033] Optionally, in an embodiment of the present invention, the electronic device can provide an evaluation function for evaluating the performance of a broadcasting exercise of the user to be evaluated. The evaluation function displays at least one broadcasting exercise that the user to be evaluated can evaluate. The user can select a broadcasting exercise of the evaluation function by operating in the electronic device, thereby generating a broadcasting exercise evaluation request. Exemplarily, the electronic device can be a mobile terminal, and the evaluation function can be a function provided by an application in the mobile terminal. In the application, broadcasting exercises A, B, and C associated with the user to be evaluated are provided. After the user to be evaluated selects broadcasting exercise A, a broadcasting exercise evaluation request corresponding to broadcasting exercise A is generated.

[0034] The target broadcasting exercise video may be a video file corresponding to the broadcasting exercise that the user to be evaluated needs to evaluate. It should be noted that the embodiment of the present invention supports the user to be evaluated to perform online broadcasting exercise performance evaluation, and the user to be evaluated needs to provide a broadcasting exercise video for performance evaluation, and then after the user to be evaluated issues a broadcasting exercise evaluation request, the user can respond to the broadcasting exercise evaluation request and load the target broadcasting exercise corresponding to the broadcasting exercise evaluation request for evaluation.

[0035] Optionally, in an embodiment of the present invention, when the user to be evaluated is evaluating his / her radio gymnastics performance, he / she can choose between two evaluation modes of the evaluation function, namely, real-time video evaluation and upload or download video evaluation; when the user to be evaluated chooses real-time video evaluation, he / she can download a standard radio gymnastics video and an evaluation metadata set, play the standard radio gymnastics video in an electronic device, and record the radio gymnastics of the user to be evaluated in real time through a camera function as a target radio gymnastics video; when the user to be evaluated chooses upload video evaluation, a video upload function can be provided, and the user to be evaluated can select a video file for upload through the video upload function, and after the upload is completed, the uploaded video file is loaded, and the video file is used as the target radio gymnastics video; when the user to be evaluated chooses download video evaluation, a video download function can be provided, and the user to be evaluated can select a video file for download through the video download function, and after the download is completed, the uploaded video file is loaded, and the video file is used as the target radio gymnastics video.

[0036] The evaluation metadata set may be a data set pre-set for evaluating the performance of a broadcasting exercise video. The evaluation metadata set is usually stored in a server, which pre-sets a one-to-one corresponding evaluation metadata set for each type of broadcasting exercise video. When the user to be evaluated evaluates the performance of one of the broadcasting exercise videos, the electronic device downloads the evaluation metadata set corresponding to the broadcasting exercise video from the server.

[0037] Optionally, the evaluation metadata set includes at least one metadata video frame, and the metadata video frame may be a video frame obtained by the server collecting video frames of a standard broadcast exercise video corresponding to each broadcast exercise video; it should be noted that the embodiment of the present invention can pre-record a standard broadcast exercise video for each broadcast exercise, and the server collects metadata video frames of the standard broadcast exercise video to obtain the evaluation metadata set.

[0038] Optionally, the evaluation metadata set may be stored in the form of a json file on the server side, multiple evaluation metadata sets may be stored in one json file, and multiple evaluation metadata sets may each be stored as a join file.

[0039] Optionally, the metadata video frame includes a set of bone point coordinates, the millisecond offset value and a similarity confidence, the bone point coordinate set includes at least one bone point coordinate, the bone point coordinates include 14 bone point coordinates of the head, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle in the metadata video frame, each bone point coordinate has a horizontal coordinate, a vertical coordinate and a coordinate accuracy score, the coordinate accuracy score is used to evaluate the accuracy of the bone point coordinates, the millisecond offset value can be the millisecond offset of the time of the metadata video frame in the entire standard radio gymnastics video, and the similarity confidence can be used to evaluate the similarity between video frames.

[0040] Specifically, in an embodiment of the present invention, in response to a radio gymnastics evaluation request from a user to be evaluated, a target radio gymnastics video corresponding to the radio gymnastics evaluation request is identified, a standard radio gymnastics video corresponding to the target radio gymnastics video is determined, and an evaluation metadata set corresponding to the target radio gymnastics video is obtained from the server, the target radio gymnastics video and the evaluation metadata set corresponding to the radio gymnastics evaluation request are loaded, and the target radio gymnastics video is played.

[0041] S120, sequentially collecting at least one video collection frame corresponding to the broadcasting exercise video of the user to be evaluated per second according to the playing progress of the target broadcasting exercise video.

[0042] Optionally, the playback progress of the target radio gymnastics video can be understood as: if the target radio gymnastics video is a real-time recording of the radio gymnastics of the user to be evaluated, then the playback progress of the target radio gymnastics video is the progress of the real-time recording of the user to be evaluated; if the target radio gymnastics video is an uploaded video file, then the playback progress of the uploaded video file will be used as the playback progress of the target radio gymnastics video.

[0043] Optionally, when the target radio exercise video is an uploaded video file, the video frames in the video file are identified, and the initial frame of the video file is adjusted to the initial frame of the standard radio exercise video.

[0044] Optionally, the broadcasting exercise video per second can be every second of the broadcasting exercise video of the target broadcasting exercise video. Exemplarily, the total duration of the target broadcasting exercise video is 5 seconds, and the broadcasting exercise video per second can be the 1st second broadcasting exercise video, the 2nd second broadcasting exercise video, the 3rd second broadcasting exercise video, the 4th second broadcasting exercise video and the 5th second broadcasting exercise video.

[0045] Among them, the video acquisition frame can be a video frame in the broadcasting exercise video per second. It should be noted that the broadcasting exercise video per second can be composed of multiple video frames, and the acquisition of the broadcasting exercise video per second can acquire multiple video acquisition frames. The video acquisition frame acquired in the embodiment of the present invention includes a skeleton point coordinate set and a millisecond offset value; illustratively, 30 video acquisition frames can be acquired per second of the broadcasting exercise video.

[0046] Optionally, when performing video capture on the target broadcast gymnastics video, the capture can be performed by calling an interface for frame capture, and sequential capture is performed according to the playback progress of the target broadcast gymnastics video, and the broadcast gymnastics video is captured every second in turn to obtain at least one video capture frame corresponding to the broadcast gymnastics video per second.

[0047] S130. For the per-second broadcast gymnastics video, score the per-second broadcast gymnastics video according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame through the evaluation metadata set to determine the per-second broadcast gymnastics score corresponding to the per-second broadcast gymnastics video.

[0048] The score of the radio gymnastics per second can be the score of the radio gymnastics movements of the user to be evaluated in the radio gymnastics video per second. It should be noted that, in the embodiment of the present invention, the radio gymnastics video per second is scored by the video acquisition frames corresponding to the radio gymnastics video per second, thereby obtaining the score of the radio gymnastics per second corresponding to the radio gymnastics video per second.

[0049] Optionally, when scoring the performance of the radio gymnastics video per second, the evaluation metadata set corresponding to the target radio gymnastics video and the skeleton point coordinate set and millisecond offset value of each video acquisition frame are scored to determine the radio gymnastics score per second of the radio gymnastics video.

[0050] Specifically, a score is given to each second of the radio gymnastics video, and a score is given according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame by evaluating the metadata data set to determine the second-by-second radio gymnastics score corresponding to each second of the radio gymnastics video.

[0051] S140, summarizing the radio gymnastics scores of each second of the target radio gymnastics video to obtain a total video score, performing a performance evaluation based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video, and determining the radio gymnastics performance of the user to be evaluated.

[0052] The total video score may be the sum of the radio gymnastics scores for each second of the target radio gymnastics video.

[0053] Optionally, after obtaining the score of each second of the radio gymnastics, all the scores of the radio gymnastics per second are summarized and summed to obtain the total video score of the target radio gymnastics video.

[0054] Among them, the total score of the radio gymnastics can be the pre-set total score of the standard radio gymnastics video corresponding to the target radio gymnastics video.

[0055] The radio gymnastics score may be a performance evaluation score of a target radio gymnastics video of the user to be evaluated.

[0056] Specifically, the radio gymnastics scores of each second of the target radio gymnastics video are summarized to obtain the total score of the video, and a performance evaluation is performed based on the total score of the video and the total score of the radio gymnastics corresponding to the target radio gymnastics video to determine the radio gymnastics performance of the user to be evaluated.

[0057] The technical solution of the embodiment of the present invention, by responding to the broadcasting exercise evaluation request of the user to be evaluated, loading the target broadcasting exercise video and the evaluation metadata set corresponding to the broadcasting exercise evaluation request, when the user selects the target broadcasting exercise video for evaluation, the target broadcasting exercise video and the evaluation metadata set can be loaded synchronously, so as to realize the synchronous evaluation of the target broadcasting exercise video, and effectively improve the efficiency of the broadcasting exercise video evaluation; according to the playback progress sequence of the target broadcasting exercise video, at least one video acquisition frame corresponding to the broadcasting exercise video of the user to be evaluated per second is collected, and the video frames of the broadcasting exercise video are collected in real time to realize the recognition of each second of the broadcasting exercise video, and effectively improve the accuracy of the evaluation; for the broadcasting exercise video per second, the evaluation metadata set is used to identify the broadcasting exercise video of the user to be evaluated, so as to effectively improve the accuracy of the evaluation; for the broadcasting exercise video per second, the evaluation metadata set is used to identify the broadcasting exercise video of the user to be evaluated, and the evaluation metadata set is used to identify the broadcasting exercise video of the user to be evaluated, so as to effectively improve the accuracy of the evaluation. The data set is used to score the score according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame, and the score of the radio gymnastics per second corresponding to the radio gymnastics video per second is determined. By scoring the score of each second of the radio gymnastics video, the accurate scoring of the radio gymnastics video can be achieved, which can improve the accuracy of the evaluation; the score of each second of the radio gymnastics of the target radio gymnastics video is summarized to obtain the total score of the video, and the score evaluation is performed according to the total score of the video and the total score of the radio gymnastics corresponding to the target radio gymnastics video, and the radio gymnastics score of the user to be evaluated is determined. By summarizing the score of each second of the radio gymnastics video, the score evaluation of the entire video is completed, which solves the technical problem that the radio gymnastics video cannot be automatically evaluated in the prior art. It can automatically monitor and evaluate the accuracy and completion of the user's radio gymnastics movements, and improve the efficiency and quality of radio gymnastics teaching.

[0058] Figure 2 This is a flowchart of another method for evaluating the performance of broadcasting exercises provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is a specific method for scoring the performance of broadcasting exercises video per second. Figure 2As shown, the method includes:

[0059] S210 . In response to a broadcasting gymnastics evaluation request from a user to be evaluated, a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request are loaded.

[0060] S220, sequentially collecting at least one video collection frame corresponding to the broadcasting exercise video of the user to be evaluated per second according to the playing progress of the target broadcasting exercise video.

[0061] S230. For each second of the broadcast exercise video, the millisecond offset value and the skeleton point coordinate set of the video acquisition frame are matched with the millisecond offset value and the skeleton point coordinate set of each metadata video frame in turn. If there is a video acquisition frame and a metadata video frame that have a successful similarity match, the video acquisition frame is determined as the target scoring frame, and the metadata video frame is determined as the similar video frame.

[0062] The target scoring frame may be a video acquisition frame used to score the performance of the broadcasting exercise video per second. It should be noted that when scoring the broadcasting exercise video per second, a reference frame for scoring a video acquisition frame of the broadcasting exercise video per second is selected, i.e., the target scoring frame, which is a video acquisition frame among all video acquisition frames that matches the metadata video frame in similarity.

[0063] The similar video frame may be a metadata video frame in the evaluation metadata set that matches the similarity of a video acquisition frame in the broadcast operation video per second.

[0064] Optionally, in the implementation of the present invention, for multiple video capture frames captured per second of the broadcast exercise video, each time a video capture frame is captured, the millisecond offset value and the skeleton point coordinate set of the video capture frame are similarly matched with the millisecond offset value and the skeleton point coordinate set of each video capture frame in the evaluation metadata set. If the video capture frame successfully matches the similarity with a metadata video frame, other video capture frames corresponding to the broadcast exercise video per second will no longer be similarly matched, and the successfully matched video capture frame will be directly used as the target scoring frame, and the metadata video frame will be determined as a similar video frame.

[0065] Optionally, if the video capture frame cannot be successfully matched with any metadata video frame, it is considered that the video capture frame cannot be matched. Then, according to the capture order of the broadcast gymnastics video per second, the millisecond offset value and skeleton point coordinate set of the next video capture frame are selected to perform similarity matching with the millisecond offset value and skeleton point coordinate set of each video capture frame in the evaluation metadata set until the target scoring frame and similar video frame corresponding to the broadcast gymnastics video per second are found.

[0066] Optionally, if all video acquisition frames corresponding to the broadcasting exercise video per second cannot be successfully matched with any metadata video frame, then the broadcasting exercise video per second does not have a target scoring frame and a similar video frame, and the score of the broadcasting exercise video per second is set to 0.

[0067] S240, scoring is performed according to the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the score of the broadcasting exercise per second.

[0068] Optionally, for the radio gymnastics video per second, after obtaining the target scoring frame of the radio gymnastics per second, the radio gymnastics video per second is scored using the target scoring frame, and the score is scored using the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to obtain the radio gymnastics score per second corresponding to the radio gymnastics score per second.

[0069] Optionally, in another optional embodiment of the present invention, the step of scoring the performance according to the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the score of the broadcasting exercise per second includes:

[0070] Acquire a second millisecond offset range corresponding to the similar video frame; if the millisecond offset value of the target scoring frame is not within the second millisecond offset range, evaluate the target scoring frame as the first score;

[0071] If the millisecond offset value of the target scoring frame is within the second millisecond offset range, the target scoring frame is evaluated as a second score value.

[0072] Among them, the second millisecond offset range can be a millisecond offset range for assigning scores to the target scoring frame. It should be noted that the second millisecond offset range is used to distinguish whether the target scoring frame and the similar video frame are perfect hits and ordinary hits. If the target scoring frame is an ordinary hit, the score is the first score. If the target scoring frame is a perfect hit, the score is the second score. In an embodiment of the present invention, the second score is greater than the first score, and the second score is twice the first score. Exemplarily, the first score can be set to 1 point and the second score is set to 2 points.

[0073] Optionally, in an embodiment of the present invention, a second millisecond offset range is set based on the millisecond offset of the similar video frame, and the second millisecond offset range is selected from 50 milliseconds before the millisecond offset of the similar video frame to 100 milliseconds after the millisecond offset of the similar video frame. Exemplarily, the millisecond offset of the similar video frame is represented by k1.offset, and the second millisecond offset range is set to: [k1.offset - 50, k1.offset + 100].

[0074] Optionally, in an embodiment of the present invention, a second millisecond offset range of a similar video frame is obtained based on the millisecond offset of the similar video frame, and the relationship between the target scoring frame and the second millisecond offset range is identified; if it is identified that the millisecond offset value of the target scoring frame is not in the second millisecond offset range, the target scoring frame is considered to be a normal hit, and the score of the target scoring frame is evaluated as the first score; if it is identified that the millisecond offset value of the target scoring frame is in the second millisecond offset range, the target scoring frame is considered to be a perfect hit, and the score of the target scoring frame is evaluated as the second score.

[0075] S250, summarizing the radio gymnastics scores of each second of the target radio gymnastics video to obtain a total video score, performing a performance evaluation based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video, and determining the radio gymnastics performance of the user to be evaluated.

[0076] Optionally, in the embodiment of the present invention, the step of performing a performance evaluation based on the total score of the video and the total score of the radio gymnastics corresponding to the target radio gymnastics video to determine the radio gymnastics performance of the user to be evaluated includes:

[0077] Calculate the score based on the total score of the video and the total score of the broadcasting exercise to determine the broadcasting exercise score of the user to be evaluated;

[0078] If the broadcasting gymnastics score is not less than the preset passing score for the broadcasting gymnastics, the broadcasting gymnastics score of the user to be evaluated is considered valid;

[0079] If the broadcasting gymnastics score is less than the preset broadcasting gymnastics passing score, the broadcasting gymnastics score of the user to be evaluated is considered invalid.

[0080] Among them, the broadcasting gymnastics score can be the percentage score of the total video score relative to the total broadcasting gymnastics score under the percentage scoring rule.

[0081] Optionally, in an embodiment of the present invention, the server sets a total score for each type of standard radio gymnastics video. When evaluating the performance, the server is requested to obtain the total score of the radio gymnastics corresponding to the target radio gymnastics video, and the radio gymnastics score of each second of the target radio gymnastics video is summarized to obtain the total score of the video. Under the percentage scoring rule, a percentage calculation is performed based on the total video score and the total radio gymnastics score to convert the total video score into a radio gymnastics score. Exemplarily, the total score of the radio gymnastics of the target radio gymnastics video is set to total_score in the background of the server. For the similarity matching of the radio gymnastics videos per second, the radio gymnastics videos per second that do not have the target scoring frame are set to not be scored. For the radio gymnastics videos per second that have the target scoring frame, the first score of the ordinary hit is set to single_score, and the second score of the perfect hit is set to perfect_score, perfect_score = single_score 2. Sum up the scores of all the radio gymnastics videos per second to get the total score of the video as sum_score. Under the percentage scoring rule, set the radio gymnastics score as percent_score. The specific calculation process is: floor(sum_score / total_score 100); where floor means rounding down.

[0082] Among them, the preset passing score of the broadcasting exercise can be a passing score pre-set for evaluating the broadcasting exercise performance of the user to be evaluated. It should be noted that when the broadcasting exercise score of the user to be evaluated is greater than or equal to the passing score of the broadcasting exercise, the broadcasting exercise performance of the user to be evaluated in the target broadcasting exercise video is considered to be valid; when the broadcasting exercise score of the user to be evaluated is less than the passing score of the broadcasting exercise, the broadcasting exercise performance of the user to be evaluated in the target broadcasting exercise video is considered to be invalid. Exemplarily, the passing score of the broadcasting exercise is set to 60 points, the score of the user to be evaluated for completing the score evaluation of the target broadcasting exercise video is 59 points, the performance of the user to be evaluated in the target broadcasting exercise video is displayed as invalid, and the user's performance is displayed as 59 points; if the score of the user to be evaluated for completing the score evaluation of the target broadcasting exercise video is 69 points, the performance of the user to be evaluated in the target broadcasting exercise video is displayed as a valid performance, and the user's performance is displayed as 69 points.

[0083] Optionally, in an embodiment of the present invention, the radio gymnastics score of each second of the target radio gymnastics video is summarized to obtain the total score of the video, and the score is calculated based on the total video score and the total radio gymnastics score to determine the radio gymnastics score of the user to be evaluated. If the radio gymnastics score is not less than the preset radio gymnastics passing score, the radio gymnastics score of the user to be evaluated is considered to be valid; if the radio gymnastics score is less than the preset radio gymnastics passing score, the radio gymnastics score of the user to be evaluated is considered to be invalid.

[0084] In the embodiment of the present invention, for each second of the broadcast gymnastics video, similarity matching is performed through the millisecond offset value of the video acquisition frame and the skeleton point coordinate set and the evaluation metadata set, and the target scoring frame of each second of the broadcast gymnastics video is determined, so that each second of the broadcast gymnastics video can be scored separately, and the efficiency of the score scoring is effectively improved; by classifying and scoring the target scoring frame, the accuracy of the scoring is further improved, and by aggregating the scores of each second of the broadcast gymnastics video, the score evaluation of the entire video is completed, and the quality and efficiency of the score evaluation of the broadcast gymnastics video are further improved.

[0085] Figure 3 This is a flowchart of another method for evaluating the performance of broadcasting exercises provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is a specific method for matching the target scoring frames of broadcasting exercises video per second. Figure 3 As shown, the method includes:

[0086] S310: In response to a broadcasting gymnastics evaluation request from a user to be evaluated, a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request are loaded.

[0087] S320, sequentially collecting at least one video collection frame corresponding to the broadcasting exercise video of the user to be evaluated per second according to the playing progress of the target broadcasting exercise video.

[0088] S330. For the broadcast operation video per second, the millisecond offset value of each video acquisition frame is matched with the millisecond offset value of each metadata video frame in turn. If there is a millisecond offset value of the video acquisition frame that successfully matches the millisecond offset value of the metadata video frame, the metadata video frame is determined as the matching video frame corresponding to the video acquisition frame.

[0089] The matching video frame may be a metadata video frame whose millisecond offset matches that of the video acquisition frame in the evaluation metadata set.

[0090] Optionally, in an embodiment of the present invention, for multiple video capture frames captured by the broadcast exercise video per second, each time a video capture frame is captured, the millisecond offset value of the video capture frame is similarly matched with the millisecond offset value of each video capture frame in the evaluation metadata set. If the video capture frame successfully matches the millisecond offset value of a metadata video frame, the millisecond offset value matching will no longer be performed for other video capture frames corresponding to the broadcast exercise video per second, and the successfully matched metadata video frame will be directly determined as the matching video frame.

[0091] Optionally, in the embodiment of the present invention, after the first matching video frame of the broadcast exercise video per second is identified, the matching of other video acquisition frames with the evaluation metadata set may be stopped directly.

[0092] Optionally, if the video capture frame cannot be successfully matched with any metadata video frame, it is considered that the video capture frame cannot be matched. Then, according to the capture order of the broadcast exercise video per second, the millisecond offset value of the next video capture frame is selected to match the millisecond offset value of each video capture frame in the evaluation metadata set until a matching video frame corresponding to the broadcast exercise video per second is found.

[0093] Optionally, if each video capture frame of the broadcasting exercise video capture per second cannot be successfully matched with any metadata video frame, it is considered that the broadcasting exercise video per second cannot be successfully matched with the evaluation metadata set, and the score of the broadcasting exercise video per second can be directly set to 0.

[0094] Specifically, for each second of broadcast exercise video, the millisecond offset value of each video acquisition frame is matched with the millisecond offset value of each metadata video frame in turn. If there is a millisecond offset value of a video acquisition frame that successfully matches the millisecond offset value of a metadata video frame, the metadata video frame is determined as the matching video frame corresponding to the video acquisition frame.

[0095] Optionally, in another optional embodiment of the present invention, sequentially matching the millisecond offset value of each of the video acquisition frames with the millisecond offset value of each of the metadata video frames, and if there is a millisecond offset value of the video acquisition frame that successfully matches the millisecond offset value of the metadata video frame, then determining the metadata video frame as the matching video frame corresponding to the video acquisition frame includes:

[0096] For each of the metadata video frames, determining a first millisecond offset range of the metadata video frame based on the millisecond offset value of the metadata video frame;

[0097] The millisecond offset value of each of the video acquisition frames is identified with the first millisecond offset range of each of the metadata video frames in turn. If there is a millisecond offset value of the video acquisition frame that is within the first millisecond offset range of a metadata video frame, it is considered that the millisecond offset value of the video acquisition frame successfully matches the millisecond offset value of the metadata video frame, and the metadata video frame is determined as the matching video frame.

[0098] Among them, the first millisecond offset range can be a millisecond offset range for matching the metadata video frame with the video acquisition frame. It should be noted that the first millisecond offset range is used to determine whether the video acquisition frame matches the metadata video frame successfully. The first millisecond offset range is set based on the millisecond offset of the metadata video frame, and the minimum millisecond value and the maximum millisecond value of the millisecond offset of the metadata video frame are selected. For each metadata video frame, the millisecond range between the minimum millisecond value and the maximum millisecond value of the metadata video frame is used as the first millisecond offset range. Exemplarily, the minimum millisecond value and the maximum millisecond value of the millisecond offset of the metadata video frame are represented by minOffset and maxOffset, respectively, and k1.offset is set to represent the millisecond offset value of the metadata video frame, and minOffset=k1.offset-200, maxOffset=k1.offset+600 are set. The first 200 milliseconds of the metadata video frame to 600 milliseconds after the millisecond offset of the metadata video frame are used as the first millisecond offset range, and the first millisecond offset range is set to: [k1.offset-200, k1.offset+600].

[0099] Optionally, for the broadcast exercise video per second, the millisecond offset value of each video acquisition frame of the broadcast exercise video per second is compared with the first millisecond offset range of each metadata video frame in turn, and the comparison process is as follows: for a video acquisition frame, if the millisecond offset value of the video acquisition frame is not within the first millisecond offset range of a metadata video frame, then compare it with the first millisecond offset range of the next metadata video frame; if the millisecond offset value of the video acquisition frame is within the first millisecond offset range of the metadata video frame, then take the metadata video frame as the matching video frame, and no longer compare other video acquisition frames with the evaluation metadata set; if the millisecond offset value of the video acquisition frame is not within the first millisecond offset range of any metadata video frame, then select the next video acquisition frame according to the acquisition order of the video acquisition frames to repeat the comparison process until the matching video frame corresponding to the broadcast exercise video per second is found.

[0100] Specifically, for each second of broadcast exercise video, the millisecond offset value of each video acquisition frame is identified with the first millisecond offset range of each metadata video frame in turn. If there is a video acquisition frame whose millisecond offset value is within the first millisecond offset range of a metadata video frame, it is considered that the millisecond offset value of the video acquisition frame successfully matches the millisecond offset value of a metadata video frame, and the metadata video frame is determined to be a matching video frame.

[0101] S340: Perform similarity calculation based on the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score.

[0102] Among them, the similarity score can be the similarity score of the radio gymnastics action in the matching video frame and the radio gymnastics action in the video acquisition frame corresponding to the matching video frame.

[0103] Optionally, for the broadcasting gymnastics movements that match the video frame and the video acquisition frame, the similarity of the broadcasting gymnastics movements in the video frame can be directly evaluated by image comparison, and a similarity score can be given.

[0104] Optionally, in an embodiment of the present invention, similarity can be calculated by matching a video frame and a video acquisition frame through a set of bone point coordinates. The specific similarity calculation process is as follows: based on the same coordinate system constructed by matching the video frame and the video acquisition frame with the video image, the similarity of the bone point coordinates of each bone part in the matching video frame and the video acquisition frame is calculated to obtain a similarity score.

[0105] Optionally, in an embodiment of the present invention, the coordinates of the bone points of each bone part in the matching video frame and the video acquisition frame can be compared one by one, and the distance calculation can be performed on the coordinates of the bone points of each bone part to obtain a distance calculation result. For the distance calculation result, a distance scoring table is set, and the similarity score of each bone point coordinate is determined based on the distance calculation result through the distance scoring table, and finally the similarity scores of all the bone point coordinates are summarized to obtain a similarity score; similarly, the distance calculation results of the bone point coordinates of each bone part can be summarized, and a similarity score can be performed based on the summarized distance calculation results through the distance scoring table to obtain a similarity score.

[0106] Specifically, a similarity calculation is performed based on the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score.

[0107] Optionally, in another optional embodiment of the present invention, the similarity calculation is performed based on the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score, including:

[0108] The skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame are normalized to determine the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame; similarity is calculated based on the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame by using a weighted Euclidean distance formula to determine the similarity score.

[0109] Among them, the skeleton comparison coordinate set of the video acquisition frame can be a data coordinate set obtained by normalizing the coordinates of each skeleton point in the skeleton point coordinate set of the video acquisition frame; the skeleton comparison coordinate set of the matching video frame can be a data coordinate set obtained by normalizing the coordinates of each skeleton point in the skeleton point coordinate set of the matching video frame.

[0110] Optionally, for each broadcast gymnastics video per second, separate data normalization is performed based on the matching video frame of the broadcast gymnastics video per second and the video acquisition frame corresponding to the matching video frame. The specific process is as follows: identify the maximum value of the horizontal coordinate, the minimum value of the horizontal coordinate, the maximum value of the vertical coordinate and the minimum value of the vertical coordinate in all the bone point coordinates in the matching video frame and the video acquisition frame, and perform normalization calculation on each bone point coordinate in the matching video frame and the video acquisition frame, that is, the horizontal coordinate of each bone point coordinate is divided by the difference between the maximum value of the horizontal coordinate and the minimum value of the horizontal coordinate, and the vertical coordinate is divided by the difference between the maximum value of the vertical coordinate and the minimum value of the vertical coordinate, to obtain the normalized bone point coordinates. Then, the normalized bone point coordinates corresponding to the video acquisition frame form the bone comparison coordinate set of the video acquisition frame, and the normalized bone point coordinates corresponding to the matching video frame form the bone comparison coordinate set of the matching video frame.

[0111] Exemplarily, k1 represents the matching video frame, k2 represents the video acquisition frame corresponding to the matching video frame, the coordinates of each bone point in k1 and k2 are expressed as (horizontal coordinate x, vertical coordinate y, coordinate accuracy score), and the maximum horizontal coordinate, minimum horizontal coordinate, maximum vertical coordinate and minimum vertical coordinate in k1 and k2 are expressed as: maxX, minX, maxY and minY respectively; the normalized calculation of each bone point coordinate can be expressed as: horizontal coordinate x: (x - minX) / (maxX - minX), vertical coordinate y: (y - minY) / (maxY - minY), and the skeleton normalized comparison coordinate set k2c of the video acquisition frame is obtained, which matches the skeleton normalized comparison coordinate set k1c of the video frame. Since the coordinate accuracy score of each bone point coordinate is not involved in the calculation, assuming that k1 and k2 only include the head and neck, the bone point coordinates of k1 and k2 are expressed as k1 head (100,100), k1 neck (200, 250), k2 head (101, 102), k2 neck (201, 251), then minX = min(100,200, 101, 201) = 100, maxX = max(100, 200, 101, 201) = 201, minY = min(100,250, 102, 251) = 100, maxY=max(100, 250, 102, 251) = 251. Then the normalized coordinates of k1's head are calculated to get the coordinates of k1c's head, which are ((100 - 100) / (201 - 100), (100 - 100) / (251 -100)) = (0, 0), and the normalized coordinates of k2's head are calculated to get the coordinates of k2c's head, which are ((101 -100) / (201 - 100), (102 - 100) / (251 - 100)) = (0.009, 0.013). Similarly, the coordinates of k1c's neck and k2c's neck can be calculated. Based on the above calculation process, when the bone points of k1 and k2 are head, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle, the coordinates of k1c and k2c of each part can be calculated by the same calculation method.

[0112] Optionally, a weighted Euclidean distance formula can be used to evaluate the similarity between the matching video frame and the video acquisition frame; the weighted Euclidean distance formula is used to perform similarity calculation based on the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame to obtain a similarity score, and the specific calculation process is as follows:

[0113] Since each bone point coordinate of the matching video frame k1 will have a coordinate accuracy score, for example, the bone point coordinate of k1's head is expressed as (head horizontal coordinate x, head vertical coordinate y, coordinate accuracy score), the bone point coordinate of k1's neck is expressed as (neck horizontal coordinate x, neck vertical coordinate y, coordinate accuracy score), and so on, the bone point coordinates of k1's left shoulder, k1's right shoulder, k1's left elbow, k1's right elbow, k1's left wrist, k1's right wrist, k1's left hip, k1's right hip, k1's left knee, k1's right knee, k1's left ankle, and k1's right ankle also have corresponding coordinate accuracy scores; the coordinate accuracy score of each bone point of k1 is summed to obtain the first coordinate score k1_score, and then the weighted Euclidean score of k2 and k1 is calculated, and the weighted Euclidean score of k2 and k1 is represented by weight_score, and the weighted Euclidean score of k2 and k1 is represented by represents the coordinate accuracy score of the i-th bone point in the matching video frame k1, where i is used to represent the i-th bone point in the order of k1 head, k1 neck, k1 left shoulder, k1 right shoulder, k1 left elbow, k1 right elbow, k1 left wrist, k1 right wrist, k1 left hip, k1 right hip, k1 left knee, k1 right knee, k1 left ankle and k1 right ankle; The x-axis coordinate of the i-th bone point in the same arrangement order as K1 is used to represent the normalized comparison coordinate set k1c of the matched video frame; The y-axis coordinate of the i-th bone point in the same arrangement order as K1 is used to represent the normalized comparison coordinate set k1c of the matched video frame; The x-axis coordinate of the ith bone point in the same arrangement order as K1 in the skeleton normalized comparison coordinate set k2c used to represent the video acquisition frame; The normalized comparison coordinate set k2c of the skeleton of the matching video frame is used to represent the y-axis coordinate of the i-th skeleton point in the same arrangement order as K1. The calculation formula of the weighted Euclidean score of k2 and k1 is as follows:

[0114]

[0115] Then calculate the similarity score, and use resultSim to represent the similarity score. The similarity score is calculated as follows: resultSim = weight_score / k1_score.

[0116] S350: If the similarity score is less than the similarity confidence of the matching video frame, it is considered that there is a successful similarity match between the video acquisition frame and the metadata video frame, and the matching video frame is determined as the similar video frame, and the video acquisition frame is determined as the target scoring frame.

[0117] Optionally, obtain the similarity confidence of the matching video frame, and judge the similarity confidence of the matching video frame and the similarity score. If the similarity score is less than the similarity confidence of the matching video frame, it is considered that the similarity between a video capture frame and a metadata video frame is successfully matched. Determine the matching video frame as the similar video frame and the video capture frame as the target scoring frame. Exemplarily, the similarity confidence of the matching video frame can be expressed as k1_sim. If the similarity score resultSim is less than the similarity confidence k1_sim, that is, resultSim < k1_sim, it is considered that k2 and k1 are successfully matched.

[0118] Optionally, if the similarity score is not less than the similarity confidence of the matching video frame, it is considered that the similarity between a video capture frame and a metadata video frame fails to match, and the score of the broadcast exercise per second of the video is directly set to 0.

[0119] S360. Perform a score evaluation based on the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the score of the broadcast exercise per second.

[0120] S370. Aggregate each score of the broadcast exercise per second of the target broadcast exercise video to obtain the total video score. Evaluate the performance based on the total video score and the total score of the broadcast exercise corresponding to the target broadcast exercise video to determine the broadcast exercise score of the user to be evaluated.

[0121] In the embodiments of the present invention, for the broadcast exercise video per second, a matching video frame is found in the evaluation metadata set through the millisecond offset value of the video capture frame. After determining the matching video frame, the matching process of the matching video frame is stopped, which can effectively improve the evaluation efficiency. After determining the matching video frame, the matching video frame and the video capture frame are normalized based on the skeletal point coordinate set, and similarity matching is performed to determine the target scoring frame of each broadcast exercise video per second, realizing a separate and accurate scoring of each second of the broadcast exercise video, effectively improving the accuracy of the score evaluation, and further improving the quality and efficiency of the performance evaluation of the broadcast exercise video.

[0122] Figure 4 It is a schematic structural diagram of a broadcast exercise performance evaluation device provided by an embodiment of the present invention. As Figure 4 shown, the device includes: a loading module 410, a collection module 420, a scoring module 430, and an evaluation module 440; wherein,

[0123] The loading module 410 is configured to load the target broadcast exercise video and the evaluation metadata set corresponding to the broadcast exercise evaluation request in response to the broadcast exercise evaluation request of the user to be evaluated;

[0124] The acquisition module 420 is used to acquire at least one video acquisition frame corresponding to each second of the broadcasting exercise video of the user to be evaluated according to the playback progress sequence of the target broadcasting exercise video; wherein the video acquisition frame includes a skeleton point coordinate set and a millisecond offset value;

[0125] Scoring module 430, for scoring the per-second broadcasting gymnastics video by using the evaluation metadata set according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame, and determining the per-second broadcasting gymnastics score corresponding to the per-second broadcasting gymnastics video;

[0126] The evaluation module 440 is used to summarize the radio gymnastics scores of each second of the target radio gymnastics video to obtain the total score of the video, and perform performance evaluation based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated.

[0127] The technical solution of the embodiment of the present invention, by responding to the broadcasting exercise evaluation request of the user to be evaluated, loading the target broadcasting exercise video and the evaluation metadata set corresponding to the broadcasting exercise evaluation request, when the user selects the target broadcasting exercise video for evaluation, the target broadcasting exercise video and the evaluation metadata set can be loaded synchronously, so as to realize the synchronous evaluation of the target broadcasting exercise video, and effectively improve the efficiency of the broadcasting exercise video evaluation; according to the playback progress sequence of the target broadcasting exercise video, at least one video acquisition frame corresponding to the broadcasting exercise video of the user to be evaluated per second is collected, and the video frames of the broadcasting exercise video are collected in real time to realize the recognition of each second of the broadcasting exercise video, and effectively improve the accuracy of the evaluation; for the broadcasting exercise video per second, the evaluation metadata set is used to identify the broadcasting exercise video of the user to be evaluated, so as to effectively improve the accuracy of the evaluation; for the broadcasting exercise video per second, the evaluation metadata set is used to identify the broadcasting exercise video of the user to be evaluated, and the evaluation metadata set is used to identify the broadcasting exercise video of the user to be evaluated, so as to effectively improve the accuracy of the evaluation. The data set is used to score the score according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame, and the score of the radio gymnastics per second corresponding to the radio gymnastics video per second is determined. By scoring the score of each second of the radio gymnastics video, the accurate scoring of the radio gymnastics video can be achieved, which can improve the accuracy of the evaluation; the score of each second of the radio gymnastics of the target radio gymnastics video is summarized to obtain the total score of the video, and the score evaluation is performed according to the total score of the video and the total score of the radio gymnastics corresponding to the target radio gymnastics video, and the radio gymnastics score of the user to be evaluated is determined. By summarizing the score of each second of the radio gymnastics video, the score evaluation of the entire video is completed, which solves the technical problem that the radio gymnastics video cannot be automatically evaluated in the prior art. It can automatically monitor and evaluate the accuracy and completion of the user's radio gymnastics movements, and improve the efficiency and quality of radio gymnastics teaching.

[0128] Optionally, the evaluation metadata set includes at least one metadata video frame; the metadata video frame includes the skeleton point coordinate set, the millisecond offset value and the similarity confidence;

[0129] Optionally, the scoring module 430 is specifically used for:

[0130] Sequentially performing similarity matching on the millisecond offset value and the skeleton point coordinate set of the video acquisition frame and the millisecond offset value and the skeleton point coordinate set of each metadata video frame; if there is a video acquisition frame and a metadata video frame that successfully match each other in similarity, the video acquisition frame is determined as a target scoring frame, and the metadata video frame is determined as a similar video frame;

[0131] The score is scored according to the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the score of the broadcasting exercise per second.

[0132] Optionally, the scoring module 430 is further configured to:

[0133] Matching the millisecond offset value of each video acquisition frame with the millisecond offset value of each metadata video frame in sequence, and if there is a millisecond offset value of the video acquisition frame that successfully matches the millisecond offset value of the metadata video frame, determining the metadata video frame as the matching video frame corresponding to the video acquisition frame;

[0134] Calculate similarity between the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score;

[0135] If the similarity score is less than the similarity confidence of the matching video frame, it is considered that there is a successful similarity match between the video acquisition frame and the metadata video frame, the matching video frame is determined as the similar video frame, and the video acquisition frame is determined as the target scoring frame.

[0136] Optionally, the scoring module 430 is further configured to:

[0137] For each of the metadata video frames, determining a first millisecond offset range of the metadata video frame based on the millisecond offset value of the metadata video frame;

[0138] The millisecond offset value of each of the video acquisition frames is identified with the first millisecond offset range of each of the metadata video frames in turn. If there is a millisecond offset value of the video acquisition frame that is within the first millisecond offset range of a metadata video frame, it is considered that the millisecond offset value of the video acquisition frame successfully matches the millisecond offset value of the metadata video frame, and the metadata video frame is determined as the matching video frame.

[0139] Optionally, the skeleton point coordinate set includes at least one skeleton point coordinate;

[0140] Optionally, the scoring module 430 is further configured to:

[0141] Normalize the coordinates of each of the skeleton points in the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to determine the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame;

[0142] The similarity score is determined by performing similarity calculation based on the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame using a weighted Euclidean distance formula.

[0143] Optionally, the scoring module 430 is further configured to:

[0144] Obtaining a second millisecond offset range corresponding to the similar video frame;

[0145] If the millisecond offset value of the target scoring frame is not within the second millisecond offset range, evaluating the target scoring frame as a first score;

[0146] If the millisecond offset value of the target scoring frame is within the second millisecond offset range, the target scoring frame is evaluated as a second score value.

[0147] Optionally, the evaluation module 440 is specifically used for:

[0148] Calculate the score based on the total score of the video and the total score of the broadcasting exercise to determine the broadcasting exercise score of the user to be evaluated;

[0149] If the broadcasting gymnastics score is not less than the preset passing score for the broadcasting gymnastics, the broadcasting gymnastics score of the user to be evaluated is considered valid;

[0150] If the broadcasting gymnastics score is less than the preset broadcasting gymnastics passing score, the broadcasting gymnastics score of the user to be evaluated is considered invalid.

[0151] The broadcasting gymnastics performance evaluation device provided in the embodiment of the present invention can execute the broadcasting gymnastics performance evaluation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0152] Figure 5A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their modes are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0153] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11 in communication, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the random access memory (RAM) 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the read-only memory (ROM) 12, and the random access memory (RAM) 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0154] A number of components in the electronic device 10 are connected to an input / output (I / O) interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0155] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as the broadcast gymnastics performance evaluation method.

[0156] In some embodiments, the broadcasting gymnastics performance evaluation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via a read-only memory (ROM) 12 and / or a communication unit 19. When the computer program is loaded into a random access memory (RAM) 13 and executed by the processor 11, one or more steps of the broadcasting gymnastics performance evaluation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the broadcasting gymnastics performance evaluation method in any other appropriate manner (e.g., by means of firmware).

[0157] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0158] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the modes / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0159] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0160] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0161] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0162] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0163] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0164] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the method for evaluating the performance of broadcasting exercises provided in any embodiment of the present invention are implemented. The method includes:

[0165] In response to a broadcasting gymnastics evaluation request from a user to be evaluated, loading a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request;

[0166] At least one video acquisition frame corresponding to each second of the broadcasting exercise video of the user to be evaluated is collected in sequence according to the playback progress of the target broadcasting exercise video; wherein the video acquisition frame includes a skeleton point coordinate set and a millisecond offset value;

[0167] For the per-second broadcasting gymnastics video, score the per-second broadcasting gymnastics video according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame through the evaluation metadata set, and determine the per-second broadcasting gymnastics score corresponding to the per-second broadcasting gymnastics video;

[0168] The radio gymnastics scores of each second of the target radio gymnastics video are summarized to obtain the total score of the video, and a performance evaluation is performed based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated.

[0169] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0170] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0171] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0172] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0173] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0174] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0175] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the performance of broadcasting exercises, characterized in that: include: In response to a broadcasting gymnastics evaluation request from a user to be evaluated, loading a target broadcasting gymnastics video and an evaluation metadata set corresponding to the broadcasting gymnastics evaluation request; At least one video acquisition frame corresponding to each second of the broadcasting exercise video of the user to be evaluated is collected in sequence according to the playback progress of the target broadcasting exercise video; wherein the video acquisition frame includes a skeleton point coordinate set and a millisecond offset value; For the per-second broadcasting gymnastics video, score the per-second broadcasting gymnastics video according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame through the evaluation metadata set, and determine the per-second broadcasting gymnastics score corresponding to the per-second broadcasting gymnastics video; The radio gymnastics scores of each second of the target radio gymnastics video are summarized to obtain the total score of the video, and a performance evaluation is performed based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated.

2. The method according to claim 1, characterized in that The evaluation metadata set includes at least one metadata video frame; the metadata video frame includes the skeleton point coordinate set, the millisecond offset value and the similarity confidence; The step of scoring the score according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame through the evaluation metadata set to determine the score of the broadcasting exercises per second corresponding to the broadcasting exercises per second video comprises: Sequentially performing similarity matching on the millisecond offset value and the skeleton point coordinate set of the video acquisition frame and the millisecond offset value and the skeleton point coordinate set of each metadata video frame; if there is a video acquisition frame and a metadata video frame that successfully match each other in similarity, the video acquisition frame is determined as a target scoring frame, and the metadata video frame is determined as a similar video frame; The score is scored according to the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the score of the broadcasting exercise per second.

3. The method according to claim 2, characterized in that The method sequentially performs similarity matching on the millisecond offset value and the skeleton point coordinate set of the video acquisition frame and the millisecond offset value and the skeleton point coordinate set of each metadata video frame, and if there is a video acquisition frame and a metadata video frame whose similarity matches successfully, then determines the video acquisition frame as a target scoring frame and determines the metadata video frame as a similar video frame, including: Matching the millisecond offset value of each video acquisition frame with the millisecond offset value of each metadata video frame in sequence, and if there is a millisecond offset value of the video acquisition frame that successfully matches the millisecond offset value of the metadata video frame, determining the metadata video frame as the matching video frame corresponding to the video acquisition frame; Calculate similarity between the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score; If the similarity score is less than the similarity confidence of the matching video frame, it is considered that there is a successful similarity match between the video acquisition frame and the metadata video frame, the matching video frame is determined as the similar video frame, and the video acquisition frame is determined as the target scoring frame.

4. The method according to claim 3, characterized in that: The step of sequentially matching the millisecond offset value of each video acquisition frame with the millisecond offset value of each metadata video frame, and if there is a millisecond offset value of the video acquisition frame that successfully matches the millisecond offset value of the metadata video frame, determining the metadata video frame as a matching video frame corresponding to the video acquisition frame, includes: For each of the metadata video frames, determining a first millisecond offset range of the metadata video frame based on the millisecond offset value of the metadata video frame; The millisecond offset value of each of the video acquisition frames is identified with the first millisecond offset range of each of the metadata video frames in turn. If there is a millisecond offset value of the video acquisition frame that is within the first millisecond offset range of a metadata video frame, it is considered that the millisecond offset value of the video acquisition frame successfully matches the millisecond offset value of the metadata video frame, and the metadata video frame is determined as the matching video frame.

5. The method according to claim 3, characterized in that: The skeleton point coordinate set includes at least one skeleton point coordinate; the similarity calculation is performed based on the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to obtain a similarity score, including: Normalize the coordinates of each of the skeleton points in the skeleton point coordinate set of the video acquisition frame and the skeleton point coordinate set of the matching video frame to determine the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame; The similarity score is determined by performing similarity calculation based on the skeleton comparison coordinate set of the video acquisition frame and the skeleton comparison coordinate set of the matching video frame using a weighted Euclidean distance formula.

6. The method according to claim 2, characterized in that The step of scoring the score according to the millisecond offset value of the target scoring frame and the millisecond offset value of the similar video frame to determine the score of the broadcasting exercise per second includes: Obtaining a second millisecond offset range corresponding to the similar video frame; If the millisecond offset value of the target scoring frame is not within the second millisecond offset range, evaluating the target scoring frame as a first score; If the millisecond offset value of the target scoring frame is within the second millisecond offset range, the target scoring frame is evaluated as a second score value.

7. The method according to claim 1, characterized in that The step of performing a performance evaluation based on the total score of the video and the total score of the radio gymnastics corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated includes: Calculate the score based on the total score of the video and the total score of the broadcasting exercise to determine the broadcasting exercise score of the user to be evaluated; If the broadcasting gymnastics score is not less than the preset passing score for the broadcasting gymnastics, the broadcasting gymnastics score of the user to be evaluated is considered valid; If the broadcasting gymnastics score is less than the preset broadcasting gymnastics passing score, the broadcasting gymnastics score of the user to be evaluated is considered invalid.

8. A radio gymnastics performance evaluation device, characterized in that: include: A loading module, configured to respond to a radio gymnastics evaluation request from a user to be evaluated and load a target radio gymnastics video and an evaluation metadata set corresponding to the radio gymnastics evaluation request; A collection module, used to collect at least one video collection frame corresponding to each second of the broadcasting exercise video of the user to be evaluated according to the playback progress sequence of the target broadcasting exercise video; wherein the video collection frame includes a skeleton point coordinate set and a millisecond offset value; A scoring module is used to score the per-second broadcasting exercise video according to the skeleton point coordinate set and the millisecond offset value of each video acquisition frame through the evaluation metadata set, and determine the per-second broadcasting exercise score corresponding to the per-second broadcasting exercise video; An evaluation module is used to summarize the radio gymnastics scores of each second of the target radio gymnastics video to obtain a total video score, and to perform a performance evaluation based on the total video score and the total radio gymnastics score corresponding to the target radio gymnastics video to determine the radio gymnastics score of the user to be evaluated.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the broadcast gymnastics performance evaluation method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the broadcast gymnastics performance evaluation method described in any one of claims 1-7 when executed.

Images