Video action scoring method, device and equipment and storage medium thereof

Abstract

The embodiment of the invention belongs to the technical field of image processing, is applied to an athlete skilled action scoring scene, and relates to a video action scoring method, device and equipment and a storage medium thereof. Carrying out first segmentation processing; carrying out segmentation processing on the first segmented video segment again to obtain a time-sequenced frame-level segmentation processing result; predicting different key actions corresponding to each frame of cutting processing result; counting relative proportions of all key actions in the target video; and based on the relative ratio and a preset scoring formula, scoring the target video to obtain a scoring result. When the video action scoring method is applied to a sports competition scoring scene, different key actions contained in a video can be recognized according to the video recorded in the competition process, fair scoring is carried out according to a statistical result of technical actions of athletes of both parties, coaches are assisted to carry out strategic analysis on the athletes, and the scoring efficiency of the athletes is improved. And thus, a related battle strategy is formulated.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and is applied to the scoring scenario of athletes' technical movements, and in particular to a video movement scoring method, device, equipment and storage medium thereof. Background Art

[0002] Accurately tracking the body's movements during an action is an important part of sports science. However, the current research focuses on placing markers at predetermined points on the subject's body, which can interfere with the subject's behavior and is only applicable to a limited number of sports. Motion capture has long been a key component of analyzing athlete performance.

[0003] Due to the high cost of motion capture, the current scoring method for sports competitions still relies heavily on manual scoring, which easily introduces personal judgment factors, which is not conducive to fair scoring and is not intelligent enough. Moreover, the analysis of athlete performance also relies on analysts to watch and mark repeatedly, which is time-consuming and labor-intensive, and is not intelligent or scientific enough. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a video action scoring method, device, equipment and storage medium thereof to solve the problems that the existing sports competition scoring method and athlete performance analysis method are not intelligent enough, are not conducive to fair scoring, and are time-consuming and labor-intensive.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a video action scoring method, which adopts the following technical solution:

[0006] A video action scoring method comprises the following steps:

[0007] Get the target video;

[0008] Inputting the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model;

[0009] Inputting the first segmented video clip into a preset second segmentation processing model to obtain a time-series frame-level segmentation processing result output by the second segmentation processing model;

[0010] Inputting the time-series frame-level cutting processing results into a pre-trained frame-level action prediction model, predicting different key actions corresponding to each frame-level cutting processing result, and obtaining a prediction result;

[0011] According to the prediction results, the relative proportions of all key actions in the target video are counted;

[0012] Based on the relative proportions of all the key actions in the target video and a preset scoring formula, the target video is scored to obtain a scoring result.

[0013] Furthermore, before executing the step of inputting the target video into the pre-trained first segmentation processing model to obtain the first segmentation video segment output by the first segmentation processing model, the method further includes:

[0014] Acquire a batch of motion videos of the same motion type as the target video, and annotate the video-level action labels respectively contained in each motion video in the batch of motion videos to obtain an annotation result;

[0015] Presetting all video-level action labels of the motion type and standard video clips corresponding to each video-level action label according to manual experience;

[0016] Deploy all video-level action labels in the motion type and the standard video clips corresponding to each video-level action label as segmentation reference samples into the first segmentation processing model to be trained, and obtain the first segmentation processing model to be trained in which the segmentation reference samples are deployed;

[0017] The batch motion videos are input into the first segmentation processing model to be trained in which the segmentation reference samples are deployed, segmentation processing training is performed on the first segmentation processing model, and the pre-trained first segmentation processing model is obtained.

[0018] Furthermore, the step of inputting the batch motion video into the first segmentation processing model to be trained in which the segmentation reference sample is deployed, performing segmentation processing training on the first segmentation processing model, and obtaining the pre-trained first segmentation processing model specifically includes:

[0019] Step 401, obtaining all video-level action labels output by the first segmentation processing model according to the batch motion video, and segmentation segments corresponding to each video-level action label, as pre-training output results;

[0020] Step 402, comparing the pre-training output result with the labeling result to obtain a comparison result;

[0021] Step 403, calculating the loss value of the pre-training output result compared to the labeling result according to the comparison result and a preset loss function;

[0022] Step 404: if the loss value does not meet the preset loss condition, adjust the hyperparameters of the first segmentation processing model, and re-execute steps 401 to 403;

[0023] Step 405: If the loss value satisfies a preset loss condition, the pre-training of the first segmentation processing model is completed, and a pre-trained first segmentation processing model is obtained.

[0024] Furthermore, before executing the step of inputting the first segmented video clips into a preset second segmentation processing model to obtain a time-series frame-level segmentation processing result output by the second segmentation processing model, the method further includes:

[0025] Get the frame window and frame step size preset in the target configuration file;

[0026] Deploy the framing window and the framing step size as frame-level segmentation parameters to a pre-built second segmentation processing model, and obtain a second segmentation processing model after the frame-level segmentation parameters are deployed as the preset second segmentation processing model;

[0027] Arranging all segmented segments output by the first segmentation processing model according to the batch motion video as input samples of the preset second segmentation processing model;

[0028] The input sample is input into the preset second segmentation processing model to obtain a time-series frame-level segmentation processing result corresponding to the input sample output by the second segmentation processing model.

[0029] Furthermore, the step of arranging all segmented segments output by the first segmentation processing model according to the batch motion video as input samples of the preset second segmentation processing model specifically includes:

[0030] According to the distinguishing identification information of the batch of motion videos, sorting out the segmented segments respectively contained in the same video from all the segmented segments;

[0031] Using image information extraction technology, extract the recording timestamp information of all the segmented segments contained in the same video to obtain the recording timestamp information;

[0032] All the segmented segments contained in the same video are time-sequentially sorted according to the recording timestamp information to obtain the time-sequentially sorted segmented segments;

[0033] The time-series sorted segmented segments are used as input samples of the preset second segmentation processing model.

[0034] Furthermore, before executing the step of inputting the time-series frame-level segmentation processing result into the pre-trained frame-level action prediction model, predicting different key actions corresponding to each frame segmentation processing result, and obtaining the prediction result, the method further includes:

[0035] Presetting all key actions of the motion type and example images corresponding to each key action according to manual experience;

[0036] Deploy all key actions in the motion type and the example images corresponding to each key action as prediction reference knowledge into the frame-level action prediction model to be trained, and obtain the frame-level action prediction model to be trained in which the prediction reference knowledge is deployed;

[0037] Arranging the time-series frame-level segmentation processing results output by the second segmentation processing model as input samples of the frame-level action prediction model to be trained;

[0038] Inputting the input sample into the frame-level motion prediction model to be trained, performing learning training on the frame-level motion prediction model to be trained, and obtaining a pre-trained frame-level motion prediction model;

[0039] The step of inputting the time-series frame-level cutting processing results into the pre-trained frame-level action prediction model, predicting different key actions corresponding to each frame cutting processing result, and obtaining the prediction results specifically includes:

[0040] According to all the key actions in the motion type and the example images corresponding to each key action, predict the probability values ​​of the current frame cutting processing result belonging to different key actions;

[0041] The probability values ​​of the current frame cutting processing result belonging to different key actions are compared in magnitude, and the key action corresponding to the maximum probability value is selected as the key action corresponding to the current frame cutting processing result;

[0042] Different key actions corresponding to each frame cutting processing result are obtained as the prediction result.

[0043] Furthermore, the step of counting the relative proportions of all key actions in the target video according to the prediction results specifically includes:

[0044] According to the prediction results, frame cutting processing results corresponding to all key actions are obtained;

[0045] Based on the frame cutting processing results corresponding to all the key actions, the number of cut images corresponding to all the key actions and the total number of cut images corresponding to all the key actions are counted;

[0046] According to the number of cut images corresponding to all key actions and the total number of cut images corresponding to all key actions, a ratio value is calculated to obtain the relative proportion of all key actions in the target video;

[0047] The step of scoring the target video based on the relative proportions of all the key actions in the target video and a preset scoring formula to obtain a scoring result specifically includes:

[0048] The preset scoring formula is used:

[0049]

[0050] Calculate the score result of the target video, where Score represents the score result of the target video, k represents the number of key action categories, and α k Indicates the relative proportion of the current key action, s k Indicates the score reference value corresponding to the current key action.

[0051] In order to solve the above technical problems, the embodiment of the present application also provides a video action scoring device, which adopts the following technical solution:

[0052] A video action scoring device, comprising:

[0053] A target video acquisition module is used to acquire a target video;

[0054] A first segmentation processing module is used to input the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model;

[0055] A second segmentation processing module, used for inputting the first segmented video clip into a preset second segmentation processing model, and obtaining a time-series frame-level segmentation processing result output by the second segmentation processing model;

[0056] A key action prediction module is used to input the time-series frame-level cutting processing results into a pre-trained frame-level action prediction model, predict different key actions corresponding to each frame cutting processing result, and obtain prediction results;

[0057] A relative proportion statistics module is used to count the relative proportions of all key actions in the target video according to the prediction results;

[0058] The target video scoring module is used to score the target video based on the relative proportions of all the key actions in the target video and a preset scoring formula to obtain a scoring result.

[0059] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0060] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned video action scoring method when executing the computer-readable instructions.

[0061] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0062] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the video action scoring method as described above.

[0063] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0064] The video action scoring method described in the embodiment of the present application is as follows: obtaining a target video; inputting the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model; inputting the first segmentation video segment into a preset second segmentation processing model to obtain a timed frame-level segmentation processing result output by the second segmentation processing model; inputting the timed frame-level segmentation processing result into a pre-trained frame-level action prediction model to predict different key actions corresponding to each frame segmentation processing result to obtain a prediction result; according to the prediction result, the relative proportion of all key actions in the target video is counted; based on the relative proportion of all key actions in the target video and a preset scoring formula, the target video is scored to obtain a scoring result. The scoring method described in the present application is applied to a sports game scoring scenario, and can identify different key actions contained in the video according to the video recorded during the game, and can be fairly scored according to the statistical results of the technical actions of the athletes on both sides, and can also assist coaches in strategically analyzing the target athletes, thereby formulating relevant battle strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0066] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0067] Figure 2 is a flow chart of an embodiment of a video action scoring method according to the present application;

[0068] Figure 3 yes Figure 2 A flowchart of a specific embodiment of performing segmentation processing training on the first segmentation processing model before step 202 is shown;

[0069] Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 304 is shown;

[0070] Figure 5 yes Figure 2 A flowchart of a specific embodiment of performing frame-level segmentation processing on all segmented segments output from the batch motion video before step 203 is shown;

[0071] Figure 6 yes Figure 5 A flowchart of a specific embodiment of step 503 is shown;

[0072] Figure 7 yes Figure 2 A flowchart of a specific embodiment of performing learning and training on the frame-level action prediction model to be trained before step 204 is shown;

[0073] Figure 8 is a structural schematic diagram of an embodiment of a video action scoring device according to the present application;

[0074] Fig. 9 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0075] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0076] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0077] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0078] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0079] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0080] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0081] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0082] It should be noted that the video action scoring method provided in the embodiment of the present application is generally executed by a terminal device, and accordingly, the video action scoring device is generally arranged in the terminal device.

[0083] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0084] Continue to refer Figure 2 , shows a flow chart of an embodiment of a video action scoring method according to the present application. The video action scoring method comprises the following steps:

[0085] Step 201, obtaining a target video;

[0086] Specifically, the target video can be understood as an athlete competition video, for example, a video recorded during a gymnastics competition. The video action scoring method is applied to the athlete competition scoring scenario, and the athlete can be scored based on the video recorded during the competition.

[0087] Step 202: input the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model;

[0088] Specifically, the first segmentation processing model can be understood as segmenting a number of main video clips containing technical movements from the target video, and deleting the more conventional (not containing technical movements) competition clips. For example, the video recorded during the above-mentioned gymnastics competition can be processed through the first segmentation processing model to first delete the competition clips that do not contain technical movements from the entire recorded video, and only retain the several video clips containing technical movements that are finally segmented and segmented. This processing method not only ensures that the athlete's performance can be scored from the perspective of key movements, but also reduces the consumption of computer processing resources.

[0089] Step 203, inputting the first segmented video clip into a preset second segmentation processing model to obtain a time-series frame-level segmentation processing result output by the second segmentation processing model;

[0090] Specifically, the second segmentation and processing model can be understood as performing frame-level processing on the main video clips containing technical movements obtained after processing by the first segmentation and processing model, and scoring the athlete's performance through the timed frame-level cutting processing results output by the second segmentation and processing model, which is more accurate than video-level scoring.

[0091] Step 204, inputting the time-series frame-level segmentation processing results into a pre-trained frame-level action prediction model, predicting different key actions corresponding to each frame segmentation processing result, and obtaining a prediction result;

[0092] The prediction results are obtained by predicting different key actions corresponding to each frame cutting processing result, so that different key actions contained in the target video and the number of times different key actions appear can be identified.

[0093] Step 205, according to the prediction result, the relative proportions of all key actions in the target video are counted;

[0094] By counting the relative proportions of all key actions in the target video, the technical actions used by the athletes on both sides can be analyzed, and fair scoring can be performed based on the statistical results of the technical actions. It can also assist coaches in conducting strategic analysis of the target athletes and thus formulate relevant combat strategies.

[0095] Step 206: Based on the relative proportions of all the key actions in the target video and a preset scoring formula, the target video is scored to obtain a scoring result.

[0096] In this embodiment, by obtaining a target video; inputting the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model; inputting the first segmentation video segment into a preset second segmentation processing model to obtain a timed frame-level segmentation processing result output by the second segmentation processing model; inputting the timed frame-level segmentation processing result into a pre-trained frame-level action prediction model to predict different key actions corresponding to each frame segmentation processing result to obtain a prediction result; according to the prediction result, the relative proportions of all key actions in the target video are counted; based on the relative proportions of all key actions in the target video and a preset scoring formula, the target video is scored to obtain a scoring result. The scoring method described in this application is applied to a sports game scoring scenario, and can identify different key actions contained in the video according to the video recorded during the game, and can be fairly scored according to the statistical results of the technical actions of the athletes on both sides, and can also assist coaches in strategically analyzing the target athletes, thereby formulating relevant combat strategies.

[0097] Continue to refer Figure 3 In some optional implementations, before step 202, a step of performing segmentation processing training on the first segmentation processing model is also included. Figure 3 yes Figure 2 The flowchart of a specific embodiment of performing segmentation processing training on the first segmentation processing model before step 202 includes the following steps:

[0098] Step 301, obtaining a batch of motion videos of the same motion type as the target video, and annotating the video-level action labels respectively contained in each motion video in the batch of motion videos to obtain an annotation result;

[0099] Specifically, the video-level action label can be understood as an action label set according to a video action.

[0100] Step 302, presetting all video-level action labels in the motion type and standard video clips corresponding to each video-level action label according to manual experience;

[0101] Step 303, deploying all video-level action labels in the motion type and the standard video clips corresponding to each video-level action label as segmentation reference samples into the first segmentation processing model to be trained, to obtain the first segmentation processing model to be trained in which the segmentation reference samples are deployed;

[0102] Step 304: input the batch motion video into the first segmentation processing model to be trained in which the segmentation reference sample is deployed, perform segmentation processing training on the first segmentation processing model, and obtain a pre-trained first segmentation processing model.

[0103] Through the first segmentation processing model, the entire recorded video is first cut out of the game clips that do not contain technical movements, and only the video clips containing technical movements that are finally cut out are retained. This processing method not only ensures that the athlete's performance can be scored from the perspective of key movements, but also reduces the consumption of computer processing resources.

[0104] Continue to refer Figure 4 , in some optional implementations, Figure 4 yes Figure 3 The flowchart of a specific embodiment of step 304 shown includes the following steps:

[0105] Step 401, obtaining all video-level action labels output by the first segmentation processing model according to the batch motion video, and segmentation segments corresponding to each video-level action label, as pre-training output results;

[0106] Step 402, comparing the pre-training output result with the labeling result to obtain a comparison result;

[0107] Step 403, calculating the loss value of the pre-training output result compared to the labeling result according to the comparison result and a preset loss function;

[0108] Step 404: if the loss value does not meet the preset loss condition, adjust the hyperparameters of the first segmentation processing model, and re-execute steps 401 to 403;

[0109] Step 405: If the loss value satisfies a preset loss condition, the pre-training of the first segmentation processing model is completed, and a pre-trained first segmentation processing model is obtained.

[0110] By comparing the pre-training output result with the annotation result, calculating the loss value of the pre-training output result compared with the annotation result according to the comparison result and a preset loss function, and adjusting the hyperparameters of the first segmentation processing model, the availability of the first segmentation processing model that has been pre-trained is ensured, and the game clips that do not contain technical movements are first deleted from the entire recorded video, and only a number of video clips containing technical movements that are finally deleted and segmented are retained. This processing method not only ensures that the athlete's performance can be scored from the perspective of key movements, but also reduces the consumption of computer processing resources.

[0111] Continue to refer Figure 5 In some optional implementations, before step 203, a step of performing frame-level segmentation processing on all the segmented segments output from the batch motion video is also included. Figure 5 yes Figure 2 The flowchart of a specific embodiment of performing frame-level segmentation processing on all the segmented segments output from the batch motion video before step 203 includes the following steps:

[0112] Step 501, obtaining the frame window and frame step size preset in the target configuration file;

[0113] Step 502: deploy the framing window and the framing step size as frame-level segmentation parameters to a pre-built second segmentation processing model, and obtain a second segmentation processing model after the frame-level segmentation parameters are deployed as the preset second segmentation processing model;

[0114] Step 503, arranging all segmented segments output by the first segmentation processing model according to the batch motion video as input samples of the preset second segmentation processing model;

[0115] Step 504: input the input sample into the preset second segmentation processing model to obtain a time-series frame-level segmentation processing result corresponding to the input sample output by the second segmentation processing model.

[0116] By performing frame-level processing on the main video clips containing technical movements obtained after processing by the first segmentation processing model, the athlete's performance is scored based on the timed frame-level cutting processing results output by the second segmentation processing model, which is more accurate than video-level scoring.

[0117] Continue to refer Figure 6 , in some optional implementations, Figure 6 yes Figure 5 The flowchart of a specific embodiment of step 503 shown includes the following steps:

[0118] Step 601, sorting out the segmented segments respectively included in the same video from all the segmented segments according to the distinguishing identification information of the batch of motion videos;

[0119] Step 602, using image information extraction technology, extracting recording timestamp information from all segmented segments contained in the same video to obtain recording timestamp information;

[0120] Step 603, time-sequentially arranging all the segmented segments contained in the same video according to the recording timestamp information to obtain the segmented segments after time-sequential arrangement;

[0121] Step 604: Use the time-series segmented fragments as input samples of the preset second segmentation processing model.

[0122] By sorting out the sliced ​​segments respectively contained in the same video from all the sliced ​​segments, using image information extraction technology, respectively extracting the recording timestamp information of all the sliced ​​segments contained in the same video to obtain the recording timestamp information, and time-sequencing all the sliced ​​segments contained in the same video according to the recording timestamp information to obtain the time-sequentially sorted sliced ​​segments, thereby ensuring that the subsequent second slice processing model outputs a timed frame-level cutting processing result.

[0123] Continue to refer Figure 7 In some optional implementations, before step 204, a step of training the frame-level action prediction model to be trained is also included. Figure 7 yes Figure 2 The flowchart of a specific embodiment of learning and training the frame-level action prediction model to be trained before step 204 is shown, comprising the following steps:

[0124] Step 701, presetting all key actions in the sports type and example images corresponding to each key action according to manual experience;

[0125] Step 702, deploying all key actions in the motion type and the example images corresponding to each key action as prediction reference knowledge into the frame-level action prediction model to be trained, to obtain the frame-level action prediction model to be trained in which the prediction reference knowledge is deployed;

[0126] Step 703, arranging the time-series frame-level segmentation processing results output by the second segmentation processing model as input samples of the frame-level action prediction model to be trained;

[0127] Step 704: input the input sample into the frame-level motion prediction model to be trained, perform learning training on the frame-level motion prediction model to be trained, and obtain a pre-trained frame-level motion prediction model.

[0128] In this embodiment, the purpose of learning and training the frame-level action prediction model to be trained is to enable the pre-trained frame-level action prediction model to identify the key actions in each frame of the cutting processing result based on the example images corresponding to each key action.

[0129] By learning and training the frame-level action prediction model, it is ensured that in actual use, the different key actions corresponding to each frame cutting processing result can be predicted according to the pre-trained frame-level action prediction model, and the prediction results can be obtained, so that the different key actions contained in the target video and the number of times different key actions appear can be identified.

[0130] In some optional implementations, the step 204, inputting the timed frame-level cutting processing results into a pre-trained frame-level action prediction model, predicting different key actions corresponding to each frame cutting processing result, and obtaining the prediction results, specifically includes: predicting the probability values ​​of the current frame cutting processing results belonging to different key actions according to all the key actions in the motion type and the example images corresponding to each key action; comparing the probability values ​​of the current frame cutting processing results belonging to different key actions, and screening out the key action corresponding to the maximum probability value as the key action corresponding to the current frame cutting processing result; obtaining the different key actions corresponding to each frame cutting processing result as the prediction result.

[0131] In some optional implementations, the step 205 is a step of counting the relative proportions of all key actions in the target video according to the prediction results, specifically including: obtaining frame cutting processing results corresponding to all key actions according to the prediction results; based on the frame cutting processing results corresponding to all key actions, counting the number of cut images corresponding to all key actions and the total number of cut images corresponding to all key actions; calculating the proportion value according to the number of cut images corresponding to all key actions and the total number of cut images corresponding to all key actions, to obtain the relative proportions of all key actions in the target video.

[0132] By counting the relative proportions of all key actions in the target video, the technical actions used by the athletes on both sides can be analyzed, and fair scoring can be performed based on the statistical results of the technical actions. It can also assist coaches in conducting strategic analysis of the target athletes and thus formulate relevant combat strategies.

[0133] In some optional implementations, the step 206 scores the target video based on the relative proportions of all the key actions in the target video and a preset scoring formula to obtain a scoring result, specifically including:

[0134] The preset scoring formula is used:

[0135]

[0136] Calculate the score result of the target video, where Score represents the score result of the target video, k represents the number of key action categories, and α k Indicates the relative proportion of the current key action, s k Indicates the score reference value corresponding to the current key action, where different score reference values ​​are set for different key actions according to the difficulty of the key actions.

[0137] Specifically, the target video is segmented twice to obtain a time-series frame-level segmentation result, and then the different key actions contained in the target video and the number of times different key actions appear are identified according to the pre-trained frame-level action prediction model. The segmentation and action prediction are performed in a model manner, which is more automated and intelligent. At the same time, the technical actions used by the athletes on both sides can be analyzed, and fair scoring can be performed based on the statistical results of the technical actions. Coaches can also be assisted in conducting a strategic analysis of the target athletes, thereby formulating relevant combat strategies and guidance strategies, reducing the time consumption of manual analysis.

[0138] The present application obtains a target video; inputs the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model; inputs the first segmentation video segment into a preset second segmentation processing model to obtain a timed frame-level segmentation processing result output by the second segmentation processing model; inputs the timed frame-level segmentation processing result into a pre-trained frame-level action prediction model to predict different key actions corresponding to each frame segmentation processing result to obtain a prediction result; based on the prediction result, the relative proportion of all key actions in the target video is counted; based on the relative proportion of all key actions in the target video and a preset scoring formula, the target video is scored to obtain a scoring result. The scoring method described in the present application is applied to a sports game scoring scenario, and can identify different key actions contained in the video according to the video recorded during the game, and can be fairly scored according to the statistical results of the technical actions of the athletes on both sides, and can also assist coaches in strategically analyzing the target athletes, thereby formulating relevant combat strategies.

[0139] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0140] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0141] In an embodiment of the present application, a target video is obtained; the target video is input into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model; the first segmentation video segment is input into a preset second segmentation processing model to obtain a timed frame-level segmentation processing result output by the second segmentation processing model; the timed frame-level segmentation processing result is input into a pre-trained frame-level action prediction model to predict different key actions corresponding to each frame segmentation processing result to obtain a prediction result; according to the prediction result, the relative proportion of all key actions in the target video is counted; based on the relative proportion of all key actions in the target video and a preset scoring formula, the target video is scored to obtain a scoring result. The scoring method described in the present application is applied to a sports game scoring scenario, and can identify different key actions contained in the video according to the video recorded during the game, and can be fairly scored according to the statistical results of the technical actions of the athletes on both sides, and can also assist coaches in strategically analyzing the target athletes, thereby formulating relevant combat strategies.

[0142] Further references Figure 8 , as a response to the above Figure 2 The present application provides an embodiment of a video action scoring device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0143] like Figure 8 As shown, the video action scoring device 800 described in this embodiment includes: a target video acquisition module 801, a first segmentation processing module 802, a second segmentation processing module 803, a key action prediction module 804, a relative proportion statistics module 805 and a target video scoring module 806. Among them:

[0144] The target video acquisition module 801 is used to acquire the target video;

[0145] The first segmentation processing module 802 is used to input the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model;

[0146] A second segmentation processing module 803 is used to input the first segmented video clip into a preset second segmentation processing model to obtain a time-series frame-level segmentation processing result output by the second segmentation processing model;

[0147] The key action prediction module 804 is used to input the time-series frame-level segmentation processing results into the pre-trained frame-level action prediction model, predict different key actions corresponding to each frame segmentation processing result, and obtain prediction results;

[0148] A relative proportion statistics module 805 is used to count the relative proportions of all key actions in the target video according to the prediction results;

[0149] The target video scoring module 806 is used to score the target video based on the relative proportions of all the key actions in the target video and a preset scoring formula to obtain a scoring result.

[0150] The present application obtains a target video; inputs the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model; inputs the first segmentation video segment into a preset second segmentation processing model to obtain a timed frame-level segmentation processing result output by the second segmentation processing model; inputs the timed frame-level segmentation processing result into a pre-trained frame-level action prediction model to predict different key actions corresponding to each frame segmentation processing result to obtain a prediction result; based on the prediction result, the relative proportion of all key actions in the target video is counted; based on the relative proportion of all key actions in the target video and a preset scoring formula, the target video is scored to obtain a scoring result. The scoring method described in the present application is applied to a sports game scoring scenario, and can identify different key actions contained in the video according to the video recorded during the game, and can be fairly scored according to the statistical results of the technical actions of the athletes on both sides, and can also assist coaches in strategically analyzing the target athletes, thereby formulating relevant combat strategies.

[0151] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0152] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0153] To solve the above technical problems, the present application also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.

[0154] The computer device 9 includes a memory 9a, a processor 9b, and a network interface 9c which are interconnected through a system bus. It should be noted that: Fig. 9 Only a computer device 9 having a component memory 9a, a processor 9b, and a network interface 9c is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0155] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0156] The memory 9a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 9a can be an internal storage unit of the computer device 9, such as a hard disk or memory of the computer device 9. In other embodiments, the memory 9a can also be an external storage device of the computer device 9, such as a plug-in hard disk equipped on the computer device 9, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 9a can also include both the internal storage unit of the computer device 9 and its external storage device. In this embodiment, the memory 9a is usually used to store the operating system and various application software installed on the computer device 9, such as a computer-readable instruction of a video action scoring method, etc. In addition, the memory 9a can also be used to temporarily store various types of data that have been output or are to be output.

[0157] The processor 9b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 9b is generally used to control the overall operation of the computer device 9. In this embodiment, the processor 9b is used to run computer-readable instructions or process data stored in the memory 9a, such as computer-readable instructions for running the video action scoring method.

[0158] The network interface 9c may include a wireless network interface or a wired network interface. The network interface 9c is generally used to establish a communication connection between the computer device 9 and other electronic devices.

[0159] The computer device proposed in this embodiment belongs to the field of image processing technology and is applied to the scoring scene of athletes' technical actions. This application obtains a target video; inputs the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model; inputs the first segmentation video segment into a preset second segmentation processing model to obtain a timed frame-level segmentation processing result output by the second segmentation processing model; inputs the timed frame-level segmentation processing result into a pre-trained frame-level action prediction model to predict different key actions corresponding to each frame segmentation processing result, and obtains a prediction result; according to the prediction result, the relative proportion of all key actions in the target video is counted; based on the relative proportion of all key actions in the target video and a preset scoring formula, the target video is scored to obtain a scoring result. The scoring method described in this application is applied to the scoring scene of sports competitions, and can identify different key actions contained in the video according to the video recorded during the game, and can be fairly scored according to the statistical results of the technical actions of both athletes, and can also assist coaches in strategic analysis of target athletes, thereby formulating relevant battle strategies.

[0160] The present application also provides another embodiment, that is, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by a processor to enable the processor to perform the steps of the video action scoring method as described above.

[0161] The computer-readable storage medium proposed in this embodiment belongs to the field of image processing technology and is applied to the scoring scene of athletes' technical actions. This application obtains a target video; inputs the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model; inputs the first segmentation video segment into a preset second segmentation processing model to obtain a timed frame-level segmentation processing result output by the second segmentation processing model; inputs the timed frame-level segmentation processing result into a pre-trained frame-level action prediction model to predict different key actions corresponding to each frame segmentation processing result, and obtains a prediction result; according to the prediction result, the relative proportion of all key actions in the target video is counted; based on the relative proportion of all key actions in the target video and a preset scoring formula, the target video is scored to obtain a scoring result. The scoring method described in this application is applied to the scoring scene of sports competitions, and can identify different key actions contained in the video according to the video recorded during the game, and can be fairly scored according to the statistical results of the technical actions of both athletes, and can also assist coaches in strategic analysis of target athletes, thereby formulating relevant battle strategies.

[0162] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0163] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A video action scoring method, characterized in that: The steps include: Get the target video; Inputting the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model; Inputting the first segmented video clip into a preset second segmentation processing model to obtain a time-series frame-level segmentation processing result output by the second segmentation processing model; Inputting the time-series frame-level cutting processing results into a pre-trained frame-level action prediction model, predicting different key actions corresponding to each frame-level cutting processing result, and obtaining a prediction result; According to the prediction results, the relative proportions of all key actions in the target video are counted; Based on the relative proportions of all the key actions in the target video and a preset scoring formula, the target video is scored to obtain a scoring result.

2. The video action scoring method according to claim 1, characterized in that: Before executing the step of inputting the target video into the pre-trained first segmentation processing model to obtain the first segmentation video segment output by the first segmentation processing model, the method further includes: Acquire a batch of motion videos of the same motion type as the target video, and annotate the video-level action labels respectively contained in each motion video in the batch of motion videos to obtain an annotation result; Presetting all video-level action labels of the motion type and standard video clips corresponding to each video-level action label according to manual experience; Deploy all video-level action labels in the motion type and the standard video clips corresponding to each video-level action label as segmentation reference samples into the first segmentation processing model to be trained, and obtain the first segmentation processing model to be trained in which the segmentation reference samples are deployed; The batch motion videos are input into the first segmentation processing model to be trained in which the segmentation reference samples are deployed, segmentation processing training is performed on the first segmentation processing model, and the pre-trained first segmentation processing model is obtained.

3. The video action scoring method according to claim 2, characterized in that: The step of inputting the batch motion video into the first segmentation processing model to be trained in which the segmentation reference sample is deployed, performing segmentation processing training on the first segmentation processing model, and obtaining the pre-trained first segmentation processing model specifically includes: Step 401, obtaining all video-level action labels output by the first segmentation processing model according to the batch motion video, and segmentation segments corresponding to each video-level action label, as pre-training output results; Step 402, comparing the pre-training output result with the labeling result to obtain a comparison result; Step 403, calculating the loss value of the pre-training output result compared to the labeling result according to the comparison result and a preset loss function; Step 404: if the loss value does not meet the preset loss condition, adjust the hyperparameters of the first segmentation processing model, and re-execute steps 401 to 403; Step 405: If the loss value satisfies a preset loss condition, the pre-training of the first segmentation processing model is completed, and a pre-trained first segmentation processing model is obtained.

4. The video action scoring method according to claim 2, characterized in that: Before executing the step of inputting the first segmented video clip into a preset second segmentation processing model to obtain a time-series frame-level segmentation processing result output by the second segmentation processing model, the method further includes: Get the frame window and frame step size preset in the target configuration file; Deploy the framing window and the framing step size as frame-level segmentation parameters to a pre-built second segmentation processing model, and obtain a second segmentation processing model after the frame-level segmentation parameters are deployed as the preset second segmentation processing model; Arranging all segmented segments output by the first segmentation processing model according to the batch motion video as input samples of the preset second segmentation processing model; The input sample is input into the preset second segmentation processing model to obtain a time-series frame-level segmentation processing result corresponding to the input sample output by the second segmentation processing model.

5. The video action scoring method according to claim 4, characterized in that: The step of arranging all segmented segments output by the first segmentation processing model according to the batch motion video as input samples of the preset second segmentation processing model specifically includes: According to the distinguishing identification information of the batch of motion videos, sorting out the segmented segments respectively contained in the same video from all the segmented segments; Using image information extraction technology, extract the recording timestamp information of all the segmented segments contained in the same video to obtain the recording timestamp information; All the segmented segments contained in the same video are time-sequentially sorted according to the recording timestamp information to obtain the time-sequentially sorted segmented segments; The time-series segmented fragments are used as input samples of the preset second segmentation processing model.

6. The video action scoring method according to claim 4, characterized in that: Before executing the step of inputting the time-series frame-level cutting processing results into the pre-trained frame-level action prediction model to predict different key actions corresponding to each frame-level cutting processing result, and obtaining the prediction results, the method further includes: Presetting all key actions of the motion type and example images corresponding to each key action according to manual experience; Deploy all key actions in the motion type and the example images corresponding to each key action as prediction reference knowledge into the frame-level action prediction model to be trained, and obtain the frame-level action prediction model to be trained in which the prediction reference knowledge is deployed; Arranging the time-series frame-level segmentation processing results output by the second segmentation processing model as input samples of the frame-level action prediction model to be trained; Inputting the input sample into the frame-level motion prediction model to be trained, performing learning training on the frame-level motion prediction model to be trained, and obtaining a pre-trained frame-level motion prediction model; The step of inputting the time-series frame-level cutting processing results into the pre-trained frame-level action prediction model, predicting different key actions corresponding to each frame cutting processing result, and obtaining the prediction results specifically includes: According to all the key actions in the motion type and the example images corresponding to each key action, predict the probability values ​​of the current frame cutting processing result belonging to different key actions; The probability values ​​of the current frame cutting processing result belonging to different key actions are compared in magnitude, and the key action corresponding to the maximum probability value is selected as the key action corresponding to the current frame cutting processing result; Different key actions corresponding to each frame cutting processing result are obtained as the prediction result.

7. The video action scoring method according to claim 6, characterized in that: The step of counting the relative proportions of all key actions in the target video according to the prediction results specifically includes: According to the prediction results, frame cutting processing results corresponding to all key actions are obtained; Based on the frame cutting processing results corresponding to all the key actions, the number of cut images corresponding to all the key actions and the total number of cut images corresponding to all the key actions are counted; According to the number of cut images corresponding to all key actions and the total number of cut images corresponding to all key actions, a ratio value is calculated to obtain the relative proportion of all key actions in the target video; The step of scoring the target video based on the relative proportions of all the key actions in the target video and a preset scoring formula to obtain a scoring result specifically includes: The preset scoring formula is used: Calculate the score result of the target video, where Score represents the score result of the target video, k represents the number of key action categories, and α k Indicates the relative proportion of the current key action, s k Indicates the score reference value corresponding to the current key action.

8. A video action scoring device, characterized in that: include: A target video acquisition module is used to acquire the target video; A first segmentation processing module is used to input the target video into a pre-trained first segmentation processing model to obtain a first segmentation video segment output by the first segmentation processing model; A second segmentation processing module, used for inputting the first segmented video clip into a preset second segmentation processing model, and obtaining a time-series frame-level segmentation processing result output by the second segmentation processing model; A key action prediction module is used to input the time-series frame-level cutting processing results into a pre-trained frame-level action prediction model, predict different key actions corresponding to each frame cutting processing result, and obtain prediction results; A relative proportion statistics module is used to count the relative proportions of all key actions in the target video according to the prediction results; The target video scoring module is used to score the target video based on the relative proportions of all the key actions in the target video and a preset scoring formula to obtain a scoring result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the video action scoring method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the video action scoring method according to any one of claims 1 to 7 are implemented.

Images