A method and device for automatically segmenting a swing-type sports video

Abstract

This invention provides a method and apparatus for automatically segmenting racket-swinging sports videos. The method includes: decomposing the racket-swinging sports video stream into video frames and inputting them into a motion recognition model to obtain a confidence set of the athlete's actions on each first-type video frame; inputting the confidence sets of each first-type video frame into a keyframe discrimination model to obtain the action video frame type to which each first-type video frame belongs; segmenting the sports video stream according to the positions of each serve start video frame and rally end video frame in the sports video stream; and combining them to obtain a set of valid rally videos for racket-swinging sports. This invention can automatically segment racket-swinging sports videos, improving the segmentation efficiency of racket-swinging sports videos.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for automatically segmenting videos of racket-swinging sports. Background Technology

[0002] Segmenting and recombining racket sports match videos by valid scoring rallies is a common way to improve the viewing experience. However, current methods for segmenting racket sports matches typically involve manual segmentation using video editing software, followed by combining all valid scoring rallies to create a highlight reel. Existing manual segmentation methods are only suitable for professional racket sports videos. Furthermore, manual segmentation of racket sports videos consumes significant human and material resources and is inefficient. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide an automatic video segmentation method and apparatus for racket-swinging sports to solve the problem of low segmentation efficiency of racket-swinging sports competition videos in the prior art.

[0004] One aspect of the present invention provides an automatic segmentation method for racket-swinging sports videos, the method comprising the following steps:

[0005] Acquire a motion video stream, decompose the motion video stream into multiple video frames, and form a video frame sequence;

[0006] The video frame sequence is input into the target detection model to obtain multiple first-class video frames with the location of the athlete. Each first-class video frame is input into the pre-trained action recognition model to obtain the set of action confidence of the athlete on each first-class video frame.

[0007] The action confidence set of each first-class video frame is input into the pre-trained keyframe discrimination model to obtain the action video frame type of each first-class video frame. The action video frame type includes: serve start video frame and rally end video frame. The positions of each serve start video frame and rally end video frame in the motion video stream are determined, and multiple effective scoring rally segments are obtained.

[0008] Multiple valid scoring round segments are segmented from the motion video stream and combined to form a set of valid round videos for racket-swinging motions.

[0009] In some embodiments, the object detection model is a FastRCNN model.

[0010] In some embodiments, the pre-training step of the action recognition model includes:

[0011] Obtain a first training sample set, which consists of multiple sample video frames containing action annotations;

[0012] Obtain the Slow Fast initial model, and train the Slow Fast initial model using the first training sample set to obtain the action recognition model.

[0013] In some embodiments, the pre-training step of the keyframe discrimination model includes:

[0014] Obtain a second training sample set, which consists of multiple samples. Each sample includes a set of sample motion confidence scores. Add the motion video frame type as a label to each set of sample motion confidence scores.

[0015] Obtain the initial random forest model, and train the initial random forest model using the second training sample set to obtain the keyframe discrimination model.

[0016] In some embodiments, a TSM model or a GCN model is obtained as an initial model, and the action recognition model is obtained by training with the first training sample set.

[0017] In some embodiments, an SVM model is obtained as an initial model, and the keyframe discrimination model is obtained by training the model using the second training sample set.

[0018] Another aspect of the present invention provides an automatic video segmentation device for racket-swinging sports, the device comprising:

[0019] Action recognition module: acquires a first training sample set, which consists of multiple sample video frames with action annotations; acquires a Slow Fast initial model, trains the Slow Fast initial model using the first training sample set to obtain an action recognition model, and forms an action recognition model file;

[0020] Keyframe discrimination module: Obtain a second training sample set, which consists of multiple samples, each sample including a sample motion confidence set, and add action video frame type as a label to each sample motion confidence set; Obtain a random forest initial model, use the second training sample set to train the random forest initial model to obtain a keyframe discrimination model, and form a keyframe discrimination model parameter file;

[0021] Automatic segmentation module: Acquires a motion video stream, decomposes the motion video stream into multiple video frames to form a video frame sequence; inputs the video frame sequence into a target detection model to obtain multiple first-class video frames containing athlete location annotations; inputs each first-class video frame into a pre-trained action recognition model to obtain the athlete's action confidence set for each first-class video frame; inputs the action confidence set of each first-class video frame into a pre-trained keyframe discrimination model to obtain the action video frame type to which each first-class video frame belongs, the action video frame type including: serve start video frame, rally end video frame; determines the position of each serve start video frame and rally end video frame in the motion video stream, and obtains multiple valid scoring rally segments; segments the multiple valid scoring rally segments from the motion video stream and combines them to form a set of valid rally videos for racket-type movements.

[0022] In some embodiments, the action recognition model file includes: the action recognition model and the target detection model.

[0023] On the other hand, the present invention also provides an electronic device including a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory, wherein when the computer instructions are executed by the processor, the device implements the steps of the above method.

[0024] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0025] The beneficial effects of the present invention are at least as follows:

[0026] The automatic video segmentation method and apparatus for racket-swinging sports described in this invention decomposes the video stream of racket-swinging sports into video frames, inputs them into a motion recognition model to obtain a confidence set of the athlete's actions on each first-class video frame, inputs the confidence sets of each video frame into a keyframe discrimination model to obtain the type of motion video frame to which each first-class video frame belongs, segments the motion video stream according to the positions of each serve start video frame and rally end video frame in the motion video stream, and combines them to obtain a set of valid rally videos of racket-swinging sports. By automatically segmenting racket-swinging sports videos by machine, no manual intervention is required, improving the segmentation efficiency of racket-swinging sports videos and reducing the waste of human and material resources.

[0027] Furthermore, by using a keyframe discrimination model to classify each first-class video frame, the serve start video frame and the rally end video frame in the first-class video frame are found. That is, the two node video frames of the valid scoring rally segment are found. The location of the valid rally segment can be found in the video stream without having to judge whether each video frame belongs to the valid rally segment frame by frame, thus improving the efficiency of video segmentation.

[0028] Furthermore, the videos of racket-swinging sports are segmented and combined into a set of effective rounds of racket-swinging sports videos, and video frames irrelevant to the competition process are removed, thereby improving the user's viewing experience of racket-swinging sports videos.

[0029] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0030] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0031] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0032] Figure 1 This is an embodiment of the automatic video segmentation device for racket-swinging sports described in this invention.

[0033] Figure 2 This is a flowchart of an automatic video segmentation method for racket-swinging sports according to an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0035] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0036] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0037] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0038] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0039] A video of a racket sport match, such as table tennis, badminton, or tennis, typically consists of several scoring rallies, several losing rallies, and other non-rally-bound periods. These videos usually have the following characteristics: short scoring rallies, many losing rallies, and long non-rally-bound periods. To a certain extent, losing rallies and non-rally-bound periods negatively impact the viewing experience of racket sport matches. Therefore, segmenting and recombining racket sport matches according to scoring rallies is a common method to improve the viewing experience. However, existing technologies typically involve manually segmenting sports videos and then combining all the scoring rallies to output a highlight video of the racket sport match. However, manual segmentation requires significant manpower and resources and is inefficient. Furthermore, existing sports video segmentation methods are only suitable for professional racket sport matches and cannot segment racket sport videos filmed by amateur users. Therefore, this invention provides an automatic segmentation method and apparatus for racket sport videos, solving the problem of low segmentation efficiency in existing technologies.

[0040] One aspect of the present invention provides an automatic segmentation method for racket-swinging sports videos, such as... Figure 2 As shown, the method includes steps S101 to S104:

[0041] S101: Acquire the motion video stream, decompose the motion video stream into multiple video frames and form a video frame sequence.

[0042] S102: Input the video frame sequence into the target detection model to obtain multiple first-class video frames with the location of the athlete. Input each first-class video frame into the pre-trained action recognition model to obtain the action confidence set of the athlete on each first-class video frame.

[0043] S103: Input the action confidence set of each first-class video frame into the pre-trained keyframe discrimination model to obtain the action video frame type of each first-class video frame. The action video frame type includes: serve start video frame and rally end video frame; determine the position of each serve start video frame and rally end video frame in the motion video stream, and obtain multiple valid scoring rally segments.

[0044] S104: Extract multiple valid scoring round segments from the motion video stream and combine them to form a set of valid round videos for racket-swinging motions.

[0045] In step S101, the camera is positioned at the angle of the competition broadcast, and then a video of a racket-swinging sport is captured. The captured video is then decomposed into multiple video frames to facilitate subsequent classification of each video frame type.

[0046] In step S102, the Fast R-CNN model is used as the object detection model. Each video frame is input into the object detection model, and the output is a first-class video frame containing the bounding boxes of the athletes' locations. The object detection model outputs the bounding boxes of all participating athletes in each video frame. For example, when the motion in the video frame is a two-person racket swing, the object detection model outputs a first-class video frame containing the bounding boxes of both athletes' locations; when the motion in the video frame is a racket swing between a single person and the ball machine, the object detection model outputs a first-class video frame containing the bounding box of a single athlete's location. Each first-class video frame is input into a pre-trained action recognition model to obtain a confidence set of all possible actions of the athletes in each first-class video frame.

[0047] In some embodiments, the SSD model is used as the target detection model to detect the location of the athlete in each video frame.

[0048] In some embodiments, the pre-training step of the action recognition model includes S201 to S202:

[0049] S201: Obtain the first training sample set, which consists of multiple sample video frames containing action annotations.

[0050] S202: Obtain the Slow Fast initial model and train the Slow Fast initial model using the first training sample set to obtain the action recognition model.

[0051] In step S201, the types of athlete movements in multiple sample video frames are labeled. The movement types include: serving movements, preparation movements, hitting movements, etc.

[0052] In some embodiments, when there are two athletes in a sample video frame, such as when the sample video frame is obtained from a table tennis match video, the action types of both athletes in the sample video frame need to be labeled. When there is only one athlete in a sample video frame, only the action type of that single athlete needs to be labeled. For example, when an athlete is performing a racket-swinging motion using a ball-serving machine, there is only one athlete in each video frame.

[0053] In step S202, the Slow Fast initial model is trained using the first training sample set, and the action recognition model is obtained by iteratively updating the Slow Fast initial model using the focal loss function.

[0054] In this embodiment, Slow Fast uses a slow, high-resolution convolutional neural network to analyze the static information in each sample video frame, while a fast, low-resolution convolutional neural network is used to analyze the dynamic information in each sample video frame. The action recognition model outputs a set of confidence scores for all possible actions of the athlete in each of the first-class video frames.

[0055] In some embodiments, a TSM model or a GCN model is obtained as an initial model, and an action recognition model is obtained by training with a first training sample set. By comparing the performance of the three action recognition models obtained by using the Slow Fast model, the TSM model, and the GCN model as initial models, the action recognition model with the highest action recognition accuracy is selected as the final action recognition model for application.

[0056] In step S103, to accurately segment valid scoring rallies, the judgment cannot rely solely on the athlete's technical movement classification results. Instead, the set of movement classification confidence scores for each frame of the athlete in the match video is input into the keyframe discrimination model. The model determines the type of action video frame to which each frame belongs based on the confidence score features. Action video frame types include: serve start video frame, rally end video frame, and other video frames. After finding the positions of each serve start video frame and each rally end video frame in the motion video stream, multiple valid scoring rally segments can be obtained from a single motion video stream. A valid scoring rally segment consists of a serve start video frame, a rally end video frame, and other video frames between the serve start and rally end video frames. Other video frames include hitting video frames, athlete retrieving the ball video frames, etc. The keyframe discrimination model's role is to find the serve start video frame and rally end video frame in each of the first-class video frames.

[0057] In some embodiments, the pre-training of the keyframe discrimination model includes steps S301 to S302:

[0058] S301: Obtain the second training sample set, which consists of multiple samples. Each sample includes a sample motion confidence set. Add the motion video frame type as a label to each sample motion confidence set.

[0059] S302: Obtain the initial random forest model, and use the second training sample set to train the initial random forest model to obtain the keyframe discrimination model.

[0060] In step S301, an action video frame type is added as a label to each sample motion confidence set. Action video frame types include: serve start video frame, rally end video frame, hit video frame, and retrieve video frame. Each sample video frame is processed by the action recognition model to output a sample confidence set.

[0061] In step S302, the initial random forest model is trained using the second training sample set, and the accuracy of the predicted action video frame type output by the initial random forest model is used as the loss to iteratively update the initial random forest model to obtain the keyframe discrimination model. Each decision tree in the random forest model judges the confidence set of each input first-class video frame to obtain an action video frame type, and the action video frame type with the most identical judgment results is output as the action video frame type of that first-class video frame.

[0062] In some embodiments, an SVM model is obtained as the initial model, and a keyframe discrimination model is obtained by training with a second training sample set. The discrimination performance of the keyframe discrimination models obtained by using the SVM model and the random forest model as the initial models is compared, and the keyframe discrimination model with the highest discrimination accuracy is used as the keyframe discrimination model for the final application.

[0063] In step S104, the video of a racket-style sport competition is segmented into valid scoring round segments and then recombined into a set of valid round videos of racket-style sports. Video frames that are irrelevant to the competition result are removed from the competition video, so that the audience can directly watch the valid scoring videos, which improves the user's viewing experience.

[0064] The automatic video segmentation method for racket-swinging sports of the present invention can improve the segmentation efficiency of racket-swinging sports videos, and the method is easy to implement and highly scalable.

[0065] Another aspect of the present invention provides an automatic video segmentation device for racket-swinging sports, such as... Figure 1 As shown, the device includes:

[0066] Action Recognition Module: This module acquires a first training sample set, consisting of multiple sample video frames with action annotations; it also acquires a Slow Fast initial model, trains the Slow Fast initial model using the first training sample set to obtain an action recognition model, and generates an action recognition model file. This module is used to train the action recognition model required by the automatic racket swing motion segmentation device using the first training sample set.

[0067] In some embodiments, the action recognition module comprises three components: A, B, and C. Component A is a training dataset for recognizing racket-type sports actions. This invention constructs a training dataset for recognizing racket-type sports actions based on a large number of racket-type sports competition videos by annotating the athletes' technical movements, including serving, preparation, and hitting actions. Component B is a machine learning-based action recognition model. In this invention, action recognition model networks such as SlowFast, GCN, and TSM can be used as initial models and trained to obtain an athlete technical movement recognition model based on a racket-type sports competition scene. Component C is a machine learning-based action recognition model file, used by the automatic segmentation device to recognize and classify the athletes' technical movements in the racket-type sports competition videos and output a set of confidence scores for all technical movements of the athletes in each frame.

[0068] In some embodiments, the action recognition model file includes an action recognition model and an object detection model. The object detection model is used to detect the location of the athlete in each video frame, and the action recognition model is used to detect the type of action of the athlete in each first-class video frame.

[0069] Keyframe discrimination module: This module acquires a second training sample set, consisting of multiple samples, each including a set of motion confidence scores. Action video frame types are added as labels to each set of motion confidence scores. It then acquires an initial random forest model and trains it using the second training sample set to obtain a keyframe discrimination model, generating a parameter file for the keyframe discrimination model. This module is used to train the keyframe discrimination model required by the automatic racket swing motion segmentation device using the second training sample set.

[0070] Comparing the action confidence data of all first-class video frames reveals similar characteristics between the action confidence of the serve start video frame or the rally end video frame in each match rally. Therefore, the first-class video frames of racket-swinging sports match videos can be classified by learning the characteristics of the athlete's technical action confidence in each video frame. This module has three components: D, E, and F. Component D is the keyframe dataset. This invention annotates the motion video frame type by the action confidence set of each first-class video frame output by the action recognition module and constructs a keyframe discrimination training dataset. Component E is the machine learning-based keyframe discrimination model. In this invention, a random forest model or an SVM model can be used as the initial model and trained to obtain the keyframe discrimination model. Component F is the parameter file of the machine learning-based keyframe discrimination model. In the automatic segmentation device, all video frames of the racket-swinging sports match video stream are classified to find the serve start video frame and the rally end video frame in each first-class video frame.

[0071] Automatic segmentation module: Acquires the motion video stream, decomposes it into multiple video frames, and forms a video frame sequence; inputs the video frame sequence into a target detection model to obtain multiple first-class video frames with athlete location annotations; inputs each first-class video frame into a pre-trained action recognition model to obtain the athlete's action confidence set for each first-class video frame; inputs the action confidence set of each first-class video frame into a pre-trained keyframe discrimination model to obtain the action video frame type of each first-class video frame, including: serve start video frame and rally end video frame; determines the position of each serve start video frame and rally end video frame in the motion video stream, and obtains multiple valid scoring rally segments; a valid scoring rally segment consists of a serve start video frame, a rally end video frame, and other video frames between the serve start video frame and the rally end video frame; segments multiple valid scoring rally segments from the motion video stream and combines them to form a set of valid rally videos for racket-type movements.

[0072] After the video of a racket-swinging sport is input into the automatic segmentation device, the automatic segmentation module uses the motion recognition model file in the motion recognition module and the key frame discrimination model parameter file in the key frame discrimination module to analyze and discriminate each video frame. Finally, it automatically outputs the segmented set of valid round videos of racket-swinging sports. The automatic segmentation module has five components: G, H, I, J, and K.

[0073] Component G is the video stream to be segmented. After the user shoots a video of a racket-style sport competition, the video is input into the automatic segmentation device. Component H is the sequence of video frames formed after decomposing component G into multiple video frames. Component I is the result obtained by the target detection model and action recognition model of component H, including the inferred results of the athlete's technical movements in each frame and the set of confidence scores for all technical movements. Component J is the motion video frame type discrimination result of each video frame output by component I after keyframe discrimination model, used to determine whether each frame is a serve start video frame or a rally end video frame. Component K is the set of valid rallies of racket-style sports videos formed by segmenting and combining components J after positional correspondence with component G.

[0074] In some embodiments, the automatic video segmentation device for racket-swinging sports connects to a video playback device via a Wi-Fi module, a Zigbee module, and / or a Bluetooth module. This device checks whether the processed set of valid rounds of racket-swinging sports videos meets the requirements. If the requirements are met, the set of valid rounds of racket-swinging sports videos is transmitted to a cloud platform for other users to download and watch. The wireless connection facilitates video transmission.

[0075] The following description uses a specific embodiment as an example:

[0076] This embodiment provides a method for automatically segmenting videos of racket-swinging sports, including:

[0077] Step 1.1: Input the training dataset for racket-swinging motion recognition into the latest machine learning human motion recognition model network to train the motion recognition model based on racket-swinging sports competition scenarios.

[0078] Step 1.2: Output a high-precision machine learning action recognition model file by comparing different machine learning networks, optimizing the dataset, and adjusting parameters.

[0079] Step 1.3: Input the motion recognition model file into the automatic segmentation module for the recognition and inference of athletes' technical movements in racket-swinging sports competition videos.

[0080] Step 2.1: Input the keyframe training dataset into a traditional machine learning classification network to train the keyframe discrimination model.

[0081] Step 2.2: Output a high-accuracy keyframe discrimination model parameter file by comparing different machine learning classification methods, data preprocessing, and parameter tuning.

[0082] Step 2.3: Input the keyframe discrimination model file into the automatic segmentation module for keyframe category discrimination in racket-swinging sports competition videos.

[0083] Step 3.1: After inputting the video stream to be segmented into the automatic segmentation device, the video segmentation tool automatically obtains each frame image and related image information of the input video frame sequence.

[0084] Step 3.2: Input the video frame sequence image set into the action recognition model for action recognition inference. First, use the target detection model in the action recognition model file to find the location of the athlete in each frame image. Then, input the human body information of the target box of the athlete in the multi-frame image sequence into the pre-trained action recognition model file for inference. The set of confidence scores for all the athlete's actions in the competition video can be obtained and output.

[0085] Step 3.3: In order to accurately segment the valid scoring rounds, we cannot rely solely on the classification results of the athletes' technical movements. Instead, we input the set of movement classification confidence scores for each frame of the two athletes in the match video into the keyframe discrimination model. Based on the confidence score features, we determine whether each frame belongs to the category of the serve start frame, the round end frame, or other frames, and determine the video location of all serve start frames and round end frames identified by the model.

[0086] Step 3.4: Combine the video positions of the serve start frame and the rally end frame to determine the video positions of all valid scoring rally segments.

[0087] Step 3.5: Divide the video into segments based on the start frame and end frame, and combine them into the final valid sphere video.

[0088] In summary, the automatic video segmentation method and apparatus for racket-type sports described in this invention decomposes the video stream of racket-type sports into video frames, inputs them into a motion recognition model to obtain a confidence set of the athlete's actions on each first-class video frame, inputs the confidence set of each video frame into a keyframe discrimination model to obtain the type of motion video frame to which each first-class video frame belongs, segments the motion video stream according to the position of each serve start video frame and rally end video frame in the motion video stream, and combines them to obtain a set of valid rally videos of racket-type sports. By automatically segmenting racket-type sports videos by machine, no manual intervention is required, improving the segmentation efficiency of racket-type sports videos and reducing the waste of human and material resources.

[0089] Furthermore, by using a keyframe discrimination model to classify each first-class video frame, the serve start video frame and the rally end video frame in the first-class video frame are found. That is, the two node video frames of the valid scoring rally segment are found. The location of the valid rally segment can be found in the video stream without having to judge whether each video frame belongs to the valid rally segment frame by frame, thus improving the efficiency of video segmentation.

[0090] Furthermore, the videos of racket-swinging sports are segmented and combined into a set of effective rounds of racket-swinging sports videos, and video frames irrelevant to the competition process are removed, thereby improving the user's viewing experience of racket-swinging sports videos.

[0091] Corresponding to the above method, the present invention also provides a system comprising a computer device including a processor and a memory, wherein the memory stores computer instructions, and the processor is configured to execute the computer instructions stored in the memory, wherein when the computer instructions are executed by the processor, the system implements the steps of the method as described above.

[0092] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0093] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0094] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0095] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0096] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatic cut detection of a swing sport video, characterized in that, The method comprises the following steps: Obtaining a sports video stream, decomposing the sports video stream into a plurality of video frames and forming a video frame sequence; Inputting the video frame sequence into a target detection model to obtain a plurality of first type video frames containing position labels of athletes, inputting each first type video frame into a pre-trained action recognition model to obtain an action confidence set of the athlete in each first type video frame; Inputting the action confidence set of each first type video frame into a pre-trained key frame discrimination model to obtain an action video frame type to which each first type video frame belongs, the action video frame type comprising a serve start video frame and a round end video frame; determining the positions of each serve start video frame and round end video frame in the sports video stream, and obtaining a plurality of effective scoring round segments; Cutting out the plurality of effective scoring round segments from the sports video stream and combining to form a racquet sports effective round video set; The pre-training step of the action recognition model comprises: obtaining a first training sample set, the first training sample set being composed of a plurality of sample video frames containing action labels; obtaining a Slow Fast initial model, and training the Slow Fast initial model using the first training sample set to obtain the action recognition model; The pre-training step of the key frame discrimination model comprises: obtaining a second training sample set, the second training sample set being composed of a plurality of samples, each sample comprising a sample motion confidence set, and adding an action video frame type as a label to each sample motion confidence set; obtaining a random forest initial model, and training the random forest initial model using the second training sample set to obtain the key frame discrimination model; Obtaining a TSM model or a GCN model as an initial model, and training the first training sample set to obtain the action recognition model.

2. The automatic swing-type sports video cut-division method according to claim 1, wherein The target detection model is a Fast RCNN model.

3. The automatic swing-type sports video cut-division method according to claim 1, wherein Obtaining an SVM model as an initial model, and training the second training sample set to obtain the key frame discrimination model.

4. An apparatus for automatically cutting a swing-type sports video, characterized by comprising: The device is used to execute the racquet sports video automatic cutting method according to any one of claims 1 to 3, and the device comprises: An action recognition module: obtaining a first training sample set, the first training sample set being composed of a plurality of sample video frames containing action labels; obtaining a Slow Fast initial model, training the Slow Fast initial model using the first training sample set to obtain an action recognition model, and forming an action recognition model file; A key frame discrimination module: obtaining a second training sample set, the second training sample set being composed of a plurality of samples, each sample comprising a sample motion confidence set, and adding an action video frame type as a label to each sample motion confidence set; obtaining a random forest initial model, training the random forest initial model using the second training sample set to obtain a key frame discrimination model, and forming a key frame discrimination model parameter file; and An action recognition module: obtaining a first training sample set, the first training sample set being composed of a plurality of sample video frames containing action labels; obtaining a Slow Fast initial model, training the Slow Fast initial model using the first training sample set to obtain an action recognition model, and forming an action recognition model file; A key frame discrimination module: obtaining a second training sample set, the second training sample set being composed of a plurality of samples, each sample comprising a sample motion confidence set, and adding an action video frame type as a label to each sample motion confidence set; obtaining a random forest initial model, training the random forest initial model using the second training sample set to obtain a key frame discrimination model, and forming a key frame discrimination model parameter file; and The automatic segmentation module: acquires a sports video stream, decomposes the sports video stream into a plurality of video frames and forms a video frame sequence; inputs the video frame sequence into a target detection model to obtain a plurality of first type video frames containing position labels of athletes; inputs each first type video frame into a pre-trained action recognition model to obtain an action confidence set of the athlete in each first type video frame; inputs the action confidence set of each first type video frame into a pre-trained key frame discrimination model to obtain an action video frame type to which each first type video frame belongs, wherein the action video frame type includes a serve start video frame and an end of round video frame; determines the positions of each serve start video frame and end of round video frame in the sports video stream, and obtains a plurality of valid scoring round segments; segments the plurality of valid scoring round segments from the sports video stream and combines them to form a set of valid round videos of the stroke type sports.

5. The apparatus according to claim 4, wherein The action recognition model file includes the action recognition model and the target detection model.

6. A system for automatic cut detection of a swing sport video, comprising a processor and a memory, wherein, The memory stores computer instructions, and the processor is configured to execute the computer instructions stored in the memory, so that the system implements the steps of the method in any one of claims 1 to 3.

7. A computer readable storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the steps of the method in any one of claims 1 to 3.

Images