Method and system for detecting basketball shooter in fixed-viewing-angle basketball event video and medium
Through the local optical flow method and the target detector combined with timing information, the problem of insufficient detection accuracy and real-time performance of shooters in basketball game videos is solved, and the effect of high accuracy and real-time detection is achieved.
Patent Information
- Application Number
- CN202411127208.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-16
AI Technical Summary
The prior art lacks the accuracy and real-timeness of shooter detection in basketball game videos, especially in complex dynamic scenarios, and traditional deep learning models are difficult to take into account both high precision and real-timeness.
The shooting fragments are intercepted by the local optical flow method, and the trained object detector combines timing information and local motion characteristics, and uses frame skipping to remove redundant video frames to accelerate inference to achieve high accuracy and real-time detection.
It significantly improves the accuracy and real-timeness of shooter detection, can work stably in complex scenarios, and provides strong technical support for basketball video analysis.
Smart Images

Figure CN119091502A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a method, system and medium for detecting a shooter in a fixed-viewing angle basketball game video. Background Art
[0002] Real-time analysis and intelligent evaluation of basketball games have become an important direction of modern sports technology research. The widespread application of video surveillance technology in sports events makes it possible to automatically detect and identify player movements through video analysis technology. In particular, the detection of shooting movements can not only provide valuable technical feedback for coaches and players, but also improve the audience's viewing experience and the professionalism of media broadcasts.
[0003] Traditional basketball shot detection methods mainly use feature-based detection algorithms. Although they can meet the needs to a certain extent, they often have the following problems:
[0004] 1. Limitations of feature detection algorithms. Detection algorithms based on traditional features (such as edge detection, motion trajectory analysis, etc.) are easily affected by factors such as video resolution, lighting conditions, and background interference, and have poor detection accuracy and robustness. 2. Complexity of dynamic scenes. In basketball game scenes, the rapid movement and complex action changes of players require the detection algorithm to have strong dynamic adaptability.
[0005] In recent years, with the development of deep learning technology, convolutional neural networks (CNN) have performed well in the field of image and video analysis. In particular, object detection algorithms based on deep learning, such as YOLO (You Only Look Once) and Faster R-CNN, have achieved remarkable results in various visual tasks. However, directly applying these algorithms to detect basketball shooters still faces some challenges:
[0006] 1. High efficiency requirements. Real-time processing of game videos places high demands on the speed of the algorithm. Traditional deep learning models often find it difficult to achieve real-time performance while maintaining high accuracy. 2. Multi-level information fusion. Shot detection not only needs to focus on the player’s motion characteristics, but also needs to combine scene context information, such as the position of the basketball stand, the spatial relationship between the player and the basketball stand, etc. Summary of the invention
[0007] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a method, system and medium for detecting shooters in fixed-view basketball game videos, so as to significantly improve the accuracy and real-time performance of shooter detection, be able to work stably in complex dynamic scenes, and provide strong technical support for basketball video analysis.
[0008] In a first aspect, a method for detecting a shooter in a fixed-view basketball game video is provided, comprising the following steps:
[0009] Obtain a basketball game video, and extract shooting clips from the basketball game video using the local optical flow method;
[0010] Perform object detection on the captured shooting clips using the trained object detector;
[0011] Accelerate object detector inference by removing redundant video frames through frame skipping;
[0012] The shooting moment is located based on the target detector inference results combined with local motion features. The shooter is found by traversing the result array in reverse order using the timing information from the shooting moment. The shooter with the highest confidence is found based on the confidence level, and the shooter with the highest quality is found based on the intersection and union ratio. Finally, the shooter with the highest confidence and the best quality is obtained.
[0013] Furthermore, the process of extracting shooting clips from a basketball game video by using the local optical flow method includes:
[0014] Read the basketball game video frame by frame and convert the pre-defined basket area in the video frame into a grayscale image;
[0015] The Farneback optical flow algorithm is used to calculate the optical flow vector of pixels between two adjacent grayscale images. The modulus of the average optical flow vector in the basket area is calculated to determine whether the net is swinging.
[0016] If the modulus of the average optical flow vector is greater than the modulus threshold, it is considered that the net swings, and the moment when the net swings is the moment of shooting and scoring. A preset number of video frames before and after the moment of shooting and scoring are intercepted to obtain a shooting clip, and the frame number of the shooting and scoring moment of the shooting clip is saved.
[0017] Furthermore, the target detector is trained by the following method:
[0018] Collect basketball game videos and annotate each frame in the basketball game videos to obtain a training sample data set;
[0019] The training dataset is used to train the YOLOv9-based object detector to obtain the final object detector.
[0020] Furthermore, the training sample data set is obtained by the following method:
[0021] Collect basketball game videos, remove video frames that do not contain players, and use the number of players to remove invalid video frames;
[0022] Label the locations and categories of objects in the remaining video frames, including basketballs, backboards, hoops, players, and shooters.
[0023] Data enhancement is performed on the labeled video frames, including random cropping, rotation, scaling, and color transformation, to obtain a training sample dataset.
[0024] Further, the accelerating the object detector reasoning by removing redundant video frames by skipping frames includes:
[0025] The frame number of the goal-scoring moment of the shooting clip is read, and the preset frame number of the video frame at the goal-scoring moment is skipped in reverse order based on the video frame at the goal-scoring moment, and the video frame to be detected is selected in forward order to the video frame at the goal-scoring moment, and the selected video frame is input into the target detector for target detection.
[0026] Furthermore, the positioning of the shooting moment based on the target detector inference result combined with the local motion feature specifically includes:
[0027] The target detection results of each frame are saved in the result array X. The contact area between the basketball and the basket is calculated to determine whether the basketball and the basket are in contact. When the basketball and the basket are in contact, it is considered as the shooting moment. The shooting moment video frame and the position information P of the basketball relative to the basket are recorded. b .
[0028] Furthermore, the shooter with the highest confidence and the best quality is obtained through the following process;
[0029] Traverse the result array X in reverse order frame by frame starting from the moment of shooting;
[0030] When a video frame with a shooter is traversed, the shooter with the highest confidence is selected from the video frame, and the position information P of the shooter relative to the basket is calculated. s ; Determine the position information P of the shooter relative to the basket s The position information P of the basketball relative to the basket at the time of shooting b Are they the same? If not, continue to traverse the video frames in reverse order; if so, determine whether the confidence of the shooter is greater than the set shooter confidence threshold. If so, it is considered to be a valid shooter and the video frame is recorded; otherwise, continue to traverse the video frames in reverse order;
[0031] Starting from the video frame with a valid shooter, the preset number of frames are traversed in reverse order to find the video frame whose shooter confidence is greater than the shooter confidence threshold and whose intersection-over-joint ratio between the shooter and the surrounding players is the lowest, so as to obtain the shooter with the highest confidence and the best quality.
[0032] Furthermore, in the process of searching for the shooter by using the timing information to reversely traverse the result array from the shooting moment, if the reverse traversal does not find a video frame greater than the shooter confidence threshold, that is, the shooter is not found, then the shooter confidence threshold is lowered according to the set attenuation coefficient, and the reverse traversal is performed again to search for the shooter; and the iteration is repeated until the shooter is found or the shooter confidence threshold is lower than the shooter confidence minimum threshold.
[0033] In a second aspect, a system for detecting a shooter in a fixed-view basketball game video is provided, comprising:
[0034] a memory having a computer program stored thereon;
[0035] A processor is used to load and execute the computer program to implement the method for detecting shooters in fixed-viewing angle basketball game videos as described above.
[0036] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the method for detecting a shooter in a fixed-view basketball game video as described above is implemented.
[0037] The present invention proposes a method, system and medium for detecting a shooter in a fixed-view basketball game video, which has the following beneficial effects:
[0038] 1. In order to solve the problem of finding the shooter in a complex basketball scene, the present invention uses a target detector, uses pre-collected basketball game video data to train the target detector, and uses the target detector to automatically detect the shooter;
[0039] 2. In order to solve the problems of low accuracy in identifying the shooter and slow processing speed, the present invention uses temporal local motion features for rule judgment to achieve high accuracy in detecting the shooter; uses the local optical flow method to capture shooting clips, and uses frame skipping and early termination to accelerate the reasoning of the target detector, thereby achieving real-time detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 It is a flow chart of a method for detecting a shooter in a fixed-viewing angle basketball game video provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0042] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.
[0043] In order to improve the accuracy of identifying the shooter in basketball videos and realize real-time identification of the shooter, Figure 1 As shown, the present invention proposes a method for detecting shooters in fixed-view basketball game videos based on programmable gradient information and a generalized efficient layer aggregation network, comprising the following steps:
[0044] S1: Training data collection.
[0045] The video data used in this embodiment are all shot by a 1080P high-definition camera in a fixed position. In order to train a high-precision shooter recognition model and obtain basketball video data based on user characteristics, after a professional specifies a court that is qualified for professional games, the 1080P high-definition camera is deployed at a fixed position on the designated court.
[0046] Collect basketball videos on the court, and have professional labelers use X-AnyLabeling to label the categories and location information of basketballs, baskets, backboards, players, and shooters that appear in the images as label data. The labeling standard is to discard video frames where no players appear, and the model does not need to learn the background information of fixed scenes. Of course, in order to further improve the training speed and effect, in some preferred embodiments, in video frames with players, video frames with less than 5 players are discarded, but if there is a shooter in the video frame, it can be directly selected. The shooter is a type of player, and the shooter has more specific characteristics. The selected shooter video frame should be a segment where the arm is raised to the top of the head or even exceeds the top of the head at the moment of shooting. The size and position of the labeling box need to be close to the target and facing the target. The target must be labeled even if it is blocked, and small targets also need to be labeled. Targets that are too blurred and have lost features do not need to be labeled.
[0047] S2: Object detector training.
[0048] After obtaining sufficient and rich basketball video data, the video data is divided and labeled according to the specified frame rate set by professionals, so as to obtain sufficient and highly distinguishable high-definition basketball video frames, effectively avoiding the problems of poor model recognition ability caused by the imbalanced distribution of data set categories during the deep learning model training process, and using the completed basketball training data set for the training of the target detector to obtain a high-precision and fast inference speed model for identifying shooters in basketball videos, which can be directly applied to the subsequent shooter detection process. The specific steps are as follows:
[0049] S21: Prepare training data, including images and corresponding labels, usually bounding boxes and their categories. The common format is COCO format. Perform data augmentation on the images, including random cropping, rotation, scaling, color transformation, etc., to increase data diversity and improve the robustness of the model.
[0050] S22: Scale the image to a fixed size to fit the network’s input requirements, normalize the pixel values to the [0,1] range, convert the relative coordinates of the bounding box and the one-hot encoding of the category label.
[0051] S23: The target detector is built based on YOLOv9. The image data is input to the GELAN network to extract multi-scale features. In order to alleviate the information bottleneck problem caused by the deepening of the model, programmable gradient information is added during the feature propagation. The main branch is used for the reasoning process, and the auxiliary reversible branch is used to solve the problems caused by the deepening of the neural network. Multi-level auxiliary information is used to deal with the error accumulation problem caused by deep supervision. Programmable gradient information is used to avoid feature loss caused by the network being too deep, and a generalized efficient layer aggregation network is used to achieve lightweight models, improve reasoning speed and detection accuracy. Bounding box regression and category prediction are performed based on feature maps, and feature maps of multiple scales are output, each scale corresponding to targets of different sizes;
[0052] S24: Calculate the position regression loss and size regression loss, use the cross entropy loss to calculate the classification loss of the target category, and calculate the confidence loss for distinguishing the foreground and background;
[0053] S25: Use BCE Loss to calculate the classification loss, use DFL Loss and CIoU Loss to calculate the regression loss, use Adam optimizer to update the model parameters, process one batch of data each time, update the model parameters once, and then use the learning rate scheduler to adjust the learning rate according to the training process to ensure model convergence and stability.
[0054] S3: Obtain a basketball game video and extract shooting clips from the basketball game video using the local optical flow method.
[0055] The local optical flow method is based on the assumption that in an image sequence, the brightness value of the same pixel in adjacent frames remains unchanged in a short period of time, that is, the brightness consistency assumption. The motion vector of the pixel is estimated by the brightness change in the local area, which is specifically expressed as:
[0056] f(x)=x T Ax+b T x+c
[0057] g(x)=f(x)+f(x)·d
[0058] E(d)=∫[g(x)-f(x+d)] 2 dx
[0059] Where f(x) represents the pixel brightness in the image at time t, and x is the coordinate vector (x, y) in two-dimensional space. T , A is a symmetric matrix that describes the second-order derivative information of the image in the area, b is a vector that represents the first-order derivative of the image, c is a constant that represents the average brightness of the area, g(x) represents the pixel brightness in the image at time t+Δt, f(x) is the gradient of the image f(x), d is the motion vector, i.e., optical flow, E(d) represents the error function, and the optical flow d can be obtained by minimizing the error function.
[0060] The game video is read frame by frame, and the region of interest in the video frame is converted into a grayscale image, where the region of interest refers to the position of the basket. The Farneback optical flow algorithm is used to calculate the optical flow vector of the pixel points between the two grayscale images. The modulus of the average optical flow vector in the basket area is calculated to determine whether the net swings. If the modulus of the average optical flow vector is greater than the modulus threshold, it is considered that the net swings. The moment when the net swings is the moment of shooting. After the moment of shooting is obtained, a preset number of video frames before and after the moment of shooting is intercepted to obtain a shooting clip. In this embodiment, 125 frames are selected before and after the moment of shooting to obtain a shooting clip, and the number of frames at the moment of shooting is saved as a txt file.
[0061] S4: Perform object detection on the captured shooting clips using the trained object detector.
[0062] The target detector is built on YOLOv9, which is a programmable gradient information and generalized efficient layer aggregation network. The programmable gradient information mainly includes three components, namely the main branch, the auxiliary reversible branch and the multi-level auxiliary information. The reasoning process of the programmable gradient information only uses the main branch, so no additional reasoning cost is required. The auxiliary reversible branch is designed to deal with the problems caused by the deepening of the neural network. The deepening of the network will lead to information bottlenecks, which will make the loss function unable to generate reliable gradients. Multi-level auxiliary information is used to deal with the error accumulation problem caused by deep supervision, which is particularly suitable for architectures with multiple prediction branches and lightweight models; the generalized efficient layer aggregation network combines two neural network architectures CSPNet and ELAN designed with gradient path planning. It is a neural network that takes into account lightweight, reasoning speed and accuracy.
[0063] More specifically, the object detector architecture is as follows:
[0064] In the object detector, it mainly includes the following three parts: Generalized ELAN (GELAN) as the backbone, Feature Pyramid Network (FPN) as the neck framework, and Programmable Gradient Information (PGI) as the information enhancement module. In the object detection algorithm, the backbone mainly determines the feature representation capability. At the same time, its design has a crucial impact on the reasoning efficiency because it bears a lot of computational costs; the neck is used to aggregate low-level physical features with high-level semantic features, and then establish pyramid feature maps at all levels; the information enhancement module is used to solve the problem of slow convergence or poor convergence of deep neural networks.
[0065] GELAN Backbone:
[0066] GELAN is a new lightweight, fast inference and high accuracy network architecture, which consists of CSPNet and ELAN with gradient path planning. CSPNet is a deep learning network structure, which is mainly used to improve the learning ability and efficiency of neural networks. CSPNet solves the problem of repeated calculations in the network by connecting parts across stages, and improves the accuracy and speed of the network while keeping the computational cost unchanged. The core idea of CSPNet is to divide the feature map into two parts, one part is directly passed to the next stage, and the other part is further convolved and then merged with the directly passed part, which can reduce repeated calculations and improve efficiency, while maintaining the diversity of features, thereby improving the performance of the network. The main structure of CSPNet includes splitting and fusion, cross-stage connection, and partial stacking. The ELAN network module is designed to improve the feature extraction ability and computational efficiency of the neural network. The ELAN module improves the performance of the model while keeping the network structure lightweight through an efficient layer aggregation method. The core idea of the ELAN module is to make full use of multi-scale information by aggregating features at different levels, thereby improving the feature representation ability. ELAN achieves efficient feature fusion through multiple parallel convolution and aggregation operations. The basic structure of the ELAN module includes multi-branch parallel convolution, layer aggregation, point-by-point convolution, and skip connection.
[0067] FPN neck network:
[0068] The feature pyramid network module is designed to efficiently fuse feature maps from different scales, thereby improving the accuracy and robustness of object detection. The feature pyramid network captures information of different scales in the image by extracting feature maps from different convolutional layers. These feature maps include high-resolution shallow features and low-resolution deep features. Shallow features contain more location information and details, while deep features contain more semantic information. In order to achieve multi-scale feature fusion, the feature pyramid network module contains upsampling and downsampling operations. Upsampling increases the low-resolution deep features to high resolution so that they are aligned with shallow features, and downsampling is used to reduce the size of high-resolution feature maps to align with low-resolution feature maps. Skip connections are key components of the feature pyramid network module. They directly transfer feature maps between different layers to ensure that information of different scales can be effectively fused. High-resolution feature maps from shallow layers can be fused with upsampled deep feature maps. Feature fusion is to merge feature maps from different sources by concatenation or weighted summation. The fused feature maps contain both shallow features with rich details and deep features with rich semantic information. This fusion method enables the network to better handle objects of various scales and improve the accuracy and robustness of detection. The feature pyramid network module usually outputs feature maps of multiple scales, which are sent to the detection head for object classification and bounding box regression.
[0069] PGI information enhancement module:
[0070] Programmable Gradient Information (PGI) is designed to solve the information loss problem encountered by deep neural networks during training. PGI enhances the training of the backbone network by providing additional gradient information, thereby improving the performance and accuracy of the model. PGI consists of a backbone network, auxiliary reversible branches, and multi-level auxiliary supervision. The backbone network of PGI is the core network for performing target detection tasks. The backbone network is GELAN. The auxiliary reversible branch is a key part of PGI. It works in parallel with the backbone network and can fuse shallow features into deep features through reversible operations to alleviate the information bottleneck problem. PGI introduces auxiliary supervision signals at different levels, such as feature pyramid network loss and path aggregation network loss. These auxiliary loss functions provide additional gradient information to help the model better learn multi-scale features. In PGI, data is forward propagated through the backbone network and auxiliary reversible branches at the same time to generate detection results and auxiliary supervision signals, calculate the main loss function and the auxiliary loss function, and obtain the total loss function. The gradient is calculated by back propagation. The gradient of the backbone network comes from the main loss function, while the gradient of the auxiliary reversible branch comes from the auxiliary loss function. The gradient of the auxiliary reversible branch is passed to the backbone network through a reversible operation to achieve gradient fusion and enhancement. The optimizer is used to update the parameters of the backbone network and the auxiliary reversible branch to complete an iteration.
[0071] Based on this, the target detection process of the target detector is as follows:
[0072] S41: Obtain the shooting clip, read the video frame data, and scale the video frame data to a fixed size (such as 640x640);
[0073] S42: Input the image data into the feature extraction network GELAN to obtain the downsampled features, and perform 8-fold downsampling, 16-fold downsampling, and 32-fold downsampling respectively to obtain the features H applicable to small, medium, and large target detection. m ;
[0074] S43: The obtained down-sampled feature H m Input into the feature pyramid for multi-scale feature fusion, construct feature maps of different resolutions, the model can detect targets of different sizes at the same time, fuse the high-resolution features of the low layer and the strong language features of the high layer, improve the detection accuracy of small targets, and obtain the fusion feature H′ m ;
[0075] S44: Fusion feature H′ m The input is sent to the regression layer to obtain the position of the center of the bounding box relative to the network unit where it is located, the size of the bounding box, and the probability that the target object exists in the predicted box. The high-level features of the feature map are input to the fully connected layer to obtain the category classification.
[0076] S5: Accelerating object detector inference by removing redundant video frames through frame skipping.
[0077] In a shooting clip, for the sake of viewing pleasure, extra video frames will be introduced after the player shoots and finishes shooting to ensure the integrity and smoothness of the video. These extra video frames are of no help in identifying the shooter, but will increase the time cost of the model. Therefore, video frames that are useless for identifying the shooter can be skipped. When the shooter is detected, the model reasoning is stopped in advance to avoid the detection and reasoning of subsequent useless frames, thereby achieving real-time detection.
[0078] The txt file recording the shooting and scoring moments of the shooting clip video is read to obtain the frame number of the shooting and scoring moment of the current shooting clip, which is recorded as F1. The existing shooting clip test data is summarized and statistically analyzed to obtain that the time from the shooter to the basketball touching the backboard is 1 to 2 seconds. Here, the maximum value is taken and converted to the number of frames. In this embodiment, the number of frames converted is 50, so the number of frames for frame skipping can be expressed as: F = F1-50. That is, based on the reverse order of the video frames of the shooting and scoring moment, the preset 50 frames of video frames are selected, and the selected video frames are input into the target detector in the forward order for target detection.
[0079] S6: Locate the shooting moment based on the target detector inference result combined with the local motion features, use the timing information to reversely traverse the result array from the shooting moment to find the shooter, find the shooter with the highest confidence based on the confidence, and find the shooter with the highest quality based on the intersection and union ratio, and finally get the shooter with the highest confidence and the best quality. When the shooter is detected, traverse 8 frames backward to get the shooter with the highest confidence and the best quality, set the detection flag to true, and when the detection flag is true, stop the program early.
[0080] S61: Save the target detection result of each frame in the result array X, and determine whether the basketball and the basket are in contact by calculating the contact area between the basketball and the basket. When the basketball and the basket are in contact, it is considered as a shooting moment, and the shooting moment video frame and the orientation information P of the basketball relative to the basket are recorded. b Since the video shooting angle is fixed, multiple areas can be obtained by pre-delineating the dividing line based on the basket, and the area where the basketball is located is used as the orientation information; for example, the basket is divided into two areas on the left and right by a line, and the basketball is located in the left area or the right area as the orientation information. Specifically expressed as:
[0081] L=(max(x1A,x1B)≤min(x2A,x2B))and(max(y1A,y1B)≤min(y2A,y2B))
[0082] Among them, L is the result of whether the basketball is in contact with the basket, where the bounding boxes of the basketball and the basket are A and B respectively, and their bounding boxes are represented by the coordinates of the upper left corner and the lower right corner A: (x1A, y1A, x2A, y2A), B: (x1B, y1B, x2B, y2B).
[0083] S62: When the basketball is in contact with the basket, it is considered that a shooting behavior may be generated, and the result array X is traversed in reverse order frame by frame from the moment when the shooting behavior is generated. Before the reverse traversal in reverse order frame by frame, a maximum threshold of the reverse traversal is set.
[0084] S63: When a video frame with a shooter is traversed, the shooter with the highest confidence is selected from the video frame, and the position information P of the shooter relative to the basket is calculated. s ; Determine the position information P of the shooter relative to the basket s The position information P of the basketball relative to the basket at the time of shooting b Whether they are the same, the position information of the shooter and the basketball can avoid the interference of other players outside the court. If not, continue to traverse the video frames in reverse order; if so, determine whether the confidence of the shooter is greater than the set shooter confidence threshold. If so, it is considered to be a valid shooter and the video frame is recorded; otherwise, continue to traverse the video frames in reverse order.
[0085] S64: Starting from the video frame with a valid shooter, traverse a preset number of frames in reverse order (for example, traverse 8 frames in reverse order continuously), find a video frame whose shooter confidence is greater than the shooter confidence threshold and whose intersection-over-union ratio between the shooter and the surrounding players is the lowest, and obtain the shooter with the highest confidence and the best quality. The lower the intersection-over-union ratio between the shooter and the surrounding players, the less foreground there is in the shooter image, and the foreground here refers to the players, and we believe that the quality of this shooter is higher.
[0086] S65: In the process of searching for the shooter by using the timing information to reversely traverse the result array from the moment of shooting, if the reverse traversal does not find a video frame greater than the shooter confidence threshold, that is, the shooter is not found, then the shooter confidence threshold is lowered according to the set attenuation coefficient, and steps S62-S64 are performed again to find the shooter; the iteration is repeated until the shooter is found or the shooter confidence threshold is lower than the shooter confidence minimum threshold.
[0087] The above-mentioned embodiment proposes a method for detecting shooters in fixed-view basketball game videos, which solves the problem of finding shooters in complex basketball playing scenes. The present invention uses a target detector, adopts pre-collected basketball game video data to train the target detector, and uses the target detector to automatically detect shooters; it solves the problems of low accuracy in identifying shooters and slow processing speed. The present invention uses temporal local motion features for rule judgment to achieve high accuracy in detecting shooters; it uses the local optical flow method to capture shooting clips, and uses frame skipping and early termination to accelerate the reasoning of the target detector, thereby achieving the effect of real-time detection.
[0088] Experimental dataset settings:
[0089] In order to fully verify the effectiveness, recognition accuracy and real-time performance of the shooter detection method in fixed-view basketball game videos, this experiment collected high-definition basketball video and image data from multiple stadiums, and integrated them into a unified data set for target detector training through the processing method described above.
[0090] In this experiment, more than 37,000 shooting data images were selected from many basketball videos as the data set for annotation. The annotated targets are basketball (basketball), basket (b), backboard (basket), player (person) and shooter (shoot). In terms of data set division, this experiment adopts an 8:1:1 ratio, that is, more than 37,000 images in the data set are randomly assigned, 80% of the images are used as training sets, 10% of the images are used as validation sets, and 10% of the images are used as test sets. In this experiment, an NVIDIA V100 graphics card is used to train the target detection model. Some parameters adopted in this experiment are shown in Table 1.
[0091] Table 1
[0092]
[0093]
[0094] Among them, setting lower learning and data enhancement methods can effectively constrain the training of the model and enhance the model's target detection effect.
[0095] Based on this experimental setting, the present invention conducts model training for the model, and after the training is completed, uses the existing video set data to test its average precision mean, shooter accuracy, and reasoning time (processing 10s shooting clips). The test results are shown in Table 2.
[0096] Table 2
[0097] Evaluation indicators Test Results Basketball average accuracy mean 0.94 Average basket accuracy 0.995 Average accuracy of players 0.992 Average accuracy of shooters 0.89 Shooter accuracy 91.47% Inference Time (GPU) 1.226s Inference Time (CPU) 2.903s
[0098] The experimental results show that this experiment has achieved relatively good results in mAP, shooter accuracy and inference time, proving that this target detection model can realize effective and real-time recognition of shooters.
[0099] The embodiment of the present invention further provides a system for detecting a shooter in a fixed-view basketball game video, comprising:
[0100] a memory having a computer program stored thereon;
[0101] A processor is used to load and execute the computer program to implement the method for detecting shooters in fixed-viewing angle basketball game videos as described above.
[0102] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method for detecting shooters in a fixed-viewing angle basketball game video as described above.
[0103] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.
[0104] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0105] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0106] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0108] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A method for detecting a shooter in a fixed-view basketball game video, characterized in that: The steps include: Obtain a basketball game video, and extract shooting clips from the basketball game video using the local optical flow method; Perform object detection on the captured shooting clips using the trained object detector; Accelerate object detector inference by removing redundant video frames through frame skipping; The shooting moment is located based on the target detector inference results combined with local motion features. The shooter is found by traversing the result array in reverse order using the timing information from the shooting moment. The shooter with the highest confidence is found based on the confidence level, and the shooter with the highest quality is found based on the intersection and union ratio. Finally, the shooter with the highest confidence and the best quality is obtained.
2. The method for detecting shooters in fixed-view basketball game videos according to claim 1, characterized in that: The process of capturing shooting segments from a basketball game video using the local optical flow method includes: Read the basketball game video frame by frame and convert the pre-defined basket area in the video frame into a grayscale image; The Farneback optical flow algorithm is used to calculate the optical flow vector of pixels between two adjacent grayscale images. The modulus of the average optical flow vector in the basket area is calculated to determine whether the net is swinging. If the modulus of the average optical flow vector is greater than the modulus threshold, it is considered that the net swings, and the moment when the net swings is the moment of shooting and scoring. A preset number of video frames before and after the moment of shooting and scoring are intercepted to obtain a shooting clip, and the frame number of the shooting and scoring moment of the shooting clip is saved.
3. The method for detecting shooters in fixed-view basketball game videos according to claim 1, characterized in that: The target detector is trained by the following method: Collect basketball game videos and annotate each frame in the basketball game videos to obtain a training sample data set; The training dataset is used to train the YOLOv9-based object detector to obtain the final object detector.
4. The method for detecting shooters in fixed-view basketball game videos according to claim 3, characterized in that: The training sample data set is obtained by the following method: Collect basketball game videos, remove video frames that do not contain players, and use the number of players to remove invalid video frames; Label the locations and categories of objects in the remaining video frames, including basketballs, backboards, hoops, players, and shooters. Data enhancement is performed on the labeled video frames, including random cropping, rotation, scaling, and color transformation, to obtain a training sample dataset.
5. The method for detecting shooters in fixed-view basketball game videos according to claim 1, characterized in that: The method of accelerating the object detector reasoning by removing redundant video frames by skipping frames includes: The frame number of the goal-scoring moment of the shooting clip is read, and the preset frame number of the video frame at the goal-scoring moment is skipped in reverse order based on the video frame at the goal-scoring moment, and the video frame to be detected is selected in forward order to the video frame at the goal-scoring moment, and the selected video frame is input into the target detector for target detection.
6. The method for detecting shooters in fixed-view basketball game videos according to claim 1, characterized in that: The method of locating the shooting moment based on the target detector inference result combined with the local motion feature specifically includes: The target detection results of each frame are saved in the result array X. The contact area between the basketball and the basket is calculated to determine whether the basketball and the basket are in contact. When the basketball and the basket are in contact, it is considered as the shooting moment. The shooting moment video frame and the position information P of the basketball relative to the basket are recorded. b .
7. The method for detecting shooters in fixed-view basketball game videos according to claim 6, characterized in that: The shooters with the highest confidence and the best quality are obtained through the following process; Traverse the result array X in reverse order frame by frame starting from the moment of shooting; When a video frame with a shooter is traversed, the shooter with the highest confidence is selected from the video frame, and the position information P of the shooter relative to the basket is calculated. s ; Determine the position information P of the shooter relative to the basket s The position information P of the basketball relative to the basket at the time of shooting b Are they the same? If not, continue to traverse the video frames in reverse order; if so, determine whether the confidence of the shooter is greater than the set shooter confidence threshold. If so, it is considered to be a valid shooter and the video frame is recorded; otherwise, continue to traverse the video frames in reverse order; Starting from the video frame with a valid shooter, the preset number of frames are traversed in reverse order to find the video frame whose shooter confidence is greater than the shooter confidence threshold and whose intersection-over-joint ratio between the shooter and the surrounding players is the lowest, so as to obtain the shooter with the highest confidence and the best quality.
8. The method for detecting a shooter in a fixed-view basketball game video according to any one of claims 1 to 7, characterized in that: In the process of searching for the shooter by using the timing information to reversely traverse the result array from the shooting moment, if the reverse traversal does not find a video frame greater than the shooter confidence threshold, that is, the shooter is not found, then the shooter confidence threshold is lowered according to the set attenuation coefficient, and the reverse traversal is performed again to search for the shooter; the iteration is repeated until the shooter is found or the shooter confidence threshold is lower than the shooter confidence minimum threshold.
9. A system for detecting shooters in fixed-view basketball game videos, characterized in that: include: a memory having a computer program stored thereon; A processor is used to load and execute the computer program to implement the method for detecting a shooter in a fixed-view basketball game video as described in any one of claims 1 to 8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program also implements the method for detecting shooters in fixed-view basketball game videos as described in any one of claims 1 to 8 when executed by the processor.
Citation Information
Patent Citations
Shooting detection method and device, electronic equipment and computer readable storage medium
CN111539294A
Intelligent basketball match analysis system, analysis method and equipment
CN112446319A
Video processing method and device, electronic equipment and readable storage medium
CN113286194A
Basketball goal detection method and system for rebroadcasting and court monitoring scene
CN113537168A
Method for acquiring position of auxiliary basketball shooter
CN115966019A
