Baseball good-ball automatic detection method based on YOLOv8

Abstract

The invention discloses a YOLOv8-based automatic detection method for a good baseball, and the method comprises the following steps: 1), obtaining an image data set, and carrying out the preprocessing of the image data set; 2) predetermining a good ball area detection model and a baseball detection model, and training and testing the models based on the image data set preprocessed in the step 1) for actual detection; 3) sequentially inputting the to-be-detected image to the good-ball area detection model and the baseball detection model, determining a good-ball area through the good-ball area detection model, and determining whether a baseball exists in the image and the position of the baseball through the baseball detection model; 4, whether any part of the baseball penetrates through the good-ball area or not is judged based on the good-ball area and the baseball position determined in the step 3, if yes, the baseball is good once, and if not, the baseball is bad. The good-ball area can be updated in real time based on movement of the batter, good-ball judgment is automatically achieved, and judgment is more accurate and objective.

Description

Technical Field

[0001] The present invention relates to baseball strike zone detection, and in particular to a baseball strike automatic detection method based on YOLOv8, belonging to the technical field of image detection. Background Art

[0002] Sports have always been a popular form of entertainment, and baseball is no exception. Especially in the United States, baseball games are often talked about by Americans of all ages, whether children or adults. In addition, baseball is becoming more popular every year. According to reports, the total number of viewers for the 2023 Major League Baseball (MLB) regular season was 70,747,365, an increase of 9.6% from last year's 64,556,636. The average number of viewers per game reached 29,295, an increase of 9.1% year-on-year, the highest increase in nearly 30 years (excluding the 2020 season). This is the first increase in the number of viewers since the expansion of Major League Baseball to 28 clubs in 1993.

[0003] Baseball is a sport played by two teams, each consisting of nine players. Figure 1 As shown in (a), a baseball game is played on a diamond-shaped field with four bases at the corners. The goal of the offensive team is to hit the ball thrown by the pitcher and score runs, while the goal of the defensive team is to prevent this by getting the offensive players out.

[0004] In baseball, the pitcher and the catcher are crucial to the team. The pitcher needs to throw a good ball, while the batter needs to judge whether the pitched baseball is a good ball. A fundamental aspect of baseball is the concept of the strike zone. According to the official rules, the strike zone is the area above the home plate, with its upper edge being a horizontal line at the elbow and its lower edge being a horizontal line below the kneecap, such as Figure 1 (b) As the pitcher prepares to pitch, the strike zone will be determined based on the batter's posture. If the pitched baseball overlaps with the strike zone, it is considered a strike. An important difficulty is that, except for MLB and American television broadcasts, which draw a rectangular box on the video frame to indicate the strike zone to assist in judgment, other regions rely heavily on the judgment of the umpire, and the umpire may make a misjudgment when judging whether the incoming ball passes through the strike zone. Since the movement of the batter will cause different strike zone detection results for each frame, and the strike zone in the MLB broadcast video image is fixed for each batter's batting, this also leads to some problems in the identification of the strike zone itself. In addition, there is no replay to determine whether the pitched baseball is a strike or a ball. Strike is essential in baseball because it is directly related to scoring. If the umpire mistakenly calls a strike and causes the batter to be out, it will affect the direction of the game. Therefore, these mistakes can determine the winning trend of the two teams, which is unfair to the losing side.

[0005] With the development of artificial intelligence (AI), target detection and tracking have been used to analyze sports events. For example, Hawkeye is a computer vision (CV) system widely used in many sports such as tennis, badminton, and volleyball to track the trajectory of the ball. The system visualizes the trajectory of the ball and determines whether the ball is out of bounds. However, since the strike zone in baseball is determined based on the batter's posture rather than fixed field markings, the Hawkeye system in traditional sports games cannot be used for automatic strike zone detection in baseball. Summary of the invention

[0006] In view of the shortcomings of the prior art that the strike zone in the video image is fixed for each batter's hit or the referee's judgment, the purpose of the present invention is to provide a baseball strike automatic detection method based on YOLOv8. The present invention can update the strike zone in real time based on the movement of the batter and automatically realize the judgment of the strike, and the judgment is more accurate and objective.

[0007] The technical solution of the present invention is achieved in this way:

[0008] The automatic detection method of baseball strike based on YOLOv8 is as follows:

[0009] 1) Obtain image data set and preprocess it;

[0010] 2) predetermining a strike zone detection model and a baseball detection model, and training and testing the models based on the image data set preprocessed in step 1) for actual detection; wherein the strike zone detection model is used to detect and determine the strike zone in the image, and the baseball detection model is used to detect whether there is a baseball and the position of the baseball in the image;

[0011] 3) The image to be detected is sequentially input into the strike zone detection model and the baseball detection model, the strike zone is determined by the strike zone detection model, and the baseball detection model is used to determine whether there is a baseball in the image and the location of the baseball;

[0012] 4) Based on the strike zone and the position of the baseball determined in step 3), determine whether any part of the baseball passes through the strike zone. If so, it is a strike; otherwise, it is a ball.

[0013] The strike zone detection model calculates and determines the strike zone by finding the positions of the home base, elbows and knees in the image and based on the detection results of the three positions.

[0014] Specifically, the strike zone detection model uses the left and right boundaries of the home base to determine the left and right boundaries of the strike zone, and uses the detected elbow and knee information to determine the upper and lower boundaries of the strike zone. The rectangular area enclosed by the left and right boundaries and the upper and lower boundaries is the strike zone; the coordinates of the upper left corner and the upper right corner of the strike zone are respectively expressed as ((X SL ,YSU ) and (X SR ,Y SU ), the coordinates of the lower left corner and the lower right corner of the strike zone are expressed as (X SL ,Y SL ) and ((X SR ,Y SL ).

[0015] Furthermore, after the baseball detection model detects the baseball, the position of the baseball in the image is obtained and represented by a rectangular box circumscribing the baseball. The coordinates of the upper left corner, upper right corner, lower left corner and lower right corner of the rectangular box are respectively represented as (X BL ,Y BU )、(X BR ,Y BU )、(X BL ,Y BL ) and (X BR ,Y BL ); if the following formula is satisfied, it is a good ball, otherwise it is a bad ball;

[0016]

[0017] This completes the judgment of a good ball.

[0018] Furthermore, the image to be detected is a real-time image of baseball in motion acquired by a camera device, the camera device is a fixed position and installed in front of home plate; the camera device is connected to a detection system, and the strike zone detection model and baseball detection model are deployed in the detection system.

[0019] Furthermore, the strike zone detection model and the baseball detection model are both built based on YOLOv8.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention introduces an automatic strike detection method for baseball by developing YOLOv8. The detection method of the present invention includes two models: (1) a strike zone detection model, namely YOLOv8 for Strikes Zone, referred to as YOLOv8-SZ, which is used to detect the strike zone formed by the elbow, knee and home plate; (2) a baseball detection model, namely YOLOv8 for Baseball, referred to as YOLOv8-B, which is used to detect baseballs and determine whether the baseball has passed through the strike zone. These two models are trained on the strike zone and baseball image datasets established on TV broadcasts and the Internet using the pre-trained YOLOv8m model. After the training and testing process, the test results on the self-built dataset and MLB TV live video show that the present invention can realize real-time recognition of the strike zone and baseball, with an average I OU is 0.71, which achieves good detection performance. The experimental results verify the effectiveness of the method in detecting the strike zone of baseball. The present invention can update the strike zone in real time based on the movement of the batter and automatically realize the judgment of the strike, and the judgment is more accurate and objective.

[0022] The YOLOv8 adopted in the present invention focuses on maintaining the best balance between accuracy and speed, and is suitable for real-time target detection tasks in various application fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 -Baseball rules diagram.

[0024] Figure 2 -The present invention is based on the good ball detection flow chart of Yolov8.

[0025] Figure 3 -Example diagram of annotated images.

[0026] Figure 4 -Schematic diagram of the structure of the YOLOv8 model.

[0027] Figure 5 -Graphs of the training and validation process.

[0028] Figure 6 - Illustration of the strike zone and baseball detection results.

[0029] Figure 7 -Visualization result diagram of YOLOv8-SZ and YOLOv8-B in the embodiment.

[0030] Figure 8 -Visualization of detection results on three videos.

[0031] Fig. 9 - Example detection result visualization diagram (strike zone and pitched baseball). DETAILED DESCRIPTION

[0032] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0033] This paper proposes a baseball strike automatic detection method based on YOLOv8, which completes the corresponding detection task by detecting the batter's home plate, elbow and knee. The detection basis is the construction of the model, and the model construction process includes data collection and preprocessing, model selection, model training and model testing. Figure 2 shown.

[0034] 1 Image collection and preprocessing

[0035] In baseball TV broadcasts, there is often a fixed camera in front of home plate to capture the batter's posture to help experts interpret the game and determine the strike zone, thereby obtaining a large number of historical images. In the image acquisition process, the present invention collected 1,267 such images from TV broadcasts and the Internet, and used LabelImg software to label important parts such as home plate, elbows, knees, and baseballs. Figure 3 The annotated images are randomly divided into training and test sets at a ratio of nearly 9:1, with 1147 image samples used as training sets and 120 samples used for testing.

[0036] 2Strike Zone Detection Based on YOLOv8

[0037] The YOLO series has been widely used in the CV field. They have extremely high accuracy while maintaining a small model size. In the YOLO series, YOLOv8 provides several improvements, anchor-free structure, multi-scale prediction, and improved backbone network, which are faster and more accurate in detecting small targets. YOLOv8 is superior to YOLOv7 in both speed and accuracy, so the present invention selects YOLOv8 for baseball strike zone detection.

[0038] 2.1YOLOv8 Architecture

[0039] like Figure 4 As shown in the figure, the YOLOv8 model consists of four main parts: input, backbone network (Backbone), neck, and head. The above four parts are introduced as follows:

[0040] (1) The input part is responsible for preprocessing the image, including data augmentation and scaling, to provide the best possible input for the model.

[0041] (2) The backbone of YOLOv8 is represented by CBS, SPPF, and C2f modules, and the performance of its feature extraction function is the main feature of YOLOv8. Specifically, CBS uses Conv, BN, and SiLU activation functions, SPPF uses a fast connection of three 5×5 max pools, and C2f uses a combined gradient transfer method to combine CBS with a residual module, which enhances both the model's learning and feature extraction.

[0042] (3) The neck part uses the FPN+PAN architecture to aggregate multi-scale features through a top-down path and a bottom-up path, effectively improving their ability to recognize objects of different scales. For more complex images, this structure improves the accuracy of image detection.

[0043] (4) Backgrounds and objects of different sizes, such as complex scenes containing a wider area and more objects.

[0044] (5) The head part consists of multiple detection heads, each of which explores the predicted bounding box, category probability, and corresponding scores at various scales. The detection head adopts an improved version of the YOLO head, which attempts to combine dynamic anchor point allocation and IoU loss function.

[0045] Due to the above-mentioned efficient components, the YOLOv8-based object detector of the present invention is characterized by considerable improvements in accuracy and performance of the baseball strike zone detection task.

[0046] 2.2YOLOv8 Model Selection

[0047] As shown in Table 1, YOLOv8 provides multiple variants, such as YOLOv8-n, YOLOv8-s, YOLOv8-m, YOLOv8-l, and YOLOv8-x. These variants provide different options, allowing users to choose the model that best suits their computing resources and application requirements.

[0048] Table I Comparison of different YOLOv8 models

[0049]

[0050] Based on the application requirements of baseball strike zone detection, the present invention selects YOLOv8m on the basis of balancing detection performance and efficiency.

[0051] 2.3 Model Training and Validation

[0052] YOLOv8 adopts inclusive training strategies to improve performance and explores multiple training resolutions and mosaic data augmentation. These training strategies enhance generalization capabilities in different scenarios.

[0053] For the proposed method, two object detection models need to be trained, one is YOLOv8 for strike zone detection (YOLOv8-SZ), and the other is YOLOv8 for baseball detection (YOLOv8-B). For YOLOv8-SZ, the present invention divides 1267 images into training set and test set at a ratio of nearly 9:1, of which 1147 images are used as training set and 120 images are used as test set. For YOLOv8-B, 346 images are divided into training set and test set at a ratio of 9:1, 311 images are used for training set and 35 images are used for testing.

[0054] The above two models are trained for 300 epochs, and the evaluation indicators of the training dataset (box_loss, cls_loss, dfl_loss, precision, recall) and the validation dataset (box_loss, cls_loss, dfl_loss, mAP50, mAP50-95) are calculated. It is worth noting that the YOLOv8 pre-trained model is explored for training. The training and validation curves are shown in Figure 2. Figure 5 As shown, Figure 5 (a) corresponds to the model YOLOv8-SZ, Figure 5 (b) corresponds to the model YOLOv8-B. Figure 5 It can be seen that the training process of YOLOv8-SZ and YOLOv8-B is effective, and they have higher accuracy and faster convergence speed. After using YOLOv8 to train the object detection model, the fine-tuned YOLOv8-SZ and YOLOv8-B models can be used for good ball detection.

[0055] 3Strike zone, baseball detection and strike judgment

[0056] After the training process is completed, YOLOv8-SZ is used to find the location of home base, elbows, and knees, and the strike zone is calculated based on the detection results, while YOLOv8-B is used to detect the baseball; finally, whether the baseball passes through the strike zone is determined based on whether there is overlap between the two.

[0057] like Figure 6 As shown in (a), the YOLOv8-SZ model is used to detect the home base, and the left and right boundaries of the strike zone are determined using the left and right boundaries of the home base. Then, the upper and lower boundaries of the strike zone are determined using the detected elbow and knee information. The top of the strike zone is defined by the lower edge of the player's elbow, and the bottom is defined by the upper boundary of the batter's knee. In this way, the strike zone is obtained, and the coordinates of the upper left and upper right corners of the strike zone are represented as ((X SL ,Y SU ) and (X SR ,Y SU ), the coordinates of the lower left corner and the lower right corner of the strike zone are expressed as (X SL ,Y SL ) and ((X SR, Y SL ). Based on the calculated strike zone, YOLOv8-B tries to capture a baseball in the video. After detecting the baseball, the position of the baseball in the image is obtained and represented by a rectangular box circumscribing the baseball. The coordinates of the upper left corner, upper right corner, lower left corner, and lower right corner of the rectangular box are represented as (X BL ,Y BU )、(X BR ,Y BU )、(X BL ,YBL ) and (X BR ,Y BL ),like Figure 6 (b) The official rules state that if any part of the ball passes through the strike zone, it is a strike. To be a strike, the following formula must be met:

[0058]

[0059] Finally, the detection of the strike zone and the judgment of the strike are completed.

[0060] The following examples are used to verify and explore the test results of the present invention.

[0061] 1) Hardware and software environment: The hardware platform for all experiments includes 24GB memory, Intel Core i7-7800 CPU and NVIDIA TITANV GPU. The software environment is Python 3.7 and TensorFlow executed on Windows 10 system to implement the proposed YOLOv8-SZ and YOLOv8-B models.

[0062] 2) Evaluation indicators:

[0063] The following measurements are applied as performance indicators to the detection results on different images, including mean intersection over union (mIoU), mean average precision with an IoU threshold of 0.5 (mAP50), and mean average precision with IoU thresholds from 0.5 to 0.95 (mAP50-95). The corresponding indicators are defined as follows:

[0064]

[0065] Among them, AP k It represents the average precision (AP) value of the kth class, and n represents the number of classes.

[0066]

[0067] Among them, TP (True Positive) represents the number of positive samples correctly predicted by the algorithm, and FP (False Positive) represents the number of positive samples incorrectly predicted by the model.

[0068] 3) Parameter setting: The model is trained on a self-constructed baseball image dataset, and 1267 images are collected and labeled to form a strike zone image dataset and 346 images to construct a baseball image dataset. The models YOLOv8-SZ and YOLOv8-B are trained with a training set to test set ratio of 9:1. In addition, in this embodiment, the hyperparameters are set to a learning rate of 0.001, a mini-batch size of 64, a momentum of 0.9, a weight decay of 0.0005, and an epoch of 300.

[0069] Results and Discussion

[0070] In order to evaluate the performance of strike zone and baseball detection, YOLOv8-SZ and YOLOv8-B were tested on the test sets of strike zone image dataset and baseball image dataset. The evaluation indicators of the two models on the two image datasets are shown in Table II, and the corresponding visualization results are shown in Figure 7 As shown. Among them, Figure 7 (a) corresponds to the model YOLOv8-SZ, Figure 7 (b) corresponds to the model YOLOv8-B.

[0071] Table II Evaluation indicators of YOLOv8-SZ and YOLOv8-B on two image datasets

[0072]

[0073] It can be seen from Table II that the YOLOv8-SZ and YOLOv8-B models proposed in this invention have achieved good performance results on the mAP50 and mAP50-95 datasets, with average mAP50 and average mAP50-95 being 0.78 and 0.43 respectively. Figure 7 The visualization results show that the model achieves good detection results. In addition, the running time of both models is very fast and can be used for real-time detection tasks.

[0074] In addition, the effectiveness of the proposed strike zone detection method is verified on broadcast television videos. Since the television broadcast of Major League Baseball has a rectangle to represent the strike zone drawn by experts on each frame of the video, this embodiment compares the I between the strike zone drawn by MLB and the strike zone detected by the YOLOv8-SZ model proposed in the present invention. O U metric. We selected three MLB videos with strike zones, marked the strike zones in each frame, and calculated I O U. I of three videos O The U metric and detection results are visualized in Table III and Figure 8 .

[0075] Table III IoU results of the models on three MLB broadcast videos

[0076]

[0077]

[0078] According to the results in Table III, the strike zones detected by this method perform well compared with the strike zones marked by MLB experts, with an average I OU is 0.71. The strike zone detected by the method proposed in the present invention is highly consistent with the strike zone drawn by MLB experts. Figure 8 However, there are still some problems. First, MLB experts may make mistakes and mark some strike zones too low, such as Figure 8 (a). In addition, due to the limited labeled samples, the proposed detection model needs to be improved. In addition, the features of the knee are not clear enough, which will affect the accuracy of the strike zone area, such as Figure 8 (b). Finally, the batsman raises his knees high, as shown in Figure 8 As shown in (e), the detection area is relatively small, indicating that the detection result is unreasonable.

[0079] In order to further improve the performance of the proposed model, the present invention conducted more tests on a video of a complete batting action in an MLB game. The results are as follows: Fig. 9 As shown. As can be seen from the figure, the method proposed by the present invention has achieved good results in detecting the strike zone and the thrown baseball, and can effectively determine whether the ball is a good ball or a bad ball. However, there are still some differences in the strike zone detected by this method in different frames of the video. The main reason is that each strike zone detected by the experimental method is obtained by real-time detection of each frame of the video. The movement of the batter in the video will cause different strike zone detection results for each frame, while the strike zone in the MLB broadcast video image is fixed for each batter's hit.

[0080] Finally, it should be noted that the above examples of the present invention are merely examples for illustrating the present invention, and are not intended to limit the embodiments of the present invention. Although the applicant has described the present invention in detail with reference to the preferred embodiments, for those of ordinary skill in the art, other different forms of changes and modifications can be made based on the above description. It is impossible to list all the embodiments here. Any obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A baseball strike automatic detection method based on YOLOv8, characterized by: The steps are as follows, 1) Obtain image data set and preprocess it; 2) predetermining a strike zone detection model and a baseball detection model, and training and testing the models based on the image data set preprocessed in step 1) for actual detection; wherein the strike zone detection model is used to detect and determine the strike zone in the image, and the baseball detection model is used to detect whether there is a baseball and the position of the baseball in the image; 3) The image to be detected is sequentially input into the strike zone detection model and the baseball detection model, the strike zone is determined by the strike zone detection model, and the baseball detection model is used to determine whether there is a baseball in the image and the location of the baseball; 4) Based on the strike zone and the position of the baseball determined in step 3), determine whether any part of the baseball passes through the strike zone. If so, it is a strike; otherwise, it is a ball.

2. The baseball strike automatic detection method based on YOLOv8 according to claim 1, characterized in that: The strike zone detection model finds the positions of the home base, elbows and knees in the image and calculates and determines the strike zone based on the detection results of the three positions.

3. The baseball strike automatic detection method based on YOLOv8 according to claim 2, characterized in that: The strike zone detection model uses the left and right boundaries of the home plate to determine the left and right boundaries of the strike zone, and uses the detected elbow and knee information to determine the upper and lower boundaries of the strike zone. The rectangular area enclosed by the left and right boundaries and the upper and lower boundaries is the strike zone; the coordinates of the upper left corner and the upper right corner of the strike zone are respectively expressed as ((X SL ,Y SU ) and (X SR ,Y SU ), the coordinates of the lower left corner and the lower right corner of the strike zone are expressed as (X SL ,Y SL ) and ((X SR ,Y SL ).

4. The baseball strike automatic detection method based on YOLOv8 according to claim 1, characterized in that: After the baseball detection model detects the baseball, the position of the baseball in the image is obtained and represented by a rectangular box circumscribing the baseball. The coordinates of the upper left corner, upper right corner, lower left corner and lower right corner of the rectangular box are respectively represented as (X BL ,Y BU )、(X BR ,Y BU )、(X BL ,Y BL ) and (X BR ,Y BL ); if the following formula is satisfied, it is a good ball, otherwise it is a bad ball; This completes the judgment of a good ball.

5. The baseball strike automatic detection method based on YOLOv8 according to claim 1, characterized in that: The image to be detected is a real-time image of baseball in motion obtained by a camera device, which is a fixed camera installed in front of the home plate; the camera device is connected to a detection system, and the strike zone detection model and the baseball detection model are deployed in the detection system.

6. The automatic baseball strike detection method based on YOLOv8 according to claim 1, characterized in that: The strike zone detection model and the baseball detection model are both built based on YOLOv8.

Images