A method and device for detecting violations in badminton serve height.

Abstract

This application provides a method and apparatus for detecting violations in badminton serve height. First, a video of the target serve is acquired by a camera at the standard serve height. A vanishing line detection algorithm is used to solve for the horizon line in the video, where the horizon line is the baseline corresponding to the standard serve height in the video. Then, the video is input into a preset serve state detection model for feature extraction and prediction to obtain the probability that the video is in a serve state, which is compared with a preset threshold to determine whether the video is in a serve state. When the video is in a serve state, a preset target detection model is used to determine the position of the shuttlecock. Finally, the position of the shuttlecock is compared with the horizon line to determine whether the serve is illegal. This method can objectively and automatically identify violations in badminton serve height at the moment of serve, improving the accuracy of serve height violation detection. Furthermore, the algorithm has a simple structure and low operating cost.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a method and apparatus for detecting violations in badminton serve height. Background Technology

[0002] In 2018, the Badminton World Federation (BWF) changed the badminton service rules: the height of the shuttlecock during the serve was strictly limited to no more than 1.15 meters, instead of the previous requirement of not exceeding waist height. In current official matches, referees use an auxiliary stick to determine whether the shuttlecock has crossed the line during the serve. The stick has two panes of glass, each with a horizontal black line engraved on it, each line 1.15 meters above the ground. The referee adjusts their viewing angle to align these two lines within their field of vision, then checks whether the shuttlecock's height exceeds this line; if it does, it's a violation. Because this method requires referees to observe the player's serve for extended periods, fatigue can easily occur, increasing the probability of misjudgments and missed calls.

[0003] Hawk-Eye, also known as a real-time replay system, consists of eight or ten high-speed cameras, four computers, and a large screen. The system first uses computer calculations to divide the three-dimensional space of the playing area into measurement units measured in millimeters. Then, high-speed cameras simultaneously capture basic data of the ball's trajectory from different angles. The computer then uses these data to generate a three-dimensional image. Finally, real-time imaging technology displays the ball's trajectory and landing point on a large screen. While this system can detect the ball's trajectory, it does not have the ability to identify player behavior. Therefore, it is mainly used to determine whether the ball lands out of bounds and cannot detect whether a player's serve is illegal. Furthermore, the Hawk-Eye system has a large amount of hardware, is complex, and expensive, limiting its widespread adoption. Therefore, there is an urgent need for a lightweight, objective detection technology to detect violations in badminton serve height. Summary of the Invention

[0004] This application provides a method and device for detecting violations in badminton serve height. It can objectively and automatically identify violations in serve height at the moment of serve, improve the accuracy of serve height violation judgment, and has a simple algorithm structure and low operating cost.

[0005] The first aspect of this application provides a method for detecting violations in badminton serve height, the method comprising:

[0006] The video to be tested is captured by the camera at the standard serve height. The horizon line in the video to be tested is solved by the vanishing line detection algorithm. The horizon line is the baseline corresponding to the standard serve height.

[0007] The above-mentioned video to be detected is input into a preset serve state detection model for feature extraction and prediction. The probability of the video to be detected being in the serve state is obtained and compared with a preset threshold to determine whether the video to be detected is in the serve state.

[0008] When the video to be detected is in the serving state, the position of the badminton shuttlecock is determined using a preset target detection model;

[0009] Compare the position of the shuttlecock with the aforementioned horizon line to determine whether the serve was illegal.

[0010] Optionally, the above-mentioned method of using the vanishing line detection algorithm to solve for the horizon line in the video to be detected includes:

[0011] Select the image to be detected from the above video to be detected, and detect all line segments in the image to be detected;

[0012] The line segments are filtered based on the angle between the line segment and the horizontal line, and the longest line segments are selected from the remaining line segments.

[0013] Extend the selected longest line segments and calculate the intersection point of the extensions of any two line segments to obtain the set of intersection points;

[0014] Cluster analysis is performed on the set of intersection points to obtain the cluster containing the most intersection points, and the corresponding set of line segments is determined.

[0015] The coordinates of the vanishing point can be obtained by solving the set of line segments corresponding to the cluster with the most intersections.

[0016] Draw a horizontal line through the vanishing point to obtain the horizon line.

[0017] Optionally, the above-mentioned selection of the image to be detected from the video to be detected, and detection of all line segments in the image to be detected, includes:

[0018] The images to be detected are selected from the above-mentioned videos and then subjected to image enhancement and region segmentation processing.

[0019] Edge detection is performed on the processed image to be detected to obtain the edge detection results;

[0020] A Hough transform is applied to the edge detection results to detect all line segments in the image to be detected.

[0021] Optionally, the line segments are filtered based on the angle between the line segment and the horizontal line, and the longest line segments are selected from the remaining line segments, including:

[0022] Calculate the angle between the line segment and the horizontal line, and delete the line segments with an angle less than a preset first threshold and the line segments with an angle greater than a preset second threshold;

[0023] Calculate the length of each of the remaining line segments, sort them in descending order of length, and select the first few line segments.

[0024] Optionally, the above method further includes:

[0025] A serve state detection model is constructed, which includes multiple convolutional layers, max pooling layers, fully connected layers, and an output layer.

[0026] Multiple sample videos were acquired. The recording parameters and height of the sample videos were the same as those of the video to be detected. The sample videos included positive and negative samples carrying the corresponding serve status labels.

[0027] Based on the above multiple sample videos, the above serve state detection model is trained, including: inputting each sample video into the above serve state detection model, extracting shallow and deep features of the above sample video through multiple above convolutional layers and max pooling layers, performing feature fusion through the above fully connected layer, and predicting the probability of the sample video being in serve state based on the feature fusion result through the above output layer to obtain a predicted value.

[0028] Based on the serve status labels and predicted values ​​of the sample videos, the cross-entropy loss function of the above serve status detection model is calculated, and the model parameters in the serve status detection model are updated.

[0029] When the cross-entropy loss function is less than the preset threshold, the update stops, the model training is completed, and the above-mentioned preset serve state detection model is obtained.

[0030] Optionally, the above method further includes:

[0031] A target detection model is constructed, which includes a feature extraction module and multiple attribute detection branches, each of which is connected to the feature extraction module.

[0032] Multiple sample images are acquired, with the same acquisition parameters and height as the video to be detected, and each sample image carries multiple attribute labels.

[0033] Based on the above multiple sample images, the above target detection model is trained, including: inputting each sample image into the above target detection model, extracting features from the sample image through the above feature extraction module to obtain the image features of the sample image, and detecting different attributes based on the image features through the attribute detection branch to obtain the corresponding attribute prediction results;

[0034] Based on the attribute labels of the sample images and the corresponding attribute prediction results, the loss function of the above object detection model is calculated, and the model parameters in the object detection model are updated.

[0035] When the loss function is less than the preset threshold, the update stops, the model training is completed, and the above-mentioned preset target detection model is obtained.

[0036] Optionally, the above attribute detection branch includes:

[0037] The badminton shuttlecock center point detection branch is used to generate a heatmap, which represents the probability that a pixel in the sample image belongs to the center point of the badminton shuttlecock, thus determining the coordinates of the center point. The loss function L of the badminton shuttlecock center point detection branch is... hm for:

[0038]

[0039] Where (x,y) represents the coordinates of a pixel in the heatmap, Y... x,y This represents the true probability that the point (x, y) is the center point of the badminton shuttlecock. The predicted value represents the probability that point (x,y) is the center point of a badminton shuttlecock. N represents the number of badminton shuttlecocks in the input sample image. α and β are two hyperparameters, where α = 2 and β = 4.

[0040] The badminton shuttlecock bounding box detection branch is used to regress the predicted bounding box size of the shuttlecock based on the determined coordinates of the shuttlecock's center point. The loss function L of the above badminton shuttlecock bounding box detection branch is... size for:

[0041]

[0042] Where k represents the center point of the badminton shuttlecock, s k This represents the true value of the bounding box corresponding to the center point of the badminton shuttlecock. This represents the predicted value of the bounding box corresponding to the center point of the badminton shuttlecock, where N represents the number of badminton shuttlecocks in the input sample image.

[0043] The center point offset detection branch is used to regress the predicted offset of the badminton shuttlecock's center point based on the determined coordinates of the shuttlecock's center point. The loss function L of the above center point offset detection branch is... off for:

[0044]

[0045] in, Indicates the center point of the badminton shuttlecock. This represents the actual offset of the center point of the badminton shuttlecock. This represents the predicted offset value of the center point of the badminton shuttlecock, and N represents the number of badminton shuttlecocks in the input sample image.

[0046] Optionally, the loss function for calculating the target detection model includes:

[0047] Based on the loss function L of the badminton center point detection branch mentioned above hm Loss function L for badminton bounding box detection branch size and the loss function L of the center point offset detection branch off Calculate the overall loss function L of the above object detection model:

[0048] L = L hm +λ size L size +λ off L off

[0049] Where, λ size and λ off There are two hyperparameters, λ size =0.1, λ off =1.

[0050] Optionally, in the above badminton center point detection branch, the true value Y of the probability that point (x,y) is the center point of the badminton shuttlecock. x,y The calculations include:

[0051] The input sample image is downsampled to obtain the coordinates of the center point of the badminton shuttlecock.

[0052] Where c represents the coordinates of the center point of the badminton shuttlecock in the sample image, and R represents the downsampling factor. Indicates rounding down;

[0053] By using a Gaussian kernel to map the sampled shuttlecock center point coordinates onto a heatmap, the true probability Y of point (x,y) being the shuttlecock center point is obtained. x,y :

[0054]

[0055] Where, σ p It is a standard deviation that adapts to the size of an object. The x-coordinate of the center point of the badminton shuttlecock. The vertical coordinate represents the center point of the badminton shuttlecock.

[0056] A second aspect of this application provides a device for detecting violations of badminton serve height, the device comprising:

[0057] The horizon line generation module is used to acquire the video to be tested captured by the camera at the standard serve height, and to solve the horizon line in the video to be tested using the vanishing line detection algorithm. The horizon line is the baseline corresponding to the standard serve height.

[0058] The serve state detection module is used to input the above-mentioned video to be detected into a preset serve state detection model for feature extraction and prediction, obtain the probability that the video to be detected is in the serve state and compare it with a preset threshold to determine whether the video to be detected is in the serve state.

[0059] The serve height detection module is used to determine the position of the badminton shuttlecock using a preset target detection model when the video to be detected is in the serve state.

[0060] The service violation judgment module is used to compare the position of the badminton shuttlecock with the aforementioned horizon line to determine whether the service is illegal.

[0061] Compared with the prior art, this application has the following advantages:

[0062] This application provides a method and apparatus for detecting violations in badminton serve height. First, a video of the target serve is acquired by a camera at the standard serve height. A vanishing line detection algorithm is used to solve for the horizon line in the video, where the horizon line is the baseline corresponding to the standard serve height in the video. Then, the video is input into a preset serve state detection model for feature extraction and prediction, obtaining the probability that the video is in a serve state and comparing it with a preset threshold to determine whether the video is in a serve state. When the video is in a serve state, a preset target detection model is used to determine the position of the shuttlecock. Finally, the position of the shuttlecock is compared with the horizon line to determine whether the serve is illegal. Therefore, this application can objectively and automatically identify violations in badminton serve height at the moment of the serve, improving the accuracy of serve height violation judgment and avoiding misjudgments caused by manual review. Furthermore, compared with existing Hawk-Eye systems, the algorithm structure of this application is simpler, the operating cost is lower, and it is conducive to widespread application. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart of a method for detecting violations of badminton serve height according to an embodiment of this application;

[0065] Figure 2 This is a flowchart of a vanishing line detection algorithm proposed in an embodiment of this application;

[0066] Figure 3This is a schematic diagram of the intermediate process of solving the horizon line in the video to be detected using the vanishing line detection algorithm proposed in an embodiment of this application;

[0067] Figure 4 This is a flowchart illustrating the training process of a serve state detection model proposed in one embodiment of this application.

[0068] Figure 5 This is a schematic diagram of the structure of a serve state detection model proposed in an embodiment of this application;

[0069] Figure 6 This is a flowchart of the training process of an object detection model proposed in an embodiment of this application;

[0070] Figure 7 This is a schematic diagram of the target detection model proposed in one embodiment of this application;

[0071] Figure 8 This is a schematic diagram of a badminton serve height violation detection device according to an embodiment of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] In 2018, the Badminton World Federation (BWF) changed the badminton service rules: the height of the shuttlecock during the serve was strictly limited to no more than 1.15 meters, instead of being below waist height. In current official matches, referees still rely on prolonged observation of players' serves to manually determine whether they violate the rules, which increases the probability of misjudgments and missed calls.

[0074] In major badminton competitions, the Hawk-Eye system is a commonly used technology to assist in scoring. Its principle involves multiple high-definition cameras capturing images, which are then processed by multiple high-performance computers to reconstruct the trajectory of the shuttlecock using virtual technology, in order to determine whether the shuttlecock landed out of bounds. However, this system lacks the ability to recognize player behavior and cannot detect whether a player's serve height is illegal. Furthermore, the Hawk-Eye system requires a large amount of hardware, is complex, and expensive, limiting its widespread adoption.

[0075] In light of this, the applicant proposes a lightweight algorithm to detect whether a badminton player's serve height is illegal. In official matches, this algorithm can assist referees in judging serve height violations, reducing the rate of misjudgments and increasing the objectivity of the game. For amateur players, they can download a corresponding application to their mobile phones, fix their phones at a suitable height to record a serve video, and then use the software to detect serve height violations, lowering the barrier for them to pursue professional skills in the sport.

[0076] For details, please refer to Figure 1 , Figure 1 This is a flowchart of a method for detecting violations of badminton serve height according to an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0077] Step S101: Acquire the video to be tested captured by the camera at the standard serve height, and use the vanishing line detection algorithm to solve for the horizon line in the video to be tested. The horizon line is the baseline corresponding to the standard serve height.

[0078] In the above steps, the standard serve height refers to the 1.15-meter serve height stipulated by the Badminton World Federation (BWF). When acquiring the video to be tested, the camera is placed horizontally at a height of 1.15 meters above the edge of the badminton court, ensuring that the camera's field of view covers the scene when the player serves. Since the badminton court background is clean, the geometric features of the white lines on the court are relatively clear, so this information can be used to determine the relative height of the camera. That is, using the vanishing line detection principle in perspective, the horizon line in the imaging field of view is determined; this horizon line is the baseline corresponding to the standard serve height (e.g., ...). Figure 3 (As shown in part f of the diagram). In other words, the horizon line generated in the video frame to be detected marks a serve height of 1.15 meters. Using the horizon line as a baseline, objects above the horizon line are at a height greater than 1.15 meters from the ground, while objects below the horizon line are at a height less than 1.15 meters from the ground. Therefore, the position of the badminton shuttlecock during the serve can be found using a corresponding detection algorithm, and compared with the horizon line height to determine whether the serve height exceeds the horizon line. If it does, it indicates a serve violation. The detection of the shuttlecock's position during the serve will be explained in detail later and will not be elaborated on here.

[0079] In its specific implementation, this application specifies that the camera's recording specifications are at least 1080p / 30fps, meaning the screen resolution is 1920×1080 and the frame rate is 30 frames per second. Here, fps is a definition in the imaging field, referring to the number of frames transmitted per second; simply put, it refers to the number of frames in an animation or video. fps measures the amount of information used to store and display dynamic video. The more frames per second, the smoother the displayed motion. Therefore, to avoid issues such as choppy motion or stuttering in the captured video to be detected, which could affect the accuracy of subsequent height detection, the recording parameter for capturing the video to be detected is at least 30 frames per second.

[0080] Step S102: Input the above-mentioned video to be detected into the preset serve state detection model for feature extraction and prediction, obtain the probability that the video to be detected is in the serve state and compare it with the preset threshold to determine whether the video to be detected is in the serve state.

[0081] The above steps primarily involve using a pre-set serve state detection model to detect the actions of individuals in the video being tested, identifying whether the current action of the individual is in a serve state. This pre-set serve state detection model can employ a pre-trained 3D convolutional neural network to extract features from the video being tested and ultimately output the probability that the video is in a serve state.

[0082] Considering that serving is a process, simply detecting the action of a person in an image is insufficient to accurately determine whether the person is in a serving state. Therefore, this application acquires video clips containing the serving process and detects whether the person is in a serving state based on these video clips. Since 3D convolution can operate not only in the horizontal and vertical directions but also in the temporal direction, meaning it can simultaneously preserve spatial and temporal information, this application utilizes 3D convolution to extract features from the video to be detected, resulting in more accurate serving state detection results.

[0083] For example, the probability value output by the serve state detection model can be a value between 0 and 1. The higher the value, the greater the probability that the person in the video is in a serve state. By setting a probability threshold, videos with a predicted probability greater than or equal to the threshold are identified as being in a serve state, while those with a probability less than the threshold are identified as being in a non-serve state. If the video is in a serve state, the next step of target detection is performed to find the position of the shuttlecock in the serve state and to determine the degree of violation. If the video is not in a serve state, no shuttlecock target detection is required. The probability threshold can be set to 0.7, but the specific setting can be adjusted according to the actual situation and is not limited here.

[0084] Step S103: When the video to be detected is in the serving state, the position of the badminton shuttlecock is determined using a preset target detection model.

[0085] In the above steps, the object detection model is built based on CenterNet and consists of three branches: a badminton shuttlecock center point detection branch, a badminton shuttlecock bounding box detection branch, and a center point offset detection branch. These branches are used to predict the heatmap value, width, height, and offset of key points, respectively. By inputting the image into the object detection model and downsampling it, a heatmap is obtained. This heatmap represents the probability that a pixel in the image belongs to the center point of the badminton shuttlecock, and the peak point of the heatmap is identified as the center point. Since downsampling introduces a bias error when remapping the heatmap to the input image, a center point offset detection branch is needed to predict the offset of the center point. Therefore, regression can be performed based on the determined coordinates of the center point to obtain the width and height information of the shuttlecock, i.e., the bounding box size, and the position offset of the center point on the input image. Then, the coordinates of the center point in the heatmap are mapped back to the input image, and the center point is corrected based on the position offset to obtain the accurate coordinates of the center point in the input image. Finally, the width and height of the shuttlecock are obtained based on the bounding box size, thus detecting the position of the shuttlecock.

[0086] For example, suppose the input image I has a width of W and a height of H, and the number of channels is 3. After passing the input image through the object detection model, the generated heatmap has a value range of [0,1], and its width and height are W and H, respectively. Right now Where R is the downsampling factor, representing the size ratio between the input image and the output heatmap. For example, if the input image size is 512×512, when R=4, the output heatmap size will be 128×128. The heatmap value represents the probability that a pixel in the image belongs to the center point of a badminton shuttlecock. This means that the point is the center of the badminton shuttlecock. This indicates that the point belongs to the background. When the coordinates of the center point of the badminton shuttlecock detected in the heatmap are (8,15), and the predicted bounding box size is 4×6 with an offset of 2, the coordinate point (8,15) is mapped back to the input image to obtain the coordinate point (32,60). Adding the offset of 2, the coordinates of the center point of the badminton shuttlecock in the input image are obtained as (34,62), with a width of 4 and a height of 6, thus obtaining the position of the badminton shuttlecock.

[0087] It should be noted that in some embodiments of this application, the video to be detected includes the serving state. After the video to be detected is input into the target detection model, the badminton center point detection is performed on each frame of the image as described above. Then, the position of the badminton in the frame with the lowest position height is taken and compared with the horizon line to determine whether the serving height is illegal.

[0088] Step S104: Compare the position of the shuttlecock with the aforementioned horizon line to determine whether the serve is illegal.

[0089] In the above steps, if the detected shuttlecock is below eye level, the serve height is within the line and the serve is compliant; if the detected shuttlecock is above eye level, the serve height is above the line and the serve is illegal.

[0090] For example, taking the top-left corner of the image as the origin, with the positive Y-axis pointing downwards and the positive X-axis pointing to the right, the vertical value of the upper boundary of the badminton shuttlecock in the frame of the video to be detected, which has the lowest detected position height, is 65, and the vertical value of the horizon line is 63. Since the vertical value of the upper boundary of the badminton shuttlecock is greater than the vertical value of the horizon line, it indicates that the position of the badminton shuttlecock is below the horizon line, and the serve height is compliant.

[0091] This application first uses the vanishing line detection principle to generate the horizon line corresponding to the standard serve height. Then, it uses a serve state detection model to detect videos in the serve state, followed by a target detection model to identify the position of the shuttlecock in the video. Finally, by comparing the shuttlecock position with the horizon line, it achieves the penalty for a violation of the serve height. On the one hand, it can objectively and automatically identify violations of serve height at the moment of serve, improving the accuracy of serve height violation judgment and avoiding misjudgments caused by manual review. On the other hand, the detection can be performed by acquiring video data with a monocular camera. Compared with the existing Hawk-Eye system, the required hardware system is simpler, the algorithm structure is lightweight, and the operating cost is lower, which is conducive to widespread application.

[0092] Please refer to Figure 2 , Figure 2 This is a flowchart of a vanishing line detection algorithm proposed in an embodiment of this application. The process includes the following steps:

[0093] Step S101-1: Select the image to be detected from the above video to be detected, and detect all line segments in the image to be detected.

[0094] In the above steps, the camera can be placed horizontally at a height of 1.15 meters at the edge of the badminton court, ensuring that the camera's field of view covers the scene when the athlete serves, and then the video to be detected is acquired. Assuming the acquired video contains 30 frames, one image can be randomly selected as the image to be detected for vanishing line detection, thereby generating a horizon line in the image to represent the standard serve height in the actual scene. Since the video is stably acquired at a fixed height, the horizon line corresponding to each frame is the same, so only one frame needs to be selected for vanishing line detection to reduce the computational load.

[0095] After selecting the image to be detected from the video to be detected (e.g.) Figure 3 As shown in part a), image enhancement and region segmentation are first performed. Then, edge detection is performed on the processed image to be detected using the Canny algorithm to obtain the edge detection results. Finally, Hough transform is applied to the edge detection results to detect all line segments (such as...) in the image to be detected. Figure 3 (as shown in part b).

[0096] Step S101-2: Filter the line segments according to the angle between the line segment and the horizontal line, and select the longest line segments from the remaining line segments.

[0097] In the above steps, firstly, the angle between the line segment and the horizontal line is calculated, and line segments with angles less than a preset first threshold and those with angles greater than a preset second threshold are deleted. Then, the lengths of the remaining line segments are calculated, and they are sorted in descending order of length. The first few line segments (e.g., ...) are selected. Figure 3 (as shown in part c).

[0098] For example, the first preset threshold can be set to 5°, and the second preset threshold can be set to 85°. By calculating the angle between the line segment and the horizontal line, if the angle is less than 5°, it means that the line segment itself is close to horizontal; if the angle is greater than 85°, it means that the line segment itself is close to vertical. Thus, line segments that are close to horizontal or vertical are discarded to avoid affecting the subsequent search for parallel line clusters through cluster analysis.

[0099] Step S101-3: Extend the selected longest line segments and calculate the intersection point of the extensions of any two line segments to obtain the set of intersection points.

[0100] Step S101-4: Perform cluster analysis on the intersection point set to obtain the cluster with the most intersection points, and determine the corresponding line segment set.

[0101] In the above steps, in single-point perspective, a group of parallel lines will intersect at a single point, which is the vanishing point. Therefore, by extending the longest selected line segments, the intersection point of the extensions of any two line segments can be calculated (e.g., ...). Figure 3 (As shown in part d in the diagram). After obtaining the set of intersection points, DBSCAN is used to perform cluster analysis on them to obtain the cluster containing the most intersection points, and the corresponding set of line segments is determined (e.g., ...). Figure 3 (As shown in part e). The more intersections a cluster contains, the more parallel line clusters it corresponds to, and the more accurate the vanishing points obtained through these parallel line clusters will be.

[0102] Step S101-5: Based on the set of line segments corresponding to the cluster with the most intersection points, solve for the coordinates of the vanishing point.

[0103] In the above steps, assume that the equation of the line containing the i-th line segment in the obtained set of line segments is:

[0104] a i u+b i v+c i =0

[0105] And constraints:

[0106]

[0107] Among them, a i b i c is the coefficient of the general form equation of the line containing the i-th line segment. i If is a constant, then the coordinates (u0, v0) of the vanishing point should satisfy the following relationship:

[0108]

[0109] Since the lines in a family of parallel lines may not intersect at exactly one point, we use the above equation to find the point that has the shortest sum of distances to the lines in the family of parallel lines, and use this point as the vanishing point. This point is the closest to the actual vanishing point.

[0110] The equation has an analytical solution, as follows:

[0111]

[0112] Therefore, based on the pixel coordinates in the image to be detected, the general equation of the line segment containing the most intersection points in the cluster can be calculated, and then the specific coordinates of the vanishing point can be obtained.

[0113] Step S101-6: Draw a horizontal line through the vanishing point to obtain the horizon line.

[0114] In the above steps, according to the principles of perspective, drawing a horizontal line through the vanishing point will create the horizon line (as shown in the image). Figure 3 (as shown in part f), which represents the height of the camera when capturing video.

[0115] Please refer to Figure 4 , Figure 4 This is a flowchart illustrating the training process of a serve state detection model proposed in one embodiment of this application. It mainly includes the following steps:

[0116] Step S102-1: Build a serve state detection model. The serve state detection model includes multiple convolutional layers, max pooling layers, fully connected layers, and an output layer.

[0117] The framework of the built serve state detection model is as follows: Figure 5 As shown, this model employs a three-dimensional convolutional neural network, comprising eight convolutional layers, five max-pooling layers, two fully connected layers, and a final softmax output layer. All 3D convolutional kernels are 3×3×3 in size, with a stride of 1 in both the temporal and spatial domains. Figure 5 The number in each box represents the number of filters. Pool1 to Pool5 represent pooling layers; Pool1 has a 1×2×2 pooling kernel, while the others have 2×2×2 kernels. Each fully connected layer has 1024 output units. The output layer outputs the probability that the input video is in a serving state. and

[0118] Step S102-2: Acquire multiple sample videos. The recording parameters and height of the sample videos are the same as those of the video to be detected. The sample videos include positive and negative samples carrying the corresponding serve status labels.

[0119] In the above steps, since currently available human behavior datasets, such as UCF101 and HMDB51, lack data on badminton serve gestures, manual data collection and annotation are required. In this application, the sample video was collected during a badminton singles match with a clean court background and no other personnel. When subsequently acquiring the video to be detected, the camera position and recording parameters should be kept as consistent as possible with those used when collecting the sample video to improve detection accuracy.

[0120] For example, the collected sample video data includes forehand serves, backhand serves, and actions during the game. Forehand serves and backhand serves are considered positive samples in the sample video dataset, while actions during the game are considered negative samples. The sample ratio for these three types of actions is 1:1:1, and each type of action has at least 100 samples. During recording, the recording parameters are 1080p / 30fps, each video segment is 0.5 seconds long, and the size is scaled to 512×512×3×16 for input into the model for training.

[0121] Step S102-3: Based on the above multiple sample videos, train the above serve state detection model, including: inputting each sample video into the above serve state detection model, extracting shallow and deep features of the sample video through multiple convolutional layers and max pooling layers, performing feature fusion through the above fully connected layer, and predicting the probability of the sample video being in a serve state based on the feature fusion result through the above output layer to obtain a predicted value.

[0122] Step S102-4: Based on the serve status labels and predicted values ​​of the sample videos, calculate the cross-entropy loss function of the above serve status detection model, and update the model parameters in the serve status detection model.

[0123] In the above steps, the loss function of the serve state detection model adopts the cross-entropy loss function, as shown below:

[0124]

[0125] in, Let y represent the network's predicted value, and y∈{0,1} represent the actual value corresponding to the input video segment.

[0126] Step S102-5: When the cross-entropy loss function is less than the preset threshold, stop updating, complete model training, and obtain the above-mentioned preset serve state detection model.

[0127] Please refer to Figure 6 , Figure 6 This is a flowchart illustrating the training process of an object detection model proposed in one embodiment of this application. It mainly includes the following steps:

[0128] Step S103-1: Build an object detection model. The object detection model includes a feature extraction module and multiple attribute detection branches. Each attribute detection branch is connected to the feature extraction module.

[0129] The framework of the object detection model is as follows: Figure 7As shown, the backbone of this model uses a ResNet-18 network, followed by three deconvolutional layers: TransConv1, TransConv2, and TransConv3. Each deconvolutional layer has a kernel size of 4×4, a stride of 2, and padding of 1. These are followed by batch normalization layers and a ReLU activation function. The ResNet-18 backbone and the three deconvolutional layers are used as a feature extraction module to extract features from the input image. Then, a three-branch structure is used to predict the center point (heatmap), width and height (WH), and offset of the center point relative to the original image, based on the extracted features. Each branch uses two convolutional layers: the first convolutional layer has a kernel size of 3×3, a stride of 1, and padding of 1; the second convolutional layer has a kernel size of 1×1, a stride of 1, and no padding. Figure 7 The number in each box represents the number of convolution kernels.

[0130] Step S103-2: Acquire multiple sample images. The acquisition parameters and height of the sample images are the same as those of the video to be detected, and each sample image carries multiple attribute labels.

[0131] Since currently available datasets, such as MS COCO and Open Images Dataset V4, lack data on badminton, manual image collection and annotation are necessary. The image annotation tool labelImg is used to annotate badminton shuttlecocks in PASCAL VOC format, obtaining corresponding labels for the shuttlecock's center point, bounding box size, and center point offset. For example, if the input image is I, with width W and height H, and (x1, y1, x2, y2) represents the actual bounding box of the badminton shuttlecock in the image, then its center point is:

[0132]

[0133] The width and height of the shuttlecock are: (x2-x1, y2-y1).

[0134] Then, the image size was standardized to 512×512 and used as input for model training. When acquiring subsequent videos to be detected, the camera position and recording parameters were kept as consistent as possible with those used when acquiring sample images to improve detection accuracy.

[0135] Step S103-3: Based on the above multiple sample images, train the above target detection model, including: inputting each sample image into the above target detection model, extracting features from the sample image through the above feature extraction module to obtain the image features of the sample image, and detecting different attributes based on the image features through the attribute detection branch to obtain the corresponding attribute prediction results.

[0136] Step S103-4: Calculate the loss function of the above object detection model based on the attribute labels of the sample images and the corresponding attribute prediction results, and update the model parameters in the object detection model.

[0137] Specifically, the attribute detection branch includes:

[0138] 1. The badminton shuttlecock center point detection branch generates a heatmap, which represents the probability that a pixel in the sample image belongs to the center point of the shuttlecock, thus determining the coordinates of the center point. The loss function L for the badminton shuttlecock center point detection branch is... hm for:

[0139]

[0140] Where (x,y) represents the coordinates of a pixel in the heatmap, Y... x,y This represents the true probability that the point (x, y) is the center point of the badminton shuttlecock. The predicted value represents the probability that point (x,y) is the center point of the badminton shuttlecock. N represents the number of badminton shuttlecocks in the input sample image. α and β are two hyperparameters, where α = 2 and β = 4.

[0141] Specifically, the true value Y of the probability that the solution point (x, y) is the center point of the badminton shuttlecock. x,y hour:

[0142] The input sample image is downsampled to obtain the coordinates of the center point of the badminton shuttlecock.

[0143] Where c represents the coordinates of the center point of the badminton shuttlecock in the sample image, and R represents the downsampling factor. Yes The result is obtained by rounding down. For example, in the input sample image, the coordinates of the center point of the badminton shuttlecock are (34, 62). When downsampling with R=4, the calculated value is originally (8.5, 15.5). However, since the pixels are displayed as integers, it is rounded down, and the actual coordinates of the center point of the badminton shuttlecock in the final heatmap are (8, 15).

[0144] Then, the Gaussian kernel is used to map the sampled shuttlecock center point coordinates onto a heatmap to obtain the true probability Y of point (x,y) being the shuttlecock center point. x,y :

[0145]

[0146] Where, σ p It is a standard deviation that adapts to the size of an object. The x-coordinate of the center point of the badminton shuttlecock. The vertical coordinate represents the center point of the badminton shuttlecock.

[0147] 2. The badminton shuttlecock bounding box detection branch is used to regress the predicted bounding box size of the shuttlecock based on the determined coordinates of the shuttlecock's center point. The loss function of the badminton shuttlecock bounding box detection branch is L. size for:

[0148]

[0149] Where k represents the center point of the badminton shuttlecock, s k This represents the true value of the bounding box corresponding to the center point of the badminton shuttlecock. This represents the predicted value of the bounding box corresponding to the center point of the badminton shuttlecock, and N represents the number of badminton shuttlecocks in the input sample image.

[0150] 3. Center point offset detection branch: This branch is used to regress the predicted offset of the badminton shuttlecock's center point based on the determined coordinates of the shuttlecock's center point. The loss function of the center point offset detection branch is L. off for:

[0151]

[0152] in, Indicates the center point of the badminton shuttlecock. This represents the actual offset of the center point of the badminton shuttlecock. This represents the predicted offset value of the center point of the badminton shuttlecock, and N represents the number of badminton shuttlecocks in the input sample image.

[0153] Since the loss function of the target detection model consists of three parts—the heatmap prediction loss of the shuttlecock's center point location, the prediction loss of the shuttlecock's width and height, and the prediction loss of the shuttlecock's center point offset—the loss function L of the shuttlecock center point detection branch can be calculated based on the above. hm Loss function L for badminton bounding box detection branch size and the loss function L of the center point offset detection branch off Calculate the overall loss function L of the object detection model:

[0154] L = L hm +λ size L size +λ off L off

[0155] Where, λ size and λ off There are two hyperparameters, λ size =0.1, λ off =1.

[0156] Step S103-5: When the loss function is less than the preset threshold, stop updating, complete model training, and obtain the above-mentioned preset target detection model.

[0157] Based on the same inventive concept, one embodiment of this application provides a device for detecting violations in badminton serve height. Please refer to... Figure 8 , Figure 8 This is a schematic diagram of a badminton serve height violation detection device according to an embodiment of this application, including:

[0158] The horizon line generation module 1 is used to acquire the video to be detected captured by the camera at the standard serve height, and to solve the horizon line in the video to be detected using the vanishing line detection algorithm. The horizon line is the baseline corresponding to the standard serve height.

[0159] The serve state detection module 2 is used to input the above-mentioned video to be detected into a preset serve state detection model for feature extraction and prediction, to obtain the probability that the video to be detected is in the serve state and compare it with a preset threshold to determine whether the video to be detected is in the serve state.

[0160] The serve height detection module 3 is used to determine the position of the badminton shuttlecock using a preset target detection model when the video to be detected is in the serve state.

[0161] The service violation judgment module 4 is used to compare the position of the badminton shuttlecock with the aforementioned horizon line to determine whether the service is illegal.

[0162] Optionally, the horizon generation module 1 includes:

[0163] The selection submodule is used to select the image to be detected from the above-mentioned video to be detected, and to detect all line segments in the image to be detected;

[0164] The filtering submodule is used to filter line segments based on the angle between the line segment and the horizontal line, and select the longest line segments from the remaining line segments.

[0165] The intersection submodule is used to extend the selected longest line segments and calculate the intersection point of the extensions of any two line segments to obtain the intersection point set.

[0166] The clustering submodule is used to perform cluster analysis on the intersection point set, obtain the cluster containing the most intersection points, and determine the corresponding line segment set;

[0167] The vanishing point solution submodule is used to solve for the coordinates of the vanishing point based on the set of line segments corresponding to the cluster with the most intersections.

[0168] The horizon line generation submodule is used to draw a horizontal line through the vanishing point mentioned above to obtain the horizon line.

[0169] Optionally, the above selection submodule includes:

[0170] The preprocessing unit is used to select the image to be detected from the above-mentioned video to be detected, and to perform image enhancement and region segmentation processing;

[0171] The edge detection unit is used to perform edge detection on the processed image to be detected and obtain the edge detection result.

[0172] The line segment detection unit is used to perform Hough transform on the edge detection results to detect all line segments in the image to be detected.

[0173] Optionally, the above filtering submodule includes:

[0174] The filtering unit is used to calculate the angle between the line segment and the horizontal line, and delete line segments with an angle less than a preset first threshold and line segments with an angle greater than a preset second threshold.

[0175] The sorting unit is used to calculate the length of the remaining line segments, sort them in descending order of length, and select the first few line segments.

[0176] Optionally, the above-mentioned device further includes:

[0177] The first model building module is used to build a serve state detection model, which includes multiple convolutional layers, max pooling layers, fully connected layers, and an output layer.

[0178] The sample video acquisition module is used to acquire multiple sample videos. The recording parameters and height of the sample videos are the same as those of the video to be detected. The sample videos include positive and negative samples carrying the corresponding serve status labels.

[0179] The first model training module is used to train the above-mentioned serve state detection model based on the above-mentioned multiple sample videos, including: inputting each sample video into the above-mentioned serve state detection model, extracting shallow and deep features of the above-mentioned sample video through multiple above-mentioned convolutional layers and max pooling layers, performing feature fusion through the above-mentioned fully connected layer, and predicting the probability of the sample video being in the serve state based on the feature fusion result through the above-mentioned output layer to obtain a predicted value.

[0180] The first model update module is used to calculate the cross-entropy loss function of the above-mentioned serve state detection model based on the serve state label and predicted value of the sample video, and update the model parameters in the serve state detection model.

[0181] When the cross-entropy loss function is less than the preset threshold, the update stops, the model training is completed, and the above-mentioned preset serve state detection model is obtained.

[0182] Optionally, the above-mentioned device further includes:

[0183] The second model building module is used to build the target detection model. The target detection model includes a feature extraction module and multiple attribute detection branches. Each attribute detection branch is connected to the feature extraction module.

[0184] The sample image acquisition module is used to acquire multiple sample images. The acquisition parameters and height of the sample images are the same as those of the video to be detected, and each sample image carries multiple attribute labels.

[0185] The second model training module is used to train the target detection model based on the above multiple sample images, including: inputting each sample image into the target detection model, extracting features from the sample image through the feature extraction module to obtain the image features of the sample image, and detecting different attributes based on the image features through the attribute detection branch to obtain the corresponding attribute prediction results.

[0186] The second model update module is used to calculate the loss function of the above object detection model based on the attribute labels of the sample images and the corresponding attribute prediction results, and to update the model parameters in the object detection model.

[0187] When the loss function is less than the preset threshold, the update stops, the model training is completed, and the above-mentioned preset target detection model is obtained.

[0188] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0189] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0190] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0191] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0192] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0193] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0194] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0195] The above provides a detailed description of the method and device for detecting violations of badminton serve height provided by this application. Specific examples have been used to illustrate the principle and implementation of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​this application. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method of detecting a violation of a serving height in badminton, characterized by, The method comprises: acquiring a to-be-detected video collected by a camera at a standard serving height, and solving a horizon line in the to-be-detected video by using a vanishing line detection algorithm, the horizon line being a reference line corresponding to the standard serving height, comprising: selecting a to-be-detected image from the to-be-detected video, and detecting all line segments in the to-be-detected image; filtering the line segments according to the included angle between the line segments and a horizontal line, and selecting a plurality of longest line segments from the remaining line segments; extending the selected plurality of longest line segments, and calculating the intersection of the extensions of any two line segments to obtain a set of intersection points; performing clustering analysis on the set of intersection points to obtain a clustering cluster containing the most intersection points, and determining a line segment set corresponding to the clustering cluster; solving the coordinates of a vanishing point according to the line segment set corresponding to the clustering cluster containing the most intersection points; drawing a horizontal line through the vanishing point to obtain the horizon line; the filtering of the line segments according to the included angle between the line segments and the horizontal line, and the selection of the plurality of longest line segments from the remaining line segments, comprising: calculating the included angle between the line segments and the horizontal line, and deleting the line segments with an included angle less than a preset first threshold value and the line segments with an included angle greater than a preset second threshold value; calculating the lengths of the remaining line segments respectively, and sorting the lengths in descending order to select the first plurality of line segments; inputting the to-be-detected video into a preset serving state detection model for feature extraction and prediction to obtain a probability that the to-be-detected video is in a serving state and compare the probability with a preset threshold value to determine whether the to-be-detected video is in a serving state; when the to-be-detected video is in a serving state, determining the position of the shuttlecock by using a preset target detection model, the target detection model comprising a feature extraction module and a plurality of attribute detection branches, each attribute detection branch being connected with the feature extraction module, the target detection model comprising a feature extraction module and a plurality of attribute detection branches, each attribute detection branch being connected with the feature extraction module, and the attribute detection branches comprising a shuttlecock center point detection branch, a shuttlecock bounding box detection branch, and a center point offset detection branch; comparing the position of the shuttlecock with the horizon line to determine whether the serving is in violation of the rules.

2. The method of claim 1, wherein, The selection of the to-be-detected image from the to-be-detected video and the detection of all line segments in the to-be-detected image, comprising: selecting a to-be-detected image from the to-be-detected video, and performing image enhancement and region segmentation processing on the to-be-detected image; performing edge detection on the processed to-be-detected image to obtain an edge detection result; performing Hough transformation on the edge detection result to detect all line segments in the to-be-detected image.

3. The method according to claim 1 or 2, characterized in that, Further comprising: building a serving state detection model, the serving state detection model comprising a plurality of convolutional layers, a maximum pooling layer, a fully connected layer, and an output layer; acquiring a plurality of sample videos, the recording parameters and height of the sample videos being the same as those of the to-be-detected video, the sample videos comprising positive samples and negative samples carrying corresponding serving state labels; Based on the plurality of sample videos, the service state detection model is trained, including: inputting each sample video into the service state detection model to extract shallow features and deep features of the sample video through a plurality of convolutional layers and a maximum pooling layer, and performing feature fusion through the fully connected layer, and predicting the probability of the sample video being in the service state according to the result of feature fusion through the output layer to obtain a prediction value; According to the service state label and the prediction value of the sample video, the cross-entropy loss function of the service state detection model is calculated, and the model parameters in the service state detection model are updated; When the cross-entropy loss function is less than a preset threshold, the updating is stopped, the model training is completed, and the preset service state detection model is obtained.

4. The method according to claim 1 or 2, characterized in that, Also includes: Build a target detection model, the target detection model includes a feature extraction module and a plurality of attribute detection branches, each attribute detection branch is connected with the feature extraction module; Obtain a plurality of sample images, the sample images have the same acquisition parameters and height as the video to be detected, and each sample image carries a plurality of attribute labels; Based on the plurality of sample images, the target detection model is trained, including: inputting each sample image into the target detection model to extract features of the sample image through the feature extraction module, and detecting different attributes based on the image features through the attribute detection branch to obtain corresponding attribute prediction results; According to the attribute label and the corresponding attribute prediction result of the sample image, the loss function of the target detection model is calculated, and the model parameters in the target detection model are updated; When the loss function is less than a preset threshold, the updating is stopped, the model training is completed, and the preset target detection model is obtained.

5. The method of claim 4, wherein, The attribute detection branch includes: The badminton center point detection branch is used for generating a heat map representing a probability that a pixel point in a sample image belongs to a badminton center point, so as to determine a badminton center point coordinate, and a loss function of the badminton center point detection branch is is: wherein, denotes the coordinates of a pixel point in the heat map, denotes the point is the true value of the probability of the center point of the shuttlecock, denotes the point is the predicted value of the probability of the center point of the shuttlecock, and N denotes the number of shuttlecocks in the input sample image, and are two hyperparameters, , ; The badminton bounding box detection branch is used for regressing the predicted bounding box size of the badminton according to the determined badminton center point coordinates, and a loss function of the badminton bounding box detection branch is is: wherein k represents the center point of the shuttlecock, represents the true value of the bounding box corresponding to the center point of the shuttlecock, represents the predicted value of the bounding box corresponding to the center point of the shuttlecock, and N represents the number of shuttlecocks in the input sample image; The center point offset detection branch is used for regressing the offset of the predicted badminton center point according to the determined badminton center point coordinates, and a loss function of the center point offset detection branch is is: wherein, represents a center point of the shuttlecock, represents an actual offset value of the center point of the shuttlecock, represents a predicted offset value of the center point of the shuttlecock, and N represents the number of shuttlecocks in the input sample image.

6. The method of claim 5, wherein, The calculation of the loss function of the target detection model includes: According to the badminton center point detection branch loss function , the badminton bounding box detection branch loss function , and the center point offset detection branch loss function , calculate the overall loss function L of the target detection model: wherein and are two hyperparameters, , .

7. The method of claim 5, wherein, The badminton center point detection branch, point The true value of the probability that the point The calculation includes: Downsample the input sample image to get the sample center point coordinate of the shuttlecock : wherein, represents the center point coordinates of the shuttlecock in the sample image, and R represents a down-sampling multiple, represents a floor function; The center point coordinates of the badminton after sampling are mapped into a heat map by using a Gaussian kernel to obtain points The true value of the probability of the center point of the badminton : wherein is a standard deviation of object size adaptation, represents the horizontal coordinate of the center point of the shuttlecock, represents the vertical coordinate of the center point of the shuttlecock.

8. A badminton service height violation detection apparatus, characterized by, The device includes: A horizon generation module is configured to obtain a to-be-detected video collected by a camera at a standard service height, and solve a horizon in the to-be-detected video by using a vanishing line detection algorithm, the horizon being a reference line corresponding to the standard service height, and the horizon including: Select a to-be-detected image from the to-be-detected video, and detect all line segments in the to-be-detected image; filter the line segments according to the included angle of the line segments with a horizontal line, and select a plurality of longest line segments from the remaining line segments; extend the selected plurality of longest line segments, and calculate the intersection point of the extensions of any two line segments to obtain a set of intersection points; perform clustering analysis on the set of intersection points to obtain a clustering cluster containing the most intersection points, and determine a line segment set corresponding to the clustering cluster; solve the coordinates of a vanishing point according to the line segment set corresponding to the clustering cluster containing the most intersection points; draw a horizontal line through the vanishing point to obtain the horizon; the filtering of the line segments according to the included angle of the line segments with a horizontal line, and the selection of a plurality of longest line segments from the remaining line segments, comprises: calculating the included angle of the line segments with a horizontal line, deleting line segments with an included angle less than a preset first threshold value and line segments with an included angle greater than a preset second threshold value; calculating the lengths of the remaining line segments respectively, and sorting the lengths in descending order to select a plurality of longest line segments; The ball serving state detection module is configured to input the to-be-detected video into a preset ball serving state detection model to perform feature extraction and prediction, obtain a probability that the to-be-detected video is in a ball serving state, and compare the probability with a preset threshold value to determine whether the to-be-detected video is in a ball serving state; The ball serving height detection module is configured to, when the to-be-detected video is in a ball serving state, determine the position of the shuttlecock by using a preset target detection model, wherein the target detection model comprises a feature extraction module and a plurality of attribute detection branches, each attribute detection branch is connected with the feature extraction module, and the attribute detection branch comprises a shuttlecock center point detection branch, a shuttlecock bounding box detection branch, and a center point offset detection branch. The ball serving violation determination module is configured to compare the position of the shuttlecock with the horizon to determine whether the ball serving is in violation.

Images