A real-time tennis serve scoring method and device

Abstract

The application discloses a real-time tennis serve scoring method and device, which identifies the two-dimensional pixel coordinates of a tennis ball through multi-camera synchronous frame recognition, constructs the three-dimensional coordinates of the tennis ball by using the ray intersection method, analyzes the continuous three-dimensional coordinate sequence, calculates the frame interval according to the y-axis movement trend, determines the serve starting point, then finds the height minimum value or inflection point through the continuous three-dimensional coordinate sequence, and determines the serve landing point. The spatial coordinate range of the serve area is dynamically calculated in combination with the serve starting point coordinates and the field reference point, and the serve area boundary of the singles or doubles mode is distinguished, the landing point state of the tennis ball is determined according to the landing point and the serve area boundary, and the serve side or the return serve side is scored according to the scoring rules. The application analyzes the y-axis change trend of the tennis ball movement through the three-dimensional trajectory data, the serve starting point and the serve landing point can be accurately identified by using the y-axis change trend, so that the accuracy and efficiency of the serve scoring discrimination are greatly improved, and the fairness of the game is ensured.

Description

Technical Field

[0001] This application relates to the field of tennis technology, specifically to a method and device for real-time tennis serve scoring. Background Technology

[0002] Tennis is a widely popular sport, attracting a large number of fans worldwide with its unique rules and high-intensity competition. In tennis matches, the umpire's decisions are crucial, especially in determining whether the ball is in or out. Traditional methods rely primarily on the umpire's eyesight and experience, but human observation is often subject to subjective bias. For example, when judging the landing point, the umpire may misjudge due to different perspectives or fatigue; furthermore, the judging process typically takes 2-5 seconds, which can disrupt the flow of the match. More importantly, human judgment can lead to inconsistencies due to differences in the umpire's understanding of the rules, especially in controversial cases where slow-motion replays are often needed for further confirmation, increasing time costs and thus affecting the accuracy of the judgment.

[0003] To ensure the fairness of matches and the accuracy of judgments, the tennis world has introduced automated judging systems. These systems use high-precision technology to track the trajectory of every ball in real time, providing referees with more accurate judgment criteria, thereby improving the fairness and professionalism of the matches. For example, existing technology, as proposed in "Research on Tennis Landing Point Localization Based on Monocular Vision," proposes a tennis landing point localization system based on monocular vision to replace or supplement expensive professional landing point tracking systems that are difficult to deploy in low-cost environments.

[0004] Please see Figure 1 The core technologies of the monocular vision-based tennis ball landing point positioning system include camera calibration, landing point tracking, and landing point positioning. Camera calibration uses the corner points of the court lines as calibration feature points to solve for camera intrinsic and extrinsic parameters and reflex parameters, and optimizes the calibration feature points based on reprojection error. Landing point tracking identifies foreground targets through background models and tracking algorithms, and determines the landing frame based on channel direction changes in adjacent frames. Landing point positioning removes reflex and reflex in the image, applies a projection model, flips the applied camera parameters, and flips the landing point coordinates to the three-dimensional world coordinate system.

[0005] The system monitors a tennis court using a fixed monocular camera. First, the image acquisition module continuously acquires video frames, and the image preprocessing module performs grayscale conversion, noise reduction, and edge enhancement on the images. Then, the tennis ball detection module uses the frame difference method to detect the position of the moving object (tennis ball) and extracts the two-dimensional pixel coordinates of the ball in the image frame using the centroid localization method. The trajectory extraction module performs linear fitting on the pixel position of the ball in the continuous frames to establish a two-dimensional trajectory.

[0006] To determine whether a tennis ball has landed and is in bounds, the system extracts the ball's motion trend through a motion analysis module and identifies possible landing frames based on a sharp drop in speed or extreme points in the y-direction. Next, the court line detection module detects the edges in the image based on Hough transform and maps the landing pixel coordinates to the actual court coordinate range. Finally, the landing point judgment module compares the mapping results with a predefined court area and outputs the judgment result.

[0007] This technology is primarily applicable to standard tennis court environments, using monocular vision to identify and determine the landing point of the tennis ball. It relies on stable image acquisition and processing algorithms, achieving a certain degree of automation in landing point detection. However, in serve recognition, this technology typically depends on a single 2D camera or simple motion analysis, failing to effectively distinguish between serves and regular shots, especially prone to misjudgments when the ball's motion changes rapidly. Furthermore, traditional tennis landing point detection algorithms are based solely on 2D coordinates, which are relatively limited and easily lead to misjudgments of the actual landing point. These misjudgments in serve recognition and landing point detection can cause errors in automatic serve scoring, thus affecting the fairness of the match. Summary of the Invention

[0008] Therefore, this application provides a real-time tennis serve scoring method and device to solve the problem that the existing technology has errors in automatic serve scoring due to misjudgment in serve recognition and landing point detection, which in turn affects the fairness of the game.

[0009] To achieve the above objectives, this application provides the following technical solution:

[0010] Firstly, a real-time tennis serve scoring method includes:

[0011] Step 1: Acquire video frames in parallel using multiple cameras and time-align the multiple video frames to obtain multiple synchronized frames;

[0012] Step 2: Use the ball detection algorithm to identify the two-dimensional pixel coordinates of the tennis ball in the multi-channel synchronous frames, and construct the three-dimensional coordinates of the tennis ball in space through the ray intersection method to obtain a continuous three-dimensional coordinate sequence;

[0013] Step 3: Analyze the continuous three-dimensional coordinate sequence of the tennis ball in space within the sliding window, and calculate the frame interval between the current frame and the previous effective movement based on the tennis ball's movement trend along the y-axis. Determine the coordinates of the serve start point based on the frame interval.

[0014] Step 4: Traverse the continuous three-dimensional coordinate sequence, and determine the minimum height point or trajectory inflection point based on the continuous three-dimensional coordinate sequence, and determine the coordinates of the serve landing point based on the minimum height point or trajectory inflection point;

[0015] Step 5: Dynamically calculate the spatial coordinate range of the service area based on the coordinates of the service starting point and the reference point of the tennis court, and distinguish the service area boundary for singles or doubles modes based on the spatial coordinate range;

[0016] Step 6: Determine the landing point of the tennis ball based on the coordinates of the serve landing point and the service court boundaries in singles or doubles mode;

[0017] Step 7: Score the server or receiver based on the landing point and the pre-set scoring rules.

[0018] Optionally, step 1, when performing time alignment on multiple video frames, specifically includes: comparing the timestamps of each frame and selecting frames whose timestamp intervals are within a set allowable error range for synchronous time alignment.

[0019] Optionally, in step 2, constructing the three-dimensional coordinates of the tennis ball in space using the ray intersection method specifically includes: using pre-calibrated camera parameters to convert the two-dimensional pixel coordinates into rays in three-dimensional space, calculating the intersection points of the rays corresponding to each camera in pairs, using a distance threshold to filter out the valid intersection points as three-dimensional reconstruction points, and recording the three-dimensional coordinates of the three-dimensional reconstruction points.

[0020] Optionally, step 3 specifically includes:

[0021] Step 301: Analyze the continuous three-dimensional coordinates of the tennis ball in space within the sliding window, and determine the motion trend of the tennis ball along the y-axis;

[0022] Step 302: If the tennis ball's motion trend on the y-axis does not change significantly, wait for the next frame; if the tennis ball's motion trend on the y-axis is continuously rising or continuously falling, calculate the frame interval between the current frame and the previous effective motion.

[0023] Step 303: Determine whether the frame interval is greater than the effective motion interval threshold;

[0024] Step 304: If the interval is less than the effective motion interval threshold, update the frame number and index of the previous effective motion and wait for the next frame; if the interval is greater than the effective motion interval threshold, determine it as a new serve and record the coordinates of the serve start point.

[0025] Optionally, in step 301, determining the motion trend of the tennis ball along the y-axis specifically includes:

[0026] Step 3011: Obtain the continuous three-dimensional coordinates of the tennis ball in space within the sliding window, and extract the y-axis coordinates of all valid points;

[0027] Step 3012: Determine whether the number of y-axis coordinates of all valid points is greater than the first threshold; if it is less than or equal to the first threshold, it is determined that the tennis ball has no obvious movement trend on the y-axis; if it is greater than the first threshold, calculate the difference between consecutive y-axis coordinates and count the rising and falling points.

[0028] Step 3013: Determine if the rising count is greater than the falling count;

[0029] Step 3014: If the upward count is greater than the downward count, calculate the percentage of the upward count and determine whether the percentage of the upward count is greater than or equal to the second threshold; if it is less than the second threshold, it is determined that the tennis ball has no obvious movement trend on the y-axis; if it is greater than or equal to the second threshold, it is determined that the tennis ball is continuously rising on the y-axis.

[0030] Step 3015: If the rising count is less than or equal to the falling count, then determine whether the falling count is greater than the rising count; if the falling count is greater than the rising count, then calculate the percentage of the falling count.

[0031] Step 3016: Determine whether the percentage of the decreasing count is greater than or equal to the third threshold; if it is less than the third threshold, it is determined that the tennis ball has no obvious movement trend on the y-axis; if it is greater than or equal to the third threshold, it is determined that the tennis ball is continuously decreasing on the y-axis.

[0032] Optionally, step 4 specifically includes:

[0033] Step 401: Traverse the continuous three-dimensional coordinate sequence and calculate the height value of each point;

[0034] Step 402: Compare the height value of each point with the height values ​​of the adjacent points before and after it, and determine whether the height of the intermediate point is the minimum height value or whether there is an inflection point in the trajectory;

[0035] Step 403: If the height of the intermediate point is the minimum value or the trajectory has an inflection point, then add the intermediate point to the landing point candidate list;

[0036] Step 404: Select the candidate point with the lowest height value from the candidate landing point list as the landing point, and record the landing point coordinates to obtain the service landing point coordinates.

[0037] Optionally, it also includes: for candidate points in the candidate list of landing points, fusing the minimum height, maximum acceleration, instantaneous velocity, velocity change and spatial constraints, and calculating a confidence score, and determining the actual ball landing point coordinates based on the confidence score.

[0038] Optionally, step 6 specifically includes:

[0039] Step 601: Determine whether the coordinates of the serve landing point are within the service area boundary in singles or doubles mode;

[0040] Step 602: If the coordinates of the serve landing point are within the service area boundary, the landing point status is determined to be valid;

[0041] Step 603: If the coordinates of the serve landing point are not within the service area boundary or land on the net, the landing point status is determined to be invalid.

[0042] Optionally, in step 7, the scoring rules are as follows: if the coordinates of the serve landing point are within the valid service area and the receiving side does not receive the ball, the serving side scores; if the coordinates of the serve landing point are outside the boundary or go into the net after the serving side serves, the receiving side scores.

[0043] Secondly, a real-time tennis serve scoring device includes:

[0044] The video frame acquisition module is used to acquire video frames in parallel from multiple cameras and to time-align the multiple video frames to obtain multiple synchronized frames.

[0045] The 3D coordinate construction module is used to identify the two-dimensional pixel coordinates of the tennis ball in multi-channel synchronous frames using a ball detection algorithm, and to construct the three-dimensional coordinates of the tennis ball in space using a ray intersection method to obtain a continuous 3D coordinate sequence.

[0046] The real-time serve recognition module is used to analyze the continuous three-dimensional coordinate sequence of the tennis ball in space within the sliding window, and calculate the frame interval between the current frame and the previous effective motion based on the tennis ball's motion trend on the y-axis, and determine the coordinates of the serve starting point based on the frame interval.

[0047] The real-time landing point detection module is used to traverse a continuous three-dimensional coordinate sequence and determine the minimum height point or trajectory inflection point based on the continuous three-dimensional coordinate sequence, and determine the coordinates of the serve landing point based on the minimum height point or trajectory inflection point.

[0048] The service boundary determination module is used to dynamically calculate the spatial coordinate range of the service area based on the coordinates of the service starting point and the reference point of the tennis court, and to distinguish the service area boundary in singles or doubles mode based on the spatial coordinate range.

[0049] The landing point detection module is used to determine the landing point status of the tennis ball based on the landing point coordinates and the service area boundary in singles or doubles mode.

[0050] The scoring module is used to score the server or receiver based on the landing point and pre-set scoring rules.

[0051] Compared with the prior art, this application has at least the following beneficial effects:

[0052] This application provides a real-time tennis serve scoring method. It identifies the two-dimensional pixel coordinates of the tennis ball using synchronized frames from multiple cameras, constructs the three-dimensional coordinates of the ball using ray intersection, analyzes the continuous three-dimensional coordinate sequence, calculates the frame interval based on the y-axis motion trend, and determines the serve's starting point. Then, it finds the height minimum or inflection point through the continuous three-dimensional coordinate sequence to determine the serve's landing point. Combining the serve's starting point coordinates and court reference points, it dynamically calculates the spatial coordinate range of the service area, distinguishes the service area boundaries for singles or doubles modes, and determines the tennis ball's landing state based on the landing point and service area boundaries. Scoring is then awarded to the serving or receiving side according to the scoring rules. This application analyzes the y-axis variation trend of the tennis ball's motion using three-dimensional trajectory data. Utilizing this trend, the serve's starting point and landing point can be accurately identified, greatly improving the accuracy and efficiency of serve scoring and ensuring the fairness of the match. Attached Figure Description

[0053] To more intuitively illustrate the prior art and this application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary drawings, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).

[0054] Figure 1 A flowchart of a monocular vision-based tennis ball landing point localization method provided for existing technologies;

[0055] Figure 2 A basic flowchart of a real-time tennis serve scoring method provided in Embodiment 1 of this application;

[0056] Figure 3 This is a flowchart illustrating a real-time tennis serve scoring method provided in Embodiment 1 of this application.

[0057] Figure 4 This is a structural block diagram of a real-time tennis serve scoring method provided in Embodiment 1 of this application;

[0058] Figure 5 This application provides a real-time serve recognition algorithm as an example of its embodiment.

[0059] Figure 6 This is a flowchart illustrating the determination of the tennis ball's motion trend along the y-axis in a real-time serve recognition algorithm provided in Embodiment 1 of this application;

[0060] Figure 7 This is a flowchart of a real-time landing point detection algorithm provided in Embodiment 1 of this application. Detailed Implementation

[0061] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] In the description of this application: unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," "third," etc., in this application are intended to distinguish the objects referred to and do not have any special meaning in terms of technical connotation (e.g., they should not be construed as an emphasis on importance or order). Expressions such as "comprising," "including," and "having" also mean "not limited to" (certain units, components, materials, steps, etc.).

[0063] The terms used in this application, such as "upper," "lower," "left," "right," and "middle," are generally used to indicate the general relative positional relationship for the purpose of intuitive understanding by referring to the accompanying drawings, and are not absolute limitations on the positional relationship in the actual product.

[0064] Example 1

[0065] Please see Figure 2 , Figure 3 and Figure 4 This embodiment provides a real-time tennis serve scoring method. By integrating serve detection and landing point analysis, dynamically calculating court boundaries and verifying multi-frame data, it achieves automated serve scoring, greatly improving the accuracy and efficiency of serve scoring and ensuring the fairness and efficiency of the rulings. The method includes:

[0066] S1: Video frames are acquired in parallel by multiple cameras, and time-aligned to obtain multiple synchronized frames;

[0067] This embodiment provides a real-time synchronization algorithm for multi-camera frames based on shared memory and a circular buffer when performing time alignment. It can efficiently solve the real-time synchronization problem of multi-camera data acquisition. Compared with existing single-camera or simple software synchronization solutions, this algorithm significantly improves synchronization accuracy and real-time performance.

[0068] In a real-time synchronization algorithm for multi-camera frames based on shared memory and a circular buffer, each camera acquires video frames in real time through an independent process, with each frame accompanied by a precise timestamp. A shared memory circular queue structure is used, where each camera process writes the acquired frames to shared memory. Frame data includes frame sequence number, timestamp, and image data. The shared memory structure includes write and read indices, and a process lock is used to prevent multi-process access contention. The synchronization process monitors the shared memory of the two cameras in real time, comparing the timestamps of each frame and selecting frames with timestamp differences within a set allowable error range for synchronization pairing. Strict time matching logic is used during the initial synchronization to ensure that the timestamps of the first paired frames from the two cameras are very close. The synchronized frame pairs are output to the processing queue in real time for use in subsequent 3D reconstruction algorithms. Compared with existing technologies, this embodiment, through an efficient shared memory mechanism and precise frame index and timestamp pairing, greatly improves the accuracy and real-time performance of multi-camera data synchronization, ensuring the efficiency and accuracy of subsequent 3D reconstruction and data processing.

[0069] S2: Use a ball detection algorithm to identify the two-dimensional pixel coordinates of the tennis ball in multi-channel synchronous frames, and construct the three-dimensional coordinates of the tennis ball in space through the ray intersection method to obtain a continuous three-dimensional coordinate sequence.

[0070] Specifically, this embodiment provides a real-time 3D spherical coordinate reconstruction algorithm when constructing 3D coordinates. This algorithm is based on synchronized multi-camera 2D image data and accurately reconstructs the real-time 3D trajectory of a tennis ball in space using a ray intersection method. Camera parameters (including intrinsic matrix and distortion coefficients) are pre-calibrated for each camera by acquiring 2D markers on the court and known 3D markers. The synchronized 2D pixel coordinates of the multiple cameras are acquired in real time, and the 2D points are converted into rays in 3D space using the camera parameters. The intersection points of the rays corresponding to each camera are calculated pairwise, and valid intersection points are selected using a distance threshold (e.g., less than 50 mm). If multiple candidate intersection points exist, the intersection point with the smallest ray distance is selected as the 3D reconstruction point for the current frame. If no valid intersection point exists, a placeholder (e.g., [-9999,-9999,-9999]) is used to indicate that there is no valid reconstruction data for this frame. Finally, the valid 3D point data is output in real time to the subsequent processing module.

[0071] S3: Analyze the continuous three-dimensional coordinate sequence of the tennis ball in space within the sliding window, and calculate the frame interval between the current frame and the previous effective movement based on the tennis ball's movement trend on the y-axis. Determine the coordinates of the serve start point based on the frame interval.

[0072] Please see Figure 5This implementation provides a real-time serve recognition algorithm for serve recognition. Based on the motion characteristics of a tennis serve (a serve typically involves the ball moving at high speed along the court's depth (y-axis) after a period of stillness or slow motion, with a significant time interval between this and the previous valid serve), the algorithm calculates the y-axis trend using a sliding window (e.g., 10 frames) for the three-dimensional points within the buffer zone. It counts the number of frames where the y-axis continuously rises or falls within the window. If this percentage exceeds a preset threshold (e.g., 0.4), a valid serve is considered to have occurred. Specifically, this includes:

[0073] S301: Analyze the continuous three-dimensional coordinates of the tennis ball in space within the sliding window, and determine the motion trend of the tennis ball along the y-axis;

[0074] For more details, please see Figure 6 In this embodiment, determining the motion trend of the tennis ball along the y-axis includes:

[0075] S3011: Obtain the continuous three-dimensional coordinates of the tennis ball in space within the sliding window, and extract the y-axis coordinates of all valid points;

[0076] S3012: Determine whether the number of y-axis coordinates of all valid points is greater than the first threshold; if it is less than or equal to the first threshold, it is determined that the tennis ball has no obvious movement trend on the y-axis; if it is greater than the first threshold, calculate the difference between consecutive y-axis coordinates and count the rising and falling points.

[0077] S3013: Determine if the rising count is greater than the falling count;

[0078] S3014: If the upward count is greater than the downward count, calculate the percentage of the upward count and determine whether the percentage of the upward count is greater than or equal to the second threshold; if it is less than the second threshold, determine that the tennis ball has no obvious movement trend on the y-axis; if it is greater than or equal to the second threshold, determine that the tennis ball is continuously rising on the y-axis.

[0079] S3015: If the rising count is less than or equal to the falling count, then determine whether the falling count is greater than the rising count; if the falling count is greater than the rising count, then calculate the percentage of the falling count.

[0080] S3016: Determine whether the percentage of the decreasing count is greater than or equal to the third threshold; if it is less than the third threshold, it is determined that the tennis ball has no obvious movement trend on the y-axis; if it is greater than or equal to the third threshold, it is determined that the tennis ball is continuously decreasing on the y-axis.

[0081] S302: If the tennis ball's motion trend on the y-axis does not change significantly, wait for the next frame; if the tennis ball's motion trend on the y-axis is continuously rising or continuously falling, calculate the frame interval between the current frame and the previous effective motion.

[0082] S303: Determine whether the frame interval is greater than the effective motion interval threshold;

[0083] S304: If the interval is less than the effective motion interval threshold, update the frame number and index of the previous effective motion and wait for the next frame; if the interval is greater than the effective motion interval threshold, determine it as a new serve and record the coordinates of the serve start point.

[0084] Specifically, in this embodiment, combined with frame indexing, if the frame interval between the current valid motion and the previous valid motion exceeds a threshold (e.g., 120 frames), it is determined to be a new serve. At this time, the algorithm backtracks to the starting point of the window, finds the first valid 3D point as the starting point of this serve, and records its spatial coordinates and frame number.

[0085] To further enhance robustness, this embodiment employs a double-ended queue (deque(maxlen=15)) to store the serve detection states of the most recent frames. Only when a serve is detected in multiple consecutive frames and the landing point is detected within 15 frames after the serve is the landing point ultimately recognized as the serve landing point. The algorithm's data structure includes a three-dimensional point buffer, a motion detection sliding window, an effective motion interval threshold, a serve state queue, and a serve start point record, etc. All parameters can be flexibly adjusted according to the actual field and game pace. Compared with existing single-camera or two-dimensional algorithms, this embodiment can accurately distinguish between serves and ordinary hits, greatly reducing the misjudgment rate. Furthermore, the multi-frame fusion strategy effectively suppresses occasional misjudgments, ensuring the real-time performance, accuracy, and versatility of the judgment.

[0086] S4: Traverse the continuous three-dimensional coordinate sequence, and determine the minimum height point or trajectory inflection point based on the continuous three-dimensional coordinate sequence, and determine the coordinates of the serve landing point based on the minimum height point or trajectory inflection point;

[0087] Please see Figure 7 This embodiment provides a real-time landing point detection algorithm for determining the coordinates of the serve landing point. The core idea of ​​this landing point detection algorithm is based on the physical characteristics of a tennis ball during its flight: after being hit, the ball's height (y-axis, usually perpendicular to the court) first rises, reaches its peak, and then begins to fall, finally decreasing rapidly upon contact with the ground. After landing, the ball's speed and direction also change significantly. Specifically, this includes:

[0088] S401: Traverse the continuous three-dimensional coordinate sequence and calculate the height value of each point;

[0089] S402: Compare the height value of each point with the height values ​​of the adjacent points before and after it, and determine whether the height of the intermediate point is a minimum value or whether there is an inflection point in the trajectory;

[0090] S403: If the height of the intermediate point is the minimum value or the trajectory has an inflection point, then add the intermediate point to the landing point candidate list;

[0091] S404: Select the candidate point with the lowest height value from the candidate list of landing points as the landing point, and record the landing point coordinates to obtain the coordinates of the serve landing point.

[0092] To avoid occasional misjudgments, this embodiment statistically analyzes the height changes over several consecutive frames. Only when the height suddenly rises after a continuous decrease, and the lowest point is below a threshold, is that point determined as the landing point. Furthermore, this embodiment calculates the instantaneous velocity of each point by measuring the spatial distance and time interval between adjacent points to determine the ball's velocity along that trajectory. If there is a significant abrupt change in velocity before or after a certain moment, such as a sudden drop from high speed to near zero, that point is highly likely to be the ball's actual landing point.

[0093] To improve the robustness of the judgment, the landing point detection algorithm uses a sliding window method (e.g., the most recent 6 to 10 frames) to calculate the average velocity and combines it with a reasonable velocity threshold (e.g., 10 to 220 km / h) to filter out velocity abrupt change points as candidate landing points. The algorithm also analyzes the ball's acceleration, paying particular attention to the acceleration extreme values ​​at the moment of impact. By performing second-order difference calculations on the valid point sequence, it finds the acceleration maxima; these points serve as auxiliary information for the landing point criterion, enhancing the accuracy of landing point determination.

[0094] Simultaneously, the algorithm incorporates spatial boundary conditions of the court (such as baselines, sidelines, and service areas) to spatially constrain all candidate landing points. Only points within the valid area of ​​the court are considered true landing points; points outside the boundaries are automatically eliminated. For each candidate landing point, the algorithm integrates multiple features, including extreme height, velocity mutations, extreme acceleration, and spatial constraints, using a multi-feature fusion approach to calculate a confidence score (e.g., selecting the lowest score). Only points with a confidence score exceeding a preset threshold are ultimately determined as actual landing points. If multiple candidate points simultaneously meet the conditions within the same timeframe, the algorithm selects the point with the highest confidence score as the final landing point.

[0095] The algorithm also incorporates a multi-frame fusion strategy. When landing points are detected within several consecutive frames, the final landing point result is only output if these landing points are highly consistent in terms of spatial location, time, and velocity. For occasional misjudgments, such as when the ball is occluded or the detection is lost, the algorithm automatically delays the output, waiting for subsequent frames to supplement the data, thereby minimizing misjudgments and missed judgments. The landing point detection algorithm provided in this embodiment achieves accurate determination of the actual landing point by deeply analyzing the ball's three-dimensional trajectory data and comprehensively considering various dynamic features such as height changes, velocity changes, and acceleration changes.

[0096] Once the actual landing point is detected, the algorithm immediately updates the current landing point status and records information such as the spatial coordinates of the landing point, the detection frame number, the detection confidence level, and the ball speed into the landing point history queue. Each landing point record includes detailed information such as the starting point (e.g., the service point or the previous hitting point), the ending point (the current landing point), the detection time, the ball speed, the ruling status (e.g., In / Out / OffNet), and the confidence level, facilitating subsequent data statistics, replay, and analysis. The landing point detection results are transmitted in real time to subsequent modules such as boundary / inbound judgment and automatic scoring. Based on the relationship between the spatial coordinates of the landing point and the court boundaries, the algorithm automatically determines the ball's boundary / inbound status and, combined with the service status and rally information, completes automatic scoring and data visualization. All rulings can be displayed in real time on the terminals of referees, spectators, and athletes, greatly improving the fairness and entertainment value of the game.

[0097] S5: Dynamically calculate the spatial coordinate range of the service area based on the coordinates of the service starting point and the reference point of the tennis court, and distinguish the service area boundary for singles or doubles modes based on the spatial coordinate range;

[0098] Specifically, this embodiment needs to verify whether the serve landing point is within the valid service area, that is, to dynamically calculate the spatial coordinate range of the service area based on the court reference point, and to distinguish the boundary between singles and doubles modes.

[0099] S6: Determine the landing point of the tennis ball based on the coordinates of the serve landing point and the service area boundary in singles or doubles mode;

[0100] Specifically, this step requires comparing the three-dimensional coordinates (x, y, z) of the landing point with the predefined service area boundary to determine whether the landing point is within the specified area. This includes:

[0101] S601: Determine whether the coordinates of the serve landing point are within the service area boundary in singles or doubles mode;

[0102] Specifically, in this embodiment, the service half zone is identified based on the coordinates of the service point, and the court zone to which the landing point belongs is determined, and it is checked whether it is within the allowed height and horizontal boundaries of the service zone.

[0103] S602: If the coordinates of the serve landing point are within the service area boundary, the landing point status is determined to be valid;

[0104] S603: If the coordinates of the serve landing point are not within the service area boundary or land on the net, the landing point status is determined to be invalid.

[0105] This step also requires combining multiple frames of data to smooth the landing point position during the verification process, reducing misjudgments caused by detection errors and ensuring the accuracy of the judgment.

[0106] S7: Score the server or receiver based on the landing point and pre-set scoring rules.

[0107] Specifically, the scoring rules are as follows: if the serve lands within the valid service area (In) and the receiver does not receive the ball, the serving side scores; if the serve lands outside the service area (Out) or in the net (OffNet) after the serve, the receiver scores.

[0108] The scoring is updated and output through a state-locking mechanism. Compared to traditional manual judging or two-dimensional video replay, this method significantly improves the smoothness and fairness of the game through three-dimensional trajectory and multi-frame verification. The automated scoring logic eliminates human error and is suitable for fast serves and complex rally scenarios.

[0109] This embodiment provides a real-time tennis serve scoring method that integrates serve detection and landing point analysis to achieve automated judgment. It uses a 10-frame sliding window combined with a 3D trajectory y-axis trend to accurately identify serve motion characteristics; it dynamically defines the service area based on court reference points and compares the landing point's 3D coordinates; it smooths errors using multi-frame data; and it executes scoring logic based on the landing point state, effectively reducing the misjudgment rate of serve recognition. Compared to traditional manual judgment and 2D video replay, this method, relying on 3D data and dynamic boundary calculation, offers high real-time performance while reducing the misjudgment rate, combining accuracy, efficiency, and stability.

[0110] In summary, this embodiment analyzes the y-axis trend of tennis movement using three-dimensional trajectory data. By utilizing the y-axis trend, the starting point and landing point of the serve can be accurately identified, thereby greatly improving the accuracy and efficiency of serve scoring and ensuring the fairness of the match.

[0111] Example 2

[0112] This embodiment provides a real-time tennis serve scoring device, including:

[0113] The video frame acquisition module is used to acquire video frames in parallel from multiple cameras and to time-align the multiple video frames to obtain multiple synchronized frames.

[0114] The 3D coordinate construction module is used to identify the two-dimensional pixel coordinates of the tennis ball in multi-channel synchronous frames using a ball detection algorithm, and to construct the three-dimensional coordinates of the tennis ball in space using a ray intersection method to obtain a continuous 3D coordinate sequence.

[0115] The real-time serve recognition module is used to analyze the continuous three-dimensional coordinate sequence of the tennis ball in space within the sliding window, and calculate the frame interval between the current frame and the previous effective motion based on the tennis ball's motion trend on the y-axis, and determine the coordinates of the serve starting point based on the frame interval.

[0116] The real-time landing point detection module is used to traverse a continuous three-dimensional coordinate sequence and determine the minimum height point or trajectory inflection point based on the continuous three-dimensional coordinate sequence, and determine the coordinates of the serve landing point based on the minimum height point or trajectory inflection point.

[0117] The service boundary determination module is used to dynamically calculate the spatial coordinate range of the service area based on the coordinates of the service starting point and the reference point of the tennis court, and to distinguish the service area boundary in singles or doubles mode based on the spatial coordinate range.

[0118] The landing point detection module is used to determine the landing point status of the tennis ball based on the landing point coordinates and the service area boundary in singles or doubles mode.

[0119] The scoring module is used to score the server or receiver based on the landing point and pre-set scoring rules.

[0120] For details on the implementation of each module in a real-time tennis serve scoring device, please refer to the above description of the limitations of a real-time tennis serve scoring method, which will not be repeated here.

[0121] The technical features of the above embodiments can be combined in any way (as long as there is no contradiction in the combination of these technical features). For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; these embodiments not explicitly written should also be considered to be within the scope of this specification.

Claims

1. A method for real-time tennis serve scoring, characterized by, The method comprises the following steps: Step 1: acquiring video frames in parallel through multiple cameras, and time-aligning the multiple video frames to obtain multiple synchronous frames; Step 2: identifying the two-dimensional pixel coordinates of the tennis ball in the multiple synchronous frames by using a ball detection algorithm, and constructing the three-dimensional coordinates of the tennis ball in space by using a ray intersection method to obtain a continuous three-dimensional coordinate sequence; Step 3: analyzing the continuous three-dimensional coordinate sequence of the tennis ball in space within a sliding window, and calculating the frame interval between the current frame and the last effective motion frame according to the motion trend of the tennis ball on the y-axis, and determining the serving starting point coordinates according to the frame interval; Step 4: traversing the continuous three-dimensional coordinate sequence, and determining the height minimum point or trajectory inflection point according to the continuous three-dimensional coordinate sequence, and determining the serving landing point coordinates according to the height minimum point or trajectory inflection point; Step 5: dynamically calculating the spatial coordinate range of the serving area according to the serving starting point coordinates and the reference point of the tennis court, and distinguishing the serving area boundary of the singles or doubles mode according to the spatial coordinate range; Step 6: determining the landing point state of the tennis ball according to the serving landing point coordinates and the serving area boundary of the singles or doubles mode; Step 7: scoring the serving side or the return serving side according to the landing point state and the pre-set scoring rule; Step 3 calculates the y-axis trend in the buffer area based on the motion characteristics of the tennis serving, counts the number of frames in which the y-axis continuously rises or falls within the window, and if the proportion exceeds a pre-set threshold, it is considered that effective motion has occurred; Step 3 specifically comprises: Step 301: analyzing the continuous three-dimensional coordinates of the tennis ball in space within a sliding window, and determining the motion trend of the tennis ball on the y-axis; Step 302: if the motion trend of the tennis ball on the y-axis has no obvious change, then wait for the next frame; if the motion trend of the tennis ball on the y-axis is continuously rising or continuously falling, then calculate the frame interval between the current frame and the last effective motion frame; Step 303: determine whether the frame interval is greater than the effective motion interval threshold; Step 304: if it is less than the effective motion interval threshold, then update the frame number and subscript of the last effective motion, and wait for the next frame; if it is greater than the effective motion interval threshold, then determine that it is a new serving, and record the serving starting point coordinates.

2. The real-time tennis serve scoring method of claim 1, wherein, In step 1, when time-aligning the multiple video frames, specifically including: comparing the time stamps of each frame, and selecting the frames with time stamp intervals within the set allowable error range for synchronous time alignment.

3. The real-time tennis serve scoring method of claim 1, wherein, In step 2, the three-dimensional coordinates of the tennis ball in space are constructed by the ray intersection method, specifically including: converting the two-dimensional pixel coordinates into rays in three-dimensional space by using the pre-calibrated camera parameters, calculating the intersection points of the rays corresponding to each camera in pairs, and screening out the effective intersection points as three-dimensional reconstruction points by using a distance threshold, and recording the three-dimensional coordinates of the three-dimensional reconstruction points.

4. The real-time tennis serve scoring method of claim 1, wherein, In step 301, when determining the motion trend of the tennis ball on the y-axis, specifically including: Step 3011: obtaining the continuous three-dimensional coordinates of the tennis ball in space within a sliding window, and extracting the y-axis coordinates of all effective points; Step 3012: judge whether the number of y-axis coordinates of all valid points is greater than the first threshold value; if less than or equal to the first threshold value, it is judged that the tennis ball has no obvious movement trend in the y-axis; if greater than the first threshold value, the difference value of the continuous y-axis coordinates is calculated, and the rising count and the falling count are counted; Step 3013: judge whether the rising count is greater than the falling count; Step 3014: if the rising count is greater than the falling count, calculate the rising count ratio, and judge whether the rising count ratio is greater than or equal to the second threshold value; if less than the second threshold value, it is judged that the tennis ball has no obvious movement trend in the y-axis; if greater than or equal to the second threshold value, it is judged that the tennis ball is continuously rising in the y-axis; Step 3015: if the rising count is less than or equal to the falling count, judge whether the falling count is greater than the rising count; if the falling count is greater than the rising count, calculate the falling count ratio; Step 3016: judge whether the falling count ratio is greater than or equal to the third threshold value; if less than the third threshold value, it is judged that the tennis ball has no obvious movement trend in the y-axis; if greater than or equal to the third threshold value, it is judged that the tennis ball is continuously falling in the y-axis.

5. The real-time tennis serve scoring method of claim 1, wherein, Step 4 specifically includes: Step 401: traverse the continuous three-dimensional coordinate sequence, and calculate the height value of each point; Step 402: compare the height value of each point with the height values of the adjacent points before and after, and determine whether the height of the intermediate point is a height minimum value or whether the trajectory has a turning point; Step 403: if the height of the intermediate point is a height minimum value or the trajectory has a turning point, the intermediate point is added to the landing point candidate list; Step 404: select the candidate point with the lowest height value in the landing point candidate list as the landing point, and record the landing point coordinates to obtain the serving landing point coordinates.

6. The real-time tennis serve scoring method of claim 5, wherein, Further includes: For the candidate points in the landing point candidate list, the height minimum value, the acceleration maximum value, the instantaneous speed, the speed mutation and the space constraint are fused, and the confidence score is calculated, and the actual serving landing point coordinates are determined according to the confidence score.

7. The real-time tennis serve scoring method of claim 1, wherein, Step 6 specifically includes: Step 601: judge whether the serving landing point coordinates are within the serving area boundary in the singles or doubles mode; Step 602: if the serving landing point coordinates are within the serving area boundary, the landing point state is determined to be valid; Step 603: if the serving landing point coordinates are not within the serving area boundary or fall on the net, the landing point state is determined to be invalid.

8. The real-time tennis serve scoring method of claim 1, wherein, In step 7, the scoring rule is specifically: if the serving landing point coordinates are within the valid serving area and the receiving party does not receive the ball, the serving party scores; after the serving party serves, if the serving landing point coordinates are out of bounds or fall below the net, the receiving party scores.

9. A real-time tennis serve scoring apparatus, characterized by, Includes: A video frame acquisition module for acquiring video frames in parallel through multiple cameras and time aligning the multiple video frames to obtain multiple synchronous frames; A three-dimensional coordinate construction module for identifying the two-dimensional pixel coordinates of the tennis ball in the multiple synchronous frames by using a ball detection algorithm, and constructing the three-dimensional coordinates of the tennis ball in space by a ray intersection method to obtain a continuous three-dimensional coordinate sequence; A serving real-time recognition module for analyzing the continuous three-dimensional coordinate sequence of the tennis ball in space in a sliding window, and calculating the frame interval between the current frame and the last valid movement frame according to the movement trend of the tennis ball in the y-axis, and determining the serving starting point coordinates according to the frame interval; The drop point real-time detection module is configured to traverse the continuous three-dimensional coordinate sequence, determine a height minimum value point or a trajectory inflection point according to the continuous three-dimensional coordinate sequence, and determine a serving drop point coordinate according to the height minimum value point or the trajectory inflection point. The serving boundary determination module is configured to dynamically calculate a spatial coordinate range of a serving area according to the serving start point coordinate and the tennis court reference point, and distinguish a serving area boundary of a singles or a doubles mode according to the spatial coordinate range. The drop point state detection module is configured to determine a drop point state of the tennis ball according to the serving drop point coordinate and the serving area boundary of the singles or the doubles mode. The scoring module is configured to score the serving side or the return serving side according to the drop point state and a pre-set scoring rule. The serving real-time recognition module calculates a y-axis variation trend of the three-dimensional point in the buffer area based on the motion characteristics of the tennis serving, counts a frame number of continuous rising or falling of the y-axis in the window, and considers that an effective motion has occurred if the proportion exceeds a pre-set threshold. Specifically, the serving real-time recognition module includes the following steps: analyzing continuous three-dimensional coordinates of the tennis ball in the space in the sliding window, and judging a motion trend of the tennis ball in the y-axis; if the motion trend of the tennis ball in the y-axis has no obvious change, waiting for the next frame; if the motion trend of the tennis ball in the y-axis is continuously rising or continuously falling, calculating a frame interval between the current frame and the last effective motion; judging whether the frame interval is greater than an effective motion interval threshold; if the frame interval is less than the effective motion interval threshold, updating the last effective motion frame number and the subscript, and waiting for the next frame; if the frame interval is greater than the effective motion interval threshold, determining that a new serving is performed, and recording a serving start point coordinate.

Images