A table tennis three-dimensional trajectory analysis method, system, device and storage medium

Abstract

The present application relates to the technical field of computer vision and image detection, and specifically discloses a table tennis three-dimensional trajectory analysis method, system, device and storage medium. The technical scheme of the present application constructs a multi-frame image sliding window and inputs a multi-frame space-time perception network for space-time feature fusion, combines ball table corner point perspective transformation matrix calculation, trajectory buffer smoothing filtering and motion direction change amount to detect the hitting point, uses opposite side constraint and height change trend to search for the landing point, and when missing detection occurs, forces completion and re-search, converts the pixel coordinates to the standard table coordinates system, reconstructs the three-dimensional trajectory, and outputs the hitting side, landing point area and line type and other structured semantics, solves the problems of insufficient detection continuity caused by high-speed small target shielding under monocular camera video input conditions, unstable hitting and landing point event recognition, missing detection result completion loss and lack of three-dimensional trajectory training semantics, and improves the usability of table tennis three-dimensional trajectory analysis in training review scenarios.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image detection technology, and in particular to a method, system, device and storage medium for three-dimensional trajectory analysis of a ping-pong ball. Background Technology

[0002] Three-dimensional trajectory analysis in table tennis is a crucial foundation for technical and tactical research, training evaluation, and competition data services. By quantitatively recording the hitting position, landing area, and flight path of each shot in a match, coaches and athletes can more intuitively assess the quality of technical execution and the effectiveness of tactical choices, providing data support for the digitalization and scientification of training. However, in live broadcasts or event streams, the small size and high speed of the table tennis ball, coupled with factors such as racket obstruction, human body obstruction, and camera switching, present significant technical challenges for trajectory detection and three-dimensional reconstruction.

[0003] In existing technologies, some solutions employ dual high-speed cameras, binocular vision calibration, or multi-camera image acquisition to obtain spatial information of the sphere, using distributed high-speed vision systems or dedicated synchronous acquisition hardware to achieve 3D positioning and trajectory reconstruction. While these methods can obtain relatively accurate spatial coordinates, they require the additional setup of multiple devices at the competition site, placing high demands on acquisition conditions and hardware synchronization, making them difficult to directly adapt to monocular camera footage in live broadcasts. Other solutions focus on trajectory prediction or landing point coordinate calculation, capable of outputting subsequent coordinates or landing point positions under specific conditions. However, they typically do not integrate occlusion and missed detection, the sequence of hit and landing events, missed detection completion, and training and replay semantic output from the monocular video stream into a complete processing chain.

[0004] In real-world broadcast scenarios, the high-speed movement of the table tennis ball easily leads to image blurring, and occlusion by the racket and players can cause the target to disappear temporarily. Furthermore, camera movement or image switching can easily interrupt the trajectory. Existing 3D information acquisition methods based on single video streams, while capable of establishing projection transformation matrices and fitting the ball's trajectory, do not adequately consider the continuity of ball detection under occlusion, the event state switching between the hitting point and landing point, and the completion of results after missed detections. This limits the usability of the output results in training and review scenarios. Moreover, existing solutions primarily focus on trajectory recording or coordinate output, failing to integrate structured semantics such as the hitting party, the division of the left and right halves of the table, and line types into the analysis results. This makes it difficult to directly serve the coach's needs for tactical statistics and the athlete's technical profiling.

[0005] Therefore, how to achieve robust detection of occlusion of high-speed small targets, stable recognition of hitting and landing events, effective completion of missed detection results, and structured output of 3D trajectory and training semantics, while relying solely on video input from a regular monocular camera, has become a pressing technical problem in the field of table tennis motion analysis. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method, system, device, and storage medium for analyzing the three-dimensional trajectory of a ping-pong ball.

[0007] Firstly, the present invention provides a method for analyzing the three-dimensional trajectory of a table tennis ball, the technical solution of which is as follows: Get consecutive image frames from the video stream and construct a sliding window containing multiple frames in chronological order; The sliding window is input into a multi-frame spatiotemporal perception network. The multi-frame spatiotemporal perception network performs two-dimensional spatial convolution on each image frame, and then performs three-dimensional temporal convolution on the convolved multi-frame features. After upsampling and feature concatenation by the encoder-decoder, a single-channel two-dimensional heat map is output by the temporal compression layer. The pixel coordinates of the target sphere in the frame to be analyzed are obtained by Softmax. Obtain the corner points of the pool table in the image, sort the corner points in a preset order, and calculate the perspective transformation matrix based on the sorted corner points and the standard corner point coordinates of the standard pool table coordinate system. The pixel coordinates of the target ball in multiple consecutive frames are written into the trajectory cache. The trajectory points in the trajectory cache are smoothed and filtered. The change in motion direction is calculated based on the adjacent trajectory points after smoothing and filtering. When the change in motion direction exceeds a preset threshold and the frame interval between the current frame and the previous hitting point is greater than a preset minimum interval, the trajectory point corresponding to the current frame is marked as the hitting point. After detecting the hitting point, the system switches to the landing point search state, filters trajectory points located within the table area from the trajectory cache, and determines the landing point based on the opposite constraint between the hitting point position and the candidate landing point position and the height change trend of the trajectory point. If no landing point is detected within a preset number of frames, the system forces a search for a landing point in the trajectory segment after the hitting point according to the opposite constraint and the lowest point constraint, and re-searches for a landing point in the trajectory segment between the two hitting points when the next hitting point is detected. Based on the perspective transformation matrix, the pixel coordinates of the target ball, the pixel coordinates of the striking point, and the pixel coordinates of the landing point are converted into planar coordinates in the standard table tennis coordinate system. The three-dimensional trajectory of the target ball in the standard table coordinate system is reconstructed based on the planar coordinates, video frame rate, and gravitational acceleration. The striking party, landing area, and trajectory type are determined based on the position of the striking point and landing point, and the structured trajectory analysis results are output.

[0008] The beneficial effects of the three-dimensional trajectory analysis method for table tennis balls proposed in this invention are as follows: The method of this invention constructs a multi-frame image sliding window and inputs it into a multi-frame spatiotemporal perception network for spatiotemporal feature fusion. It combines table corner perspective transformation matrix calculation, trajectory buffer smoothing filtering, and motion direction change detection of the hitting point. It uses the opposite side constraint and height change trend to search for the landing point and forces completion and re-search when there is a missed detection. After converting the pixel coordinates to the standard table coordinate system, it reconstructs the three-dimensional trajectory and outputs structured semantics such as the hitting party, landing area, and line type. It solves the problems of insufficient detection continuity caused by high-speed small target occlusion under monocular camera video input conditions, unstable recognition of hitting and landing events, missing results after missed detection, and lack of training semantics for three-dimensional trajectory. It improves the usability of table tennis three-dimensional trajectory analysis in training and review scenarios.

[0009] Based on the above scheme, the three-dimensional trajectory analysis method for table tennis ball of the present invention can be further improved as follows.

[0010] In one alternative approach, the multi-frame spatiotemporal perception network synchronously applies at least one of scaling, cropping, rotation, flipping, and color perturbation to consecutive frames during the training phase, sets visibility-weighted loss weights for randomly occluded samples, uses Gaussian soft labels as supervision signals for completely occluded samples, and incorporates velocity consistency loss and acceleration consistency loss to constrain the motion continuity of the predicted trajectory.

[0011] The beneficial effects of adopting the above optional methods are as follows: by simultaneously applying multiple image enhancements during the training phase, setting a visibility-weighted loss, using Gaussian soft label supervision, and adding velocity and acceleration consistency loss, the network's adaptability to occluded scenes is enhanced, the continuity of ball detection is improved, and the motion smoothness of the predicted trajectory is constrained.

[0012] In one alternative approach, the encoder of the multi-frame spatiotemporal sensing network saves spatial and temporal features during downsampling, and the decoder splices the corresponding spatial and temporal features during upsampling; the temporal compression layer uses N×1×1 convolution to compress the multi-frame features into a single-channel two-dimensional heatmap.

[0013] The advantages of adopting the above optional method are as follows: by saving spatial and temporal features during encoder downsampling, splicing the corresponding features during decoder upsampling, and using N×1×1 convolution to compress multi-frame features, the integrity of spatiotemporal information is preserved, the accuracy of heatmap output is improved, and pixel coordinate positioning is more accurate.

[0014] In one alternative approach, the step of smoothing the trajectory points in the trajectory buffer includes: smoothing the trajectory points in the trajectory buffer using a Savitzky-Golay filter.

[0015] The advantages of using the above-mentioned optional method are as follows: by further smoothing the trajectory points in the trajectory buffer using the Savitzky-Golay filter, noise interference is suppressed while preserving the original trajectory motion trend, making the calculation of subsequent motion direction changes more stable, thereby improving the reliability of the hitting point detection.

[0016] In one alternative approach, the step of marking the trajectory point corresponding to the current frame as the hitting point includes: The trajectory point corresponding to the current frame is used as a candidate hitting point. The position of the racket in the image is obtained, and the position of the candidate hitting point is corrected according to the position of the racket. For the first candidate hitting point detected at the beginning of the video segment, the first candidate hitting point is corrected based on the nearest racket center position, and the corrected candidate hitting point is marked as the hitting point.

[0017] The advantages of adopting the above-mentioned optional method are as follows: further position correction of candidate hitting points by obtaining racket position, and correction of the first candidate hitting point in the beginning segment of the video based on the nearest racket center position, reducing the misjudgment of hitting point caused by trajectory jitter or detection deviation, and making the hitting point mark closer to the actual hitting position.

[0018] In one alternative approach, the contralateral constraint is that the landing point is located in the half-table area opposite to the hitting point; the height change trend of the trajectory point is determined by the position change of the trajectory point in the vertical direction in the image coordinate system, and the trajectory point in the trajectory descent phase is preferentially selected as the landing point.

[0019] The advantages of adopting the above-mentioned optional method are as follows: by further utilizing the constraint that the landing point is located on the opposite side of the half-table area of ​​the hitting point, and combining the vertical position change of the trajectory point in the image coordinate system to determine the height change trend, the trajectory point in the trajectory descent phase is preferentially selected as the landing point, so that the landing point recognition is more in line with the physical laws of table tennis.

[0020] In one alternative approach, the step of forcing the search for the landing point in the trajectory segment after the point of impact according to the opposite-side constraint and the lowest point constraint includes: In the trajectory segment after the hitting point, select trajectory points that satisfy the opposite side constraint. According to the lowest point constraint, select the trajectory point with the smallest vertical position coordinate value in the image coordinate system from the trajectory points that satisfy the opposite side constraint as the landing point. The striking party, landing area, and trajectory type are determined based on the planar coordinates of the landing point and the striking point in the standard table coordinate system.

[0021] The advantages of adopting the above-mentioned optional method are as follows: by forcing the search for the landing point in the trajectory segment after the hitting point according to the opposite side constraint and the lowest point constraint in the case of missed detection, and determining the hitting party, landing point area and line type according to the planar coordinates under the standard table coordinate system, the missing landing point information caused by missed detection is made up for, and the complete output of structured trajectory analysis results is guaranteed.

[0022] Secondly, this invention provides a three-dimensional trajectory analysis system for table tennis balls, the technical solution of which is as follows: The module is used to acquire consecutive image frames from the video stream and construct a sliding window containing multiple frames in chronological order. The processing module is used to input the sliding window into a multi-frame spatiotemporal perception network, perform two-dimensional spatial convolution on each image frame through the multi-frame spatiotemporal perception network, and then perform three-dimensional temporal convolution on the convolved multi-frame features. After upsampling and feature concatenation by the encoder-decoder, a single-channel two-dimensional heat map is output by the temporal compression layer, and the pixel coordinates of the target sphere in the frame to be analyzed are obtained by Softmax. The calculation module is used to obtain the corner points of the table in the image, sort the corner points in a preset order, and calculate the perspective transformation matrix based on the sorted corner points and the standard corner point coordinates of the standard table coordinate system. The marking module is used to write the pixel coordinates of the target ball in multiple consecutive frames into the trajectory cache, perform smoothing filtering on the trajectory points in the trajectory cache, calculate the change in motion direction based on the adjacent trajectory points after smoothing filtering, and mark the trajectory point corresponding to the current frame as the hitting point when the change in motion direction exceeds a preset threshold and the frame interval between the current frame and the previous hitting point is greater than a preset minimum interval. The search module is used to switch to landing point search mode after detecting the hitting point, filter trajectory points located within the table area from the trajectory cache, and determine the landing point based on the opposite constraint between the hitting point position and the candidate landing point position and the height change trend of the trajectory point; if no landing point is detected within a preset number of frames, the module forces the search for landing point in the trajectory segment after the hitting point according to the opposite constraint and the lowest point constraint, and re-searches for landing point in the trajectory segment between the two hitting points when the next hitting point is detected; The conversion module is used to convert the pixel coordinates of the target ball, the pixel coordinates of the hitting point, and the pixel coordinates of the landing point into planar coordinates in the standard table tennis coordinate system according to the perspective transformation matrix. The output module is used to reconstruct the three-dimensional trajectory of the target ball in the standard table coordinate system based on the planar coordinates, video frame rate, and gravitational acceleration in the standard table coordinate system, and to determine the striking party, landing area, and trajectory type based on the position of the striking point and landing point, and output the structured trajectory analysis results.

[0023] The beneficial effects of the three-dimensional trajectory analysis system for table tennis ball of the present invention are as follows: The system of this invention constructs a multi-frame image sliding window and inputs it into a multi-frame spatiotemporal perception network for spatiotemporal feature fusion. It combines table tennis corner perspective transformation matrix calculation, trajectory buffer smoothing filtering, and motion direction change detection of the hitting point. It uses the opposite side constraint and height change trend to search for the landing point and forces completion and re-search when there is a missed detection. After converting the pixel coordinates to the standard table tennis coordinate system, it reconstructs the three-dimensional trajectory and outputs structured semantics such as the hitting party, landing area, and line type. It solves the problems of insufficient detection continuity caused by high-speed small target occlusion under monocular camera video input conditions, unstable recognition of hitting and landing events, missing results after missed detection, and lack of training semantics for three-dimensional trajectory. It improves the usability of table tennis three-dimensional trajectory analysis in training and review scenarios.

[0024] Thirdly, the technical solution of an electronic device according to the present invention is as follows: It includes a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the ping-pong ball three-dimensional trajectory analysis method of the present invention.

[0025] Fourthly, the technical solution of a computer-readable storage medium provided by the present invention is as follows: The computer-readable storage medium stores instructions that, when read, cause the computer-readable storage medium to perform the steps of the ping-pong ball three-dimensional trajectory analysis method of the present invention.

[0026] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0027] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating an embodiment of a three-dimensional trajectory analysis method for table tennis balls according to the present invention. Figure 2 This is a schematic diagram of the overall process; Figure 3 This is a schematic diagram of the structure of a multi-frame spatiotemporal sensing network; Figure 4 A schematic diagram illustrating the mapping relationship between perspective calibration, normalization transformation, and region division of the table corners; Figure 5 This is a flowchart illustrating the logic for detecting the point of impact and the point of impact. Figure 6 This is a schematic diagram of an embodiment of a three-dimensional trajectory analysis system for table tennis according to the present invention; Figure 7 This is a schematic diagram of an embodiment of an electronic device according to the present invention. Detailed Implementation

[0028] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0029] Figure 1 This diagram illustrates a flowchart of an embodiment of a three-dimensional trajectory analysis method for table tennis provided by the present invention. This method can be executed by electronic devices such as terminal devices or servers. The terminal device can be any fixed or mobile terminal, such as user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, or wearable device. The server can be a single server or a server cluster consisting of multiple servers. Any electronic device can implement the three-dimensional trajectory analysis method for table tennis by having its processor call computer-readable instructions stored in its memory. Figure 1 As shown, it includes the following steps: S1. Obtain consecutive image frames from the video stream and construct a sliding window containing multiple frames in chronological order.

[0030] In this context, a video stream refers to a continuous sequence of images captured in chronological order. For example, in a television broadcast of a table tennis match, cameras capture the playing field in real time, and each frame is arranged in chronological order of capture time to form continuous video data from the start to the end of the match. A continuous image frame refers to each independent still image that constitutes the video stream. For example, extracting the frame corresponding to the 100th millisecond from a table tennis match broadcast might record the spatial positions of player A, player B, and the table tennis ball at that moment.

[0031] The sliding window refers to a sampling method that selects multiple consecutive frames of images in a fixed length over time. For example, in a live broadcast of a match, 32 consecutive frames of images are selected from the current time as the end point of the window, and these 32 frames are used as a whole to input into the subsequent processing network.

[0032] S2. The sliding window is input into a multi-frame spatiotemporal perception network. The multi-frame spatiotemporal perception network performs two-dimensional spatial convolution on each image frame, and then performs three-dimensional temporal convolution on the convolved multi-frame features. After upsampling and feature concatenation by the encoder-decoder, a single-channel two-dimensional heatmap is output by the time compression layer. The pixel coordinates of the target sphere in the frame to be analyzed are obtained by Softmax.

[0033] Among them, the multi-frame spatiotemporal perception network refers to a deep neural network structure used to simultaneously extract spatial appearance information and temporal motion information from an image sequence and output the probability distribution of the target position. For example, after inputting 32 frames of images into the network, the network extracts the appearance features of the ping-pong ball in each frame through convolution operations, and captures the motion trajectory features of the ping-pong ball within the 32-frame time span through temporal convolution, and finally outputs the probability distribution of the ping-pong ball's position in the image at the current moment.

[0034] Two-dimensional spatial convolution refers to local feature extraction operations performed on the height and width dimensions of a single frame of an image. For example, in each of 32 frames of images, a 3×3 convolution kernel slides across the image plane to extract the local texture and edge features of the ping-pong ball and its surrounding area. Three-dimensional temporal convolution refers to feature extraction operations performed simultaneously on the height, width, and time dimensions of an image. For example, the feature maps obtained after two-dimensional spatial convolution of 32 frames of images are stacked in temporal order, and a 3×3×3 three-dimensional convolution kernel slides simultaneously in the height, width, and time directions to capture the motion trajectory and speed change features of the ping-pong ball across multiple frames.

[0035] Upsampling and feature concatenation refer to the operation of restoring a low-resolution feature map to a high-resolution feature map and merging it with the feature map of the corresponding size in the encoding stage. For example, the low-resolution feature map output by the encoder in a multi-frame spatiotemporal perception network is enlarged by interpolation and then merged with the high-resolution feature map stored in the corresponding layer of the decoder in the channel dimension, thereby preserving the detailed information of the ping-pong ball's position.

[0036] The temporal compression layer refers to a network layer that aggregates feature maps from multiple frames along the temporal dimension to output a single-frame heatmap. For example, 32 feature maps output from a multi-frame spatiotemporal awareness network decoder are weighted and fused along the temporal dimension using a 1×1 convolutional kernel, compressing the information from 32 frames into a single feature map. A single-channel two-dimensional heatmap refers to a two-dimensional probability distribution map containing only one channel. The value of each pixel in the map represents the probability of the target ball appearing at that location. For example, the network outputs a grayscale image of the same size as the input image; higher pixel values ​​indicate a greater probability of the ping-pong ball appearing at that location, while pixel values ​​close to 0 indicate a less likely ping-pong ball appearing at that location.

[0037] Here, the frame to be analyzed refers to the current image frame in the sliding window where target detection and position output are performed; for example, in a 32-frame sliding window, the last frame is used as the frame to be analyzed, and the network outputs the probability distribution of the ping-pong ball's position in the frame. The target ball refers to the ping-pong ball to be detected and tracked in the video frame; for example, in a match broadcast, a 40mm diameter ping-pong ball is the object to be detected and tracked by the analysis method. Pixel coordinates refer to the position of the target ball in the image, represented by the row and column numbers of the image; for example, after the heatmap output by the network is processed by Softmax, the position corresponding to the maximum probability is taken, resulting in the pixel position of the ping-pong ball in the frame to be analyzed as row number 360 and column number 640.

[0038] S3. Obtain the corner points of the table in the image, sort the corner points in a preset order, and calculate the perspective transformation matrix based on the sorted corner points and the standard corner point coordinates of the standard table coordinate system.

[0039] In this context, "table tennis table" refers to a standard table used in table tennis matches, including the table surface, sidelines, end lines, and net. For example, in a match video, a green or blue rectangular table has a table surface measuring 2.74m long and 1.525m wide, divided into two halves by a central net. "Corner points" refers to the vertices of the table in an image; for example, in a match video, the pixel positions of the top-left, top-right, bottom-right, and bottom-left corners of the table.

[0040] The preset order refers to the order rules set in advance when sorting the corner points; for example, arranging the four corner points in a clockwise order of top left, top right, bottom right, and bottom left.

[0041] The standard table tennis table coordinate system refers to a two-dimensional plane coordinate system established with reference to a standard table tennis table. For example, it is a plane coordinate system with the top left corner of the table as the origin, the long side of the table as the X-axis, and the short side of the table as the Y-axis, with units of meters (m). Standard corner coordinates refer to the coordinate values ​​corresponding to the four corner points in the standard table tennis table coordinate system. For example, in the standard table tennis table coordinate system, the top left corner coordinates are (0,0), the top right corner coordinates are (2.74,0), the bottom right corner coordinates are (2.74,1.525), and the bottom left corner coordinates are (0,1.525).

[0042] The perspective transformation matrix refers to the projection transformation parameter matrix that maps a point in the image pixel coordinate system to the corresponding point in the standard table tennis coordinate system. For example, based on the pixel coordinates of the four corner points in the image and their corresponding coordinates in the standard table tennis coordinate system, a 3×3 transformation matrix is ​​calculated, and the matrix is ​​used to transform any pixel in the image to the standard table tennis coordinate system.

[0043] S4. Write the pixel coordinates of the target ball in multiple consecutive frames into the trajectory cache, perform smoothing filtering on the trajectory points in the trajectory cache, calculate the change in motion direction based on the adjacent trajectory points after smoothing filtering, and mark the trajectory point corresponding to the current frame as the hitting point when the change in motion direction exceeds a preset threshold and the frame interval between the current frame and the previous hitting point is greater than a preset minimum interval.

[0044] The trajectory buffer refers to a fixed-length queue used to store the pixel coordinates of target balls across multiple consecutive frames. For example, maintaining a queue of 64 frames, whenever the pixel coordinates of a new ping-pong ball are detected, the coordinates are added to the tail of the queue, and the coordinates of the oldest frame at the head of the queue are removed. Adjacent trajectory points after smoothing filtering refer to the positions at two consecutive time points obtained after filtering the coordinate sequence in the trajectory buffer. For example, after processing the pixel coordinate sequence of 64 consecutive frames of ping-pong balls stored in the trajectory buffer using a filtering algorithm, two adjacent points are obtained: (358, 642) in frame t and (355, 638) in frame t+1.

[0045] The change in motion direction refers to the change in the angle of motion direction between adjacent trajectory points. For example, the motion direction is calculated based on the adjacent trajectory points after smoothing and filtering. The motion direction angle from frame t to frame t+1 is 120 degrees, and the motion direction angle from frame t+1 to frame t+2 becomes 150 degrees, with a change in direction of 30 degrees.

[0046] The preset threshold refers to a pre-set critical angle value used to determine whether the direction of motion has changed significantly. For example, if the threshold for the change in direction is set to 45 degrees, when the calculated change in the direction of motion exceeds 45 degrees, it is determined that the ping-pong ball has undergone a significant change in direction. The preset minimum interval refers to a pre-set minimum number of frames allowed between two hitting events. For example, if the minimum interval is set to 3 frames, if the current frame is only 1 frame away from the last marked hitting point, the frame will not be marked as a new hitting point.

[0047] The point of impact refers to the position of the ping-pong ball when the racket collides with it. For example, in a match, when athlete A swings the racket to hit the ball, the position of the ping-pong ball in the image frame corresponding to the instant the ball bounces off the racket surface is the point of impact for that shot.

[0048] S5. After detecting the hitting point, switch to the landing point search state, filter the trajectory points located in the table area from the trajectory cache, and determine the landing point according to the opposite constraint between the hitting point position and the candidate landing point position and the height change trend of the trajectory point; if no landing point is detected within a preset number of frames, force the search for landing point in the trajectory segment after the hitting point according to the opposite constraint and the lowest point constraint, and re-search for landing point in the trajectory segment between the two hitting points when the next hitting point is detected.

[0049] The landing point search state refers to a working mode specifically for searching the landing point after the hitting point is detected. For example, after a hit is detected in the 100th frame, the working state is switched to the landing point search state, and the position of the ping-pong ball landing on the table is continuously monitored and searched in subsequent frames.

[0050] The striking point location refers to the spatial position of the striking point in the image coordinate system or the standard table tennis table coordinate system. For example, in the image frame corresponding to the striking point, the pixel position of the ping-pong ball in the image is row number 320 and column number 512; this position is the striking point location. The candidate landing point location refers to the position of a trajectory point that satisfies some constraints during the landing point search process. For example, in the landing point search state, five trajectory points located within the table tennis table area are selected from the trajectory cache, and their positions are used as candidate landing point locations.

[0051] Among these, the contralateral constraint refers to the restriction that the landing point must be located within the area of ​​the opposite half of the table where the hitting point is located; for example, if the hitting point is located on the left half of the table, the landing point must be searched within the right half of the table and is not allowed to land in the left half. The height change trend refers to the direction of change of the position of the trajectory point in the vertical direction of the image over time; for example, continuously monitoring the vertical coordinate values ​​of the trajectory point in the image, if the vertical coordinate values ​​continue to increase, it indicates that the ping-pong ball is in the descending phase. The landing point refers to the contact point of the ping-pong ball on the table surface; for example, in a match, if the ping-pong ball flies over the net and lands on the right half of the table near the end line, that position is the landing point of the next shot.

[0052] The minimum point constraint refers to the restriction that the landing point must be the lowest point in the vertical direction within the trajectory segment. For example, in the trajectory segment after the point of impact, the arc of the ping-pong ball reaches its lowest position at the landing point, which is the point with the smallest vertical coordinate value in the trajectory segment. A trajectory segment refers to a continuous trajectory from the point of impact to the next point of impact; for example, from the point of impact in frame 100 to the next point of impact in frame 150, the ping-pong ball positions detected continuously for 50 frames constitute a complete trajectory segment.

[0053] S6. Based on the perspective transformation matrix, convert the pixel coordinates of the target ball, the pixel coordinates of the striking point, and the pixel coordinates of the landing point into planar coordinates in the standard table tennis coordinate system.

[0054] Among them, planar coordinates refer to the positional information in the standard table coordinate system that only includes the X and Y axes; for example, after the perspective transformation matrix is ​​mapped, the coordinates of the hitting point in the standard table coordinate system are (1.20, 0.75), which are the planar coordinates of the hitting point.

[0055] S7. Reconstruct the three-dimensional trajectory of the target ball in the standard table coordinate system based on the planar coordinates, video frame rate, and gravitational acceleration. Determine the striking party, landing area, and trajectory type based on the position of the striking point and landing point, and output the structured trajectory analysis results.

[0056] Here, video frame rate refers to the number of image frames contained in a video stream per second; for example, a video frame rate of 50fps in a match broadcast means that 50 consecutive images are captured and displayed per second. Gravitational acceleration refers to the standard value of the acceleration an object experiences under the influence of Earth's gravity; for example, in the process of 3D trajectory reconstruction, a gravitational acceleration value of 9.8 m / s² is used to calculate the vertical trajectory of a ping-pong ball. A 3D trajectory refers to the spatial trajectory in a standard table tennis coordinate system, including the changes in position over time along the X, Y, and Z axes; for example, a ping-pong ball takes off from the hitting point (1.20, 0.75, 0.76), flies through the air, and reaches the landing point (1.80, 0.85, 0), where the Z-axis coordinate represents the height of the ping-pong ball above the ground.

[0057] In this context, "hitting player" refers to the side of the player who makes the hitting motion; for example, it can be determined by whether the hitting point is on the left or right half of the table. If the hitting point is on the left half and the half corresponds to player A, then player A is the hitting player. "Landing area" refers to the spatial range of the landing point within the standard table coordinate system; for example, if the standard table is divided into a nine-square grid, and the landing point falls in the lower center square of the right half, then the landing area is the lower center of the right half. "Path type" refers to the type of path the ping-pong ball takes from the hitting point to the landing point; for example, based on the relative relationship between the hitting point and the landing area, it can be determined whether the shot is a forehand cross-court shot, a backhand straight shot, or a center shot.

[0058] The structured trajectory analysis result refers to an analysis report containing formatted data such as the hitting player, landing area, trajectory type, and three-dimensional trajectory coordinates. For example, the complete information of a single shot is output as: hitting player A, landing area is the lower middle right half of the table, trajectory type is a forehand diagonal line, and the three-dimensional trajectory contains 100 coordinate points.

[0059] The technical solution of this embodiment constructs a multi-frame image sliding window and inputs it into a multi-frame spatiotemporal perception network for spatiotemporal feature fusion. It combines the calculation of the perspective transformation matrix of the table corner, trajectory buffer smoothing filtering, and detection of the hitting point by the change in motion direction. It uses the opposite side constraint and the height change trend to search for the landing point and forces completion and re-search when there is a missed detection. After converting the pixel coordinates to the standard table coordinate system, it reconstructs the three-dimensional trajectory and outputs structured semantics such as the hitting party, landing area, and line type. It solves the problems of insufficient detection continuity caused by high-speed small target occlusion under monocular camera video input conditions, unstable recognition of hitting and landing events, missing results after missed detection, and lack of training semantics for three-dimensional trajectory. It improves the usability of table tennis three-dimensional trajectory analysis in training and review scenarios.

[0060] In one alternative approach, the multi-frame spatiotemporal perception network synchronously applies at least one of scaling, cropping, rotation, flipping, and color perturbation to consecutive frames during the training phase, sets visibility-weighted loss weights for randomly occluded samples, uses Gaussian soft labels as supervision signals for completely occluded samples, and incorporates velocity consistency loss and acceleration consistency loss to constrain the motion continuity of the predicted trajectory.

[0061] The training phase refers to the process of learning parameters for a multi-frame spatiotemporal perception network using labeled training data. For example, a match video containing 10,000 frames of labeled ping-pong ball positions can be used as training data, and the weight parameters of each layer of the network can be adjusted through the backpropagation algorithm.

[0062] Random occlusion samples refer to samples in the training data where occlusions are randomly added, making the target ball partially or completely invisible. For example, in the training data, a rectangular occlusion block is randomly generated to cover the location of the ping-pong ball in some image frames, generating occlusion samples to train the network's detection ability under occlusion conditions. Visibility-weighted loss weights refer to different weight coefficients assigned to the loss function based on the degree to which the target ball is occluded in the image. For example, a loss weight of 1 is assigned to fully visible samples, and a loss weight of 2 is assigned to samples where the occlusion area exceeds 50%, making the network pay more attention to the detection error of occluded samples.

[0063] Here, a fully occluded sample refers to a sample in an image where the target ball is completely covered and invisible; for example, in the training data, when the ping-pong ball is completely obscured by player A's arm, the image frame is labeled as a fully occluded sample. A Gaussian soft label refers to a probability label generated according to a Gaussian distribution centered on the true location of the target ball; for example, for fully occluded samples, instead of hard-coded one-hot labels, a two-dimensional Gaussian distribution with a large variance is generated around the true location of the target ball as a supervision signal. The supervision signal refers to a reference standard used during training to calculate the difference between the network output and the true value; for example, for visible samples, the supervision signal is a peak heatmap at the true location of the ping-pong ball; for occluded samples, the supervision signal is a Gaussian soft label or a loss value weighted by visibility.

[0064] The velocity consistency loss refers to a loss term used to constrain the predicted positions between adjacent frames to conform to the law of uniform motion. For example, during training, the velocity vector between the predicted positions of three adjacent frames is calculated. If the magnitude and direction of the velocity vector change too much in a short period of time, the velocity consistency loss increases. The acceleration consistency loss constraint refers to a loss term used to constrain the predicted positions between adjacent frames to conform to the law of constant acceleration motion. For example, during training, the acceleration vector between adjacent frames is calculated. If the acceleration vector changes abruptly, the acceleration consistency loss increases.

[0065] Among the above optional methods, further improvements are made by simultaneously applying multiple image enhancements during the training phase, setting a visibility-weighted loss, employing Gaussian soft label supervision, and adding velocity and acceleration consistency loss. These improvements enhance the network's adaptability to occluded scenes, improve the continuity of ball detection, and constrain the motion smoothness of the predicted trajectory.

[0066] In one alternative approach, the encoder of the multi-frame spatiotemporal sensing network saves spatial and temporal features during downsampling, and the decoder splices the corresponding spatial and temporal features during upsampling; the temporal compression layer uses N×1×1 convolution to compress the multi-frame features into a single-channel two-dimensional heatmap.

[0067] Spatial features refer to information in the image related to the shape of the target ball; for example, the circular outline, color distribution, and surrounding background texture information of the ping-pong ball saved by the encoder during downsampling. Temporal features refer to the motion pattern information of the target ball in the time dimension; for example, the speed change, motion direction continuity, and trajectory smoothness information of the ping-pong ball saved by the encoder during downsampling.

[0068] In the above optional methods, spatial and temporal features are further preserved during encoder downsampling, and corresponding features are spliced ​​together during decoder upsampling. Multi-frame features are compressed using N×1×1 convolution to preserve the integrity of spatiotemporal information, improve the accuracy of heatmap output, and make pixel coordinate positioning more accurate.

[0069] In one alternative approach, the step of smoothing the trajectory points in the trajectory buffer includes: smoothing the trajectory points in the trajectory buffer using a Savitzky-Golay filter.

[0070] Among them, the Savitzky-Golay filter refers to a data smoothing method based on local polynomial least squares fitting; for example, the Savitzky-Golay filter with a window length of 7 and a polynomial order of 3 is used to smooth the coordinates of 64 consecutive pixels in the trajectory buffer, removing random detection noise while keeping the overall shape of the trajectory unchanged.

[0071] In the above-mentioned optional methods, the trajectory points in the trajectory buffer are further smoothed by using a Savitzky-Golay filter. This preserves the original trajectory motion trend while suppressing noise interference, making the calculation of subsequent motion direction changes more stable, thereby improving the reliability of the hitting point detection.

[0072] In one alternative approach, the step of marking the trajectory point corresponding to the current frame as the hitting point includes: The trajectory point corresponding to the current frame is used as a candidate hitting point. The position of the racket in the image is obtained, and the position of the candidate hitting point is corrected according to the position of the racket.

[0073] For the first candidate hitting point detected at the beginning of the video segment, the first candidate hitting point is corrected based on the nearest racket center position, and the corrected candidate hitting point is marked as the hitting point.

[0074] The video start segment refers to the preset frame interval at the beginning of the video; for example, in the first 30 frames of the video, before the athlete starts the match, this interval is marked as the video start segment. The racket center position refers to the coordinates of the center point of the racket in the image, output by the racket detection algorithm; for example, in a match video, the object detection algorithm detects the bounding rectangle of the area where athlete A holds the racket, and calculates the pixel coordinates of the center point of the rectangle as the racket center position.

[0075] The corrected candidate hitting point refers to the hitting point position obtained after racket position calibration; for example, if the candidate hitting point is moved from pixel coordinates (320, 512) to the nearest racket center position (318, 508), the corrected candidate hitting point is (319, 510).

[0076] In the above-mentioned optional methods, the candidate hitting point is further corrected by obtaining the racket position, and the first candidate hitting point in the beginning segment of the video is corrected based on the nearest racket center position, reducing the misjudgment of the hitting point caused by trajectory jitter or detection deviation, and making the hitting point mark closer to the actual hitting position.

[0077] In one alternative approach, the contralateral constraint is that the landing point is located in the half-table area opposite to the hitting point; the height change trend of the trajectory point is determined by the position change of the trajectory point in the vertical direction in the image coordinate system, and the trajectory point in the trajectory descent phase is preferentially selected as the landing point.

[0078] The opposite half of the table refers to the space on the other half of the table opposite the half where the ball is struck. For example, if the ball is struck in the left half of the table, then all positions on the right half of the table are the opposite half of the table; if the ball is struck in the right half of the table, then all positions on the left half of the table are the opposite half of the table.

[0079] The vertical direction refers to the dimension of positional change along the column direction of the image coordinate system. For example, in the image coordinate system, the vertical coordinate value is the row number of the pixel. A change in the ping-pong ball trajectory point from row number 200 to row number 400 indicates a vertical displacement. The trajectory descent phase refers to the flight phase in which the ping-pong ball trajectory continuously decreases in position in the vertical direction. For example, by continuously monitoring the vertical coordinate values ​​of the trajectory points, when the coordinate value continuously increases from row number 200 to row number 350, it is determined that the ping-pong ball is in the trajectory descent phase.

[0080] In the above-mentioned optional methods, the height change trend is further determined by utilizing the constraint that the landing point is located on the opposite side of the half-table area of ​​the hitting point, combined with the vertical position change of the trajectory point in the image coordinate system. The trajectory point in the descent phase is selected as the landing point, so that the landing point recognition is more in line with the physical laws of table tennis.

[0081] In one alternative approach, the step of forcing the search for the landing point in the trajectory segment after the point of impact according to the opposite-side constraint and the lowest point constraint includes: In the trajectory segment following the hitting point, trajectory points that satisfy the opposite-side constraint are selected. According to the lowest point constraint, the trajectory point with the smallest vertical position coordinate value in the image coordinate system is selected from the trajectory points that satisfy the opposite-side constraint as the landing point.

[0082] The point with the smallest position coordinate value is the point with the smallest vertical coordinate value among all the points in the trajectory segment. For example, if the vertical coordinate values ​​of all the points in the trajectory segment are 320, 340, 360, 380, and 400, and the coordinate value 320 is the smallest, then the point with the smallest position coordinate value is the point with the smallest position coordinate value, which is the lowest point of the trajectory.

[0083] The striking party, landing area, and trajectory type are determined based on the planar coordinates of the landing point and the striking point in the standard table coordinate system.

[0084] In the above-mentioned optional methods, the landing point is further searched in the trajectory segment after the hitting point according to the opposite side constraint and the lowest point constraint in the case of missed detection, and the hitting party, landing point area and line type are determined according to the planar coordinates under the standard table coordinate system. This makes up for the missing landing point information caused by missed detection and ensures the complete output of structured trajectory analysis results.

[0085] Figure 2 The overall processing chain of this embodiment is illustrated. After the competition video is input, consecutive image frames are formed according to the frame rate to maintain a fixed number of frames. A sliding window is constructed as the input for subsequent networks. This method relies solely on the monocular video from the broadcast feed and does not require the deployment of additional acquisition equipment such as dual high-speed cameras, multiple cameras, or distributed high-speed vision systems within the venue.

[0086] Figure 3 The structure of a multi-frame spatiotemporal awareness network is shown. The network input tensor has the following shape: ,in Indicates batch size. Indicates the number of input frames. Indicates the number of image channels. Indicates the image height. This represents the image width. The network first merges the batch dimension with the frame dimension, performing a 2D spatial convolution on each frame to extract the local visual features of the ball. The features are then rearranged into... The algorithm performs 3D temporal convolutions in both the temporal and spatial dimensions, preserving motion cues through residual connections. The encoder saves spatial and temporal features during downsampling, while the decoder upsamples step-by-step and concatenates the corresponding spatial and temporal features. The temporal compression layer employs... Convolution compresses multi-frame features into a single-channel two-dimensional heatmap. After a softmax operation, the output is the probability distribution of the target sphere's position, yielding the pixel coordinates of the target sphere in the frame to be analyzed. The encoding stage preserves both spatial and temporal features, while the decoding stage concatenates these features for sphere position recovery in occluded scenarios.

[0087] During network training, image enhancement operations such as scaling, cropping, rotation, flipping, color perturbation, and random sphere occlusion are applied synchronously to consecutive frames. Different loss weights are set for visible samples, partially occluded samples, and completely occluded samples. For completely occluded samples, Gaussian soft labels with a large standard deviation are used as supervision signals, enabling the network to learn the uncertain probability distribution of the occlusion position. Velocity consistency loss and acceleration consistency loss are also incorporated to constrain the motion continuity of the predicted trajectory. Through training, the network can handle situations where the ball is blurred, occluded by a racket or human, or disappears briefly in the broadcast footage.

[0088] Figure 4 This demonstrates the process of establishing the geometric constraints of the ping-pong table. The four corner points of the table in the image are detected or manually calibrated, ordered in the order of top left, top right, bottom right, and bottom left. A standard ping-pong table coordinate system is established with the plane of the standard table as a reference; the length of the standard table is... Width Calculate the perspective transformation matrix based on the four sorted corner points and the coordinates of the standard corner points in the standard table coordinate system. Perspective view in image pixel coordinate system after... Mapped to a standard view in the standard table tennis coordinate system. Four corner points. to After calibration, it is mapped to a standard corner point. to The trajectory points are determined by the image pixel coordinates. Convert to standard table tennis coordinate system The perspective transformation matrix is ​​used to map points in the image pixel coordinate system to planar coordinates in the standard table tennis coordinate system, and also to determine whether a trajectory point is within the table tennis area. In the standard table tennis coordinate system, the left and right halves of the table are each divided into three columns and three rows, for a total of nine areas.

[0089] In the trajectory buffering and smoothing stage, a fixed-length trajectory buffer is used to store the target sphere pixel coordinates of the most recent few frames. A Savitzky-Golay filter is used to smooth the coordinate sequence in the trajectory buffer, suppressing noise interference while preserving the original trajectory motion trend.

[0090] Figure 5 The logic for detecting the hitting point and landing point is illustrated. During hitting point identification, the angle between the preceding and following motion vectors, the change in direction, and the change in horizontal velocity are calculated based on the smoothed and filtered adjacent trajectory points. A candidate point is identified as the hitting point when it satisfies a sudden change in direction, a reversal of horizontal velocity, and an interval exceeding a preset minimum interval from the previous hitting frame. For the first hitting point detected at the beginning of the video segment, position correction is performed using the racket center closest to the target ball to reduce false detections during the serve phase.

[0091] During landing point recognition, the system switches to landing point search mode after detecting the striking point. It filters trajectory points located within the polygonal area of ​​the table from the trajectory cache, and further filters those located in the opposite half of the table. Trajectory points in the descent phase are prioritized as landing points. If no landing point is detected within a preset number of frames, a forced search is triggered, searching for the landing point position in the trajectory segment after the striking point according to the opposite side constraint and the lowest point constraint. When the next striking point is detected, the trajectory segment between the two striking points is re-analyzed to complete or update the landing point results.

[0092] In the 3D trajectory reconstruction and semantic output stage, the pixel coordinates of the target ball, the hitting point, and the landing point are converted into planar coordinates in the standard table tennis coordinate system based on the perspective transformation matrix. The trajectory height is estimated based on the planar coordinates in the standard table tennis coordinate system, the video frame rate, the net height, and the gravitational acceleration, thus obtaining the 3D trajectory coordinates of the target ball in the standard table tennis coordinate system. The system determines the striking party based on the planar coordinates of the striking point and landing point in the standard table coordinate system, defines the landing area by dividing the table into nine grids on the left and right halves, and outputs the trajectory type based on the relative relationship between the striking position and the landing area, generating structured trajectory analysis results.

[0093] Figure 6 A schematic diagram of an embodiment of a three-dimensional trajectory analysis system 200 for table tennis provided by the present invention is shown. Figure 6 As shown, the ping-pong ball three-dimensional trajectory analysis system 200 includes: Module 201 is used to acquire consecutive image frames in the video stream and construct a sliding window containing multiple frames in chronological order. Processing module 202 is used to input the sliding window into a multi-frame spatiotemporal perception network, perform two-dimensional spatial convolution on each image frame through the multi-frame spatiotemporal perception network, and then perform three-dimensional temporal convolution on the convolved multi-frame features. After upsampling and feature concatenation by encoder-decoder, a single-channel two-dimensional heat map is output by the time compression layer, and the pixel coordinates of the target sphere in the frame to be analyzed are obtained by Softmax. Calculation module 203 is used to obtain the corner points of the table in the image, sort the corner points in a preset order, and calculate the perspective transformation matrix based on the sorted corner points and the standard corner point coordinates of the standard table coordinate system. The marking module 204 is used to write the pixel coordinates of the target ball in multiple consecutive frames into the trajectory cache, perform smoothing filtering on the trajectory points in the trajectory cache, calculate the change in motion direction based on the adjacent trajectory points after smoothing filtering, and mark the trajectory point corresponding to the current frame as the hitting point when the change in motion direction exceeds a preset threshold and the frame interval between the current frame and the previous hitting point is greater than a preset minimum interval. The search module 205 is used to switch to the landing point search state after detecting the hitting point, filter the trajectory points located in the table area from the trajectory cache, and determine the landing point according to the opposite constraint between the hitting point position and the candidate landing point position and the height change trend of the trajectory point; if no landing point is detected within a preset number of frames, the search for the landing point is forced in the trajectory segment after the hitting point according to the opposite constraint and the lowest point constraint, and the landing point is re-searched in the trajectory segment between the two hitting points when the next hitting point is detected; The conversion module 206 is used to convert the pixel coordinates of the target ball, the pixel coordinates of the hitting point, and the pixel coordinates of the landing point into planar coordinates in the standard table tennis coordinate system according to the perspective transformation matrix. The output module 207 is used to reconstruct the three-dimensional trajectory of the target ball in the standard table coordinate system based on the planar coordinates, video frame rate and gravitational acceleration in the standard table coordinate system, and to determine the striking party, landing area and trajectory type based on the position of the striking point and landing point, and output the structured trajectory analysis results.

[0094] In one alternative approach, the multi-frame spatiotemporal perception network synchronously applies at least one of scaling, cropping, rotation, flipping, and color perturbation to consecutive frames during the training phase, sets visibility-weighted loss weights for randomly occluded samples, uses Gaussian soft labels as supervision signals for completely occluded samples, and incorporates velocity consistency loss and acceleration consistency loss to constrain the motion continuity of the predicted trajectory.

[0095] In one alternative approach, the encoder of the multi-frame spatiotemporal sensing network saves spatial and temporal features during downsampling, and the decoder splices the corresponding spatial and temporal features during upsampling; the temporal compression layer uses N×1×1 convolution to compress the multi-frame features into a single-channel two-dimensional heatmap.

[0096] In an alternative embodiment, the marking module 204 is specifically used for: The trajectory points in the trajectory buffer are smoothed using a Savitzky-Golay filter.

[0097] In an alternative embodiment, the marking module 204 is specifically used for: The trajectory point corresponding to the current frame is used as a candidate hitting point. The position of the racket in the image is obtained, and the position of the candidate hitting point is corrected according to the position of the racket. For the first candidate hitting point detected at the beginning of the video segment, the first candidate hitting point is corrected based on the nearest racket center position, and the corrected candidate hitting point is marked as the hitting point.

[0098] In one alternative approach, the contralateral constraint is that the landing point is located in the half-table area opposite to the hitting point; the height change trend of the trajectory point is determined by the position change of the trajectory point in the vertical direction in the image coordinate system, and the trajectory point in the trajectory descent phase is preferentially selected as the landing point.

[0099] In an alternative embodiment, the search module 205 is specifically used for: In the trajectory segment after the hitting point, select trajectory points that satisfy the opposite side constraint. According to the lowest point constraint, select the trajectory point with the smallest vertical position coordinate value in the image coordinate system from the trajectory points that satisfy the opposite side constraint as the landing point. The striking party, landing area, and trajectory type are determined based on the planar coordinates of the landing point and the striking point in the standard table coordinate system.

[0100] It should be noted that the beneficial effects of the ping-pong ball three-dimensional trajectory analysis system 200 provided in the above embodiments are the same as those of the ping-pong ball three-dimensional trajectory analysis method described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0101] The ping-pong ball three-dimensional trajectory analysis system 200 of the present invention can be a computer program (including program code) running on a computer device. For example, the ping-pong ball three-dimensional trajectory analysis system 200 of the present invention is an application software that can be used to execute the corresponding steps in the ping-pong ball three-dimensional trajectory analysis method of the present invention.

[0102] In some embodiments, the ping-pong ball three-dimensional trajectory analysis system 200 of the present invention can be implemented in a combination of hardware and software. As an example, the ping-pong ball three-dimensional trajectory analysis system 200 of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the ping-pong ball three-dimensional trajectory analysis method of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0103] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0104] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned three-dimensional trajectory analysis methods for ping-pong balls. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the three-dimensional trajectory analysis method for ping-pong balls shown in any embodiment of the present invention by calling the computer program.

[0105] In one alternative embodiment, an electronic device is provided, such as Figure 7 As shown, Figure 7 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0106] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0107] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0108] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0109] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0110] Among them, electronic devices can also be terminal devices. A terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0111] It should be noted that, Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0112] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described methods for analyzing the three-dimensional trajectory of a ping-pong ball.

[0113] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0114] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned ping-pong ball three-dimensional trajectory analysis method.

[0115] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0116] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0117] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0118] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0119] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0120] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0121] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0122] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for analyzing the three-dimensional trajectory of a table tennis ball, characterized in that, include: Acquire consecutive image frames from the video stream and construct a sliding window containing multiple frames in chronological order; The sliding window is input into a multi-frame spatiotemporal perception network. The multi-frame spatiotemporal perception network performs two-dimensional spatial convolution on each image frame, and then performs three-dimensional temporal convolution on the convolved multi-frame features. After upsampling and feature concatenation by the encoder-decoder, a single-channel two-dimensional heat map is output by the temporal compression layer. The pixel coordinates of the target sphere in the frame to be analyzed are obtained by Softmax. Obtain the corner points of the pool table in the image, sort the corner points in a preset order, and calculate the perspective transformation matrix based on the sorted corner points and the standard corner point coordinates of the standard pool table coordinate system. The pixel coordinates of the target ball in multiple consecutive frames are written into the trajectory cache. The trajectory points in the trajectory cache are smoothed and filtered. The change in motion direction is calculated based on the adjacent trajectory points after smoothing and filtering. When the change in motion direction exceeds a preset threshold and the frame interval between the current frame and the previous hitting point is greater than a preset minimum interval, the trajectory point corresponding to the current frame is marked as the hitting point. After detecting the hitting point, the system switches to the landing point search state, filters trajectory points located within the table area from the trajectory cache, and determines the landing point based on the lateral constraints between the hitting point position and the candidate landing point position, as well as the height change trend of the trajectory points. If no landing point is detected within a preset number of frames, the landing point will be forcibly searched in the trajectory segment after the hitting point according to the opposite side constraint and the lowest point constraint, and the landing point will be searched again in the trajectory segment between the two hitting points when the next hitting point is detected. Based on the perspective transformation matrix, the pixel coordinates of the target ball, the pixel coordinates of the striking point, and the pixel coordinates of the landing point are converted into planar coordinates in the standard table tennis coordinate system. The three-dimensional trajectory of the target ball in the standard table coordinate system is reconstructed based on the planar coordinates, video frame rate, and gravitational acceleration. The striking party, landing area, and trajectory type are determined based on the position of the striking point and landing point, and the structured trajectory analysis results are output.

2. The method for analyzing the three-dimensional trajectory of a ping-pong ball according to claim 1, characterized in that, During the training phase, the multi-frame spatiotemporal perception network synchronously applies at least one of the following image enhancement methods: scaling, cropping, rotation, flipping, and color perturbation to consecutive frames. It sets visibility-weighted loss weights for randomly occluded samples, uses Gaussian soft labels as supervision signals for completely occluded samples, and incorporates velocity consistency loss and acceleration consistency loss to constrain the motion continuity of the predicted trajectory.

3. The method for analyzing the three-dimensional trajectory of a ping-pong ball according to claim 2, characterized in that, The encoder of the multi-frame spatiotemporal sensing network saves spatial and temporal features during downsampling, and the decoder splices the corresponding spatial and temporal features during upsampling. The temporal compression layer uses N×1×1 convolution to compress the multi-frame features into a single-channel two-dimensional heatmap.

4. The method for analyzing the three-dimensional trajectory of a ping-pong ball according to claim 1, characterized in that, The step of smoothing the trajectory points in the trajectory buffer includes: smoothing the trajectory points in the trajectory buffer using a Savitzky-Golay filter.

5. The method for analyzing the three-dimensional trajectory of a ping-pong ball according to claim 4, characterized in that, The steps of marking the trajectory point corresponding to the current frame as the hitting point include: The trajectory point corresponding to the current frame is used as a candidate hitting point. The position of the racket in the image is obtained, and the position of the candidate hitting point is corrected according to the position of the racket. For the first candidate hitting point detected at the beginning of the video segment, the first candidate hitting point is corrected based on the nearest racket center position, and the corrected candidate hitting point is marked as the hitting point.

6. The method for analyzing the three-dimensional trajectory of a ping-pong ball according to claim 1, characterized in that, The opposite constraint is that the landing point is located in the half-table area opposite to the hitting point; the height change trend of the trajectory point is determined by the position change of the trajectory point in the vertical direction in the image coordinate system, and the trajectory point in the trajectory descent phase is preferentially selected as the landing point.

7. The method for analyzing the three-dimensional trajectory of a ping-pong ball according to claim 6, characterized in that, The step of forcing the search for the landing point within the trajectory segment after the point of impact, according to the aforementioned contralateral constraints and lowest point constraints, includes: In the trajectory segment after the hitting point, select trajectory points that satisfy the opposite side constraint. According to the lowest point constraint, select the trajectory point with the smallest vertical position coordinate value in the image coordinate system from the trajectory points that satisfy the opposite side constraint as the landing point. The striking party, landing area, and trajectory type are determined based on the planar coordinates of the landing point and the striking point in the standard table coordinate system.

8. A three-dimensional trajectory analysis system for table tennis balls, characterized in that, include: The module is used to acquire consecutive image frames from the video stream and construct a sliding window containing multiple frames in chronological order. The processing module is used to input the sliding window into a multi-frame spatiotemporal perception network, perform two-dimensional spatial convolution on each image frame through the multi-frame spatiotemporal perception network, and then perform three-dimensional temporal convolution on the convolved multi-frame features. After upsampling and feature concatenation by the encoder-decoder, a single-channel two-dimensional heat map is output by the temporal compression layer, and the pixel coordinates of the target sphere in the frame to be analyzed are obtained by Softmax. The calculation module is used to obtain the corner points of the table in the image, sort the corner points in a preset order, and calculate the perspective transformation matrix based on the sorted corner points and the standard corner point coordinates of the standard table coordinate system. The marking module is used to write the pixel coordinates of the target ball in multiple consecutive frames into the trajectory cache, perform smoothing filtering on the trajectory points in the trajectory cache, calculate the change in motion direction based on the adjacent trajectory points after smoothing filtering, and mark the trajectory point corresponding to the current frame as the hitting point when the change in motion direction exceeds a preset threshold and the frame interval between the current frame and the previous hitting point is greater than a preset minimum interval. The search module is used to switch to the landing point search state after detecting the hitting point, filter the trajectory points located in the table area from the trajectory cache, and determine the landing point based on the opposite constraint between the hitting point position and the candidate landing point position and the height change trend of the trajectory point. If no landing point is detected within a preset number of frames, the landing point will be forcibly searched in the trajectory segment after the hitting point according to the opposite side constraint and the lowest point constraint, and the landing point will be searched again in the trajectory segment between the two hitting points when the next hitting point is detected. The conversion module is used to convert the pixel coordinates of the target ball, the pixel coordinates of the hitting point, and the pixel coordinates of the landing point into planar coordinates in the standard table tennis coordinate system according to the perspective transformation matrix. The output module is used to reconstruct the three-dimensional trajectory of the target ball in the standard table coordinate system based on the planar coordinates, video frame rate, and gravitational acceleration in the standard table coordinate system, and to determine the striking party, landing area, and trajectory type based on the position of the striking point and landing point, and output the structured trajectory analysis results.

9. An electronic device, characterized in that, The electronic device includes a processor coupled to a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the three-dimensional trajectory analysis method for a ping-pong ball as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which, when executed by a processor, implements the three-dimensional trajectory analysis method for a ping-pong ball as described in any one of claims 1 to 7.

Images