Badminton drop point prediction method and system based on error correction

Abstract

The invention provides a badminton drop point prediction method and system based on error correction, and the method comprises the steps: carrying out the spatial feature extraction of a standardized frame image sequence in a badminton movement process through a small target positioning and tracking network based on thermodynamic diagram contour detection, and obtaining an initial thermodynamic diagram of a badminton candidate region in each frame image; performing time sequence memory error correction on the initial thermodynamic diagram of each frame of image based on the initial thermodynamic diagrams of all previous historical frame images in the frame image sequence to obtain a corrected thermodynamic diagram of each frame of image; performing shuttlecock contour detection and central point extraction on the corrected thermodynamic diagram to obtain accurate pixel coordinates of the shuttlecock head in each frame of image; and constructing a flight path sequence based on all the accurate pixel coordinates, and further predicting to obtain the falling point position of the shuttlecock. On the basis of the scheme, high-precision positioning and tracking can be performed on a badminton small target moving at a high speed so as to accurately predict a badminton falling point.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a method and system for predicting the landing point of a badminton shuttlecock based on error correction. Background Technology

[0002] In the field of image processing and intelligent analysis related to badminton, badminton landing point prediction technology has become a research hotspot in the field of image processing technology due to its important application value in scenarios such as competition assistance, training analysis, and intelligent refereeing. The core of this technology relies on visual capture and trajectory analysis of the badminton flight process to achieve accurate prediction of the landing point. However, as a small target moving at high speed, the badminton shuttlecock is prone to problems such as motion blur and background interference in the image, which puts forward stringent requirements on the accuracy and real-time performance of image acquisition, target positioning and tracking, and trajectory prediction.

[0003] In existing technologies, mainstream badminton shuttlecock landing point prediction methods mostly extract shuttlecock position information using conventional visual detection algorithms, and then perform trajectory fitting and landing point estimation based on the extracted coordinates. However, these methods generally have significant technical shortcomings: on the one hand, conventional detection algorithms lack sufficient positioning accuracy for small targets like high-speed moving shuttlecocks, and are easily affected by factors such as court background and lighting changes, resulting in positioning deviations; on the other hand, they do not consider the temporal continuity of badminton movement, and cannot use historical movement information to correct positioning errors in the current frame, leading to jitter and distortion in the extracted coordinates. This results in low trajectory fitting accuracy and large deviations in landing point prediction results, making it difficult to meet the accuracy and stability requirements of landing point prediction in practical applications. Therefore, how to achieve high-precision positioning and tracking of small targets like high-speed moving shuttlecocks to accurately predict their landing points has become a challenge for the industry. Summary of the Invention

[0004] This application provides a badminton shuttlecock landing point prediction method and system based on error correction, which can achieve high-precision positioning and tracking of a small target shuttlecock moving at high speed to accurately predict the landing point of the shuttlecock.

[0005] In a first aspect, this application provides a badminton shuttlecock landing point prediction method based on error correction, comprising the following steps: Acquire multiple consecutive video images during badminton action, and preprocess all video images to obtain a standardized frame image sequence; Spatial features are extracted from the frame image sequence using a small target localization and tracking network based on heatmap contour detection to obtain an initial heatmap of the badminton shuttlecock candidate region in each frame image; For each frame in the frame image sequence, if the current frame image is not the first frame image, the initial heatmap of the current frame image is corrected for temporal memory error based on the initial heatmap of all historical frame images before the current frame to obtain the corrected heatmap of the current frame image. The modified heatmap is subjected to badminton shuttlecock contour detection and center point extraction to obtain the precise pixel coordinates of the shuttlecock head in the current frame image, and then the precise pixel coordinates of the shuttlecock head in multiple consecutive frames are obtained. The flight trajectory sequence of the badminton shuttlecock during its movement is constructed based on the precise pixel coordinates of multiple consecutive frames, thereby predicting the landing position of the shuttlecock.

[0006] In some embodiments, preprocessing all video images to obtain a standardized frame image sequence specifically includes: Spatial domain denoising is performed on all video images to obtain a smooth frame image sequence with imaging noise removed; The size of each smooth frame image in the smooth frame image sequence is normalized, and then the normalized images are arranged in the order of acquisition time to obtain a standardized frame image sequence.

[0007] In some embodiments, the initial heatmap of the badminton shuttlecock candidate region in each frame image is obtained by extracting spatial features from the frame image sequence using a small target localization and tracking network based on heatmap contour detection. Specifically, this includes: For each frame in the frame image sequence, the image is input into a small target localization and tracking network based on heatmap contour detection for multi-level downsampling convolution, and the spatial features of the badminton target are extracted layer by layer to obtain a multi-scale deep feature map. The deep feature map is upsampled at multiple levels, and then the features of the corresponding layers are connected by skip connections to obtain the recovered feature map. The initial heatmap of the badminton candidate region in the image is determined based on the recovered feature map, and then the initial heatmap of the badminton candidate region in each frame of the frame image sequence is obtained.

[0008] In some embodiments, the process of correcting the temporal memory error of the initial heatmap of the current frame image based on the initial heatmaps of all historical frame images preceding the current frame, to obtain the corrected heatmap of the current frame image, specifically includes: The initial heatmaps of all historical frames before the current frame are concatenated with the initial heatmap of the current frame in the channel dimension to obtain a multi-channel temporal feature map. The temporal feature map is input into the temporal memory error correction module of the small target localization and tracking network and temporal convolution is performed along the channel dimension to extract the motion correlation features between frames; The positioning deviation in the initial heatmap of the current frame is adaptively corrected using a neural network Kalman filter based on the motion correlation features to obtain the corrected heatmap of the current frame image.

[0009] In some embodiments, performing badminton shuttlecock contour detection and center point extraction on the modified heatmap to obtain the precise pixel coordinates of the shuttlecock head in the current frame image specifically includes: The modified heatmap is binarized and segmented to obtain a binarized heatmap of the badminton shuttlecock candidate region; Based on the binarized heatmap, connected component analysis is performed, and then the contour boundaries of all independent closed regions are extracted to obtain multiple candidate badminton shuttlecock contours. The actual target area for the badminton shuttlecock is determined based on the area of ​​the fitted rectangle of all candidate shuttlecock profiles. Calculate the geometric center coordinates of the target area of ​​the badminton shuttlecock to obtain the precise pixel coordinates of the shuttlecock head in the current frame image.

[0010] In some embodiments, constructing a sequence of the shuttlecock's flight trajectory during motion based on precise pixel coordinates across multiple consecutive frames, and then predicting the shuttlecock's landing point, specifically includes: Organize the precise pixel coordinates of multiple consecutive frames into a two-dimensional spatiotemporal coordinate sequence in chronological order; The motion state parameters of the spatiotemporal coordinate sequence are calculated to construct the original flight trajectory of the badminton shuttlecock. The original flight trajectory is smoothed and filtered, and then the trajectory end is extrapolated and extended based on the smoothed flight trajectory to obtain the predicted coordinate sequence of the badminton trajectory end. The landing point of the badminton shuttlecock is obtained by performing a spatial intersection operation between the predicted coordinate sequence and the pre-marked boundary line of the badminton court.

[0011] In some embodiments, the small target localization and tracking network based on heatmap contour detection is a deep learning model that employs an encoder-decoder architecture and combines spatial attention feature extraction and temporal memory error correction mechanisms.

[0012] Secondly, this application provides a badminton shuttlecock landing point prediction system based on error correction, used to execute a badminton shuttlecock landing point prediction method based on error correction, including: The acquisition module is used to acquire multiple consecutive video images during badminton, and preprocess all video images to obtain a standardized frame image sequence; The processing module is used to extract spatial features from the frame image sequence through a small target localization and tracking network based on heatmap contour detection, so as to obtain an initial heatmap of the badminton shuttlecock candidate region in each frame image; The processing module is further configured to, for each frame image in the frame image sequence, if the current frame image is not the first frame image, perform time-series memory error correction on the initial heatmap of the current frame image based on the initial heatmap of all historical frame images before the current frame, to obtain the corrected heatmap of the current frame image. The processing module is also used to perform badminton shuttlecock contour detection and center point extraction on the modified heatmap to obtain the precise pixel coordinates of the shuttlecock head in the current frame image, and then obtain the precise pixel coordinates of the shuttlecock head in multiple consecutive frames. The execution module is used to construct a sequence of the badminton shuttlecock's flight trajectory during motion based on the precise pixel coordinates of multiple consecutive frames, and then predict the landing position of the badminton shuttlecock.

[0013] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described method for predicting the landing point of a badminton shuttlecock based on error correction.

[0014] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for predicting the landing point of a badminton shuttlecock based on error correction.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: This application provides a badminton shuttlecock landing point prediction method and system based on error correction. The method involves acquiring multiple consecutive video images during the badminton shuttlecock's movement and preprocessing all video images to obtain a standardized frame image sequence. Spatial features are extracted from the frame image sequence using a small target localization and tracking network based on heatmap contour detection to obtain an initial heatmap of the shuttlecock candidate region in each frame. For each frame in the frame image sequence, if the current frame is not the first frame, the initial heatmap of the current frame is corrected for temporal memory error based on the initial heatmaps of all previous historical frames to obtain a corrected heatmap. The corrected heatmap is then used for shuttlecock contour detection and center point extraction to obtain the precise pixel coordinates of the shuttlecock's head in the current frame, and subsequently, the precise pixel coordinates of the shuttlecock's head in multiple consecutive frames. Based on the precise pixel coordinates of these multiple frames, a flight trajectory sequence of the shuttlecock during its movement is constructed, thereby predicting the landing point of the shuttlecock.

[0016] Therefore, this application constructs a sequence of badminton shuttlecock flight trajectories during motion based on precise pixel coordinates of multiple consecutive frames, thereby predicting the landing position of the shuttlecock. First, determining the initial heatmap yields a two-dimensional probability distribution map with the same spatial size as the input image, where each pixel value represents the probability that the location belongs to the shuttlecock target. The initial heatmap is determined by a small target localization and tracking network based on heatmap contour detection, which extracts spatial features from the standardized frame image sequence. Through multi-level downsampling convolution, upsampling, and skip connections, multi-scale contextual information is fused, transforming the shuttlecock target localization into a pixel-level probability prediction problem. The generated initial heatmap retains the spatial location and uncertainty information of the shuttlecock candidate region in the form of a probability distribution. This achieves accurate extraction of the spatial features of the small shuttlecock target, effectively enhancing the perception ability of the shuttlecock target under motion blur and background interference. It also provides a fusionable continuous probability expression basis for subsequent temporal memory error correction, ensuring reliable raw data support for subsequent error correction stages. This improves the accuracy of shuttlecock target localization from the source, laying a solid foundation of spatial features for subsequent precise pixel coordinate extraction, flight trajectory construction, and accurate landing point prediction. The foundation is established by determining the corrected heatmap, which then yields the single-channel probability map output by the temporal memory error correction module after fusing motion-related features. The corrected heatmap is determined based on the initial heatmap, fusing heatmaps from all historical frames preceding the current frame. This is achieved by stitching together heatmaps along the channel dimension to form a temporal feature map and extracting inter-frame motion-related features. An adaptive correction mechanism, similar to a Kalman filter in a neural network, compensates for the positioning deviation of the initial heatmap in the current frame, effectively eliminating positioning errors caused by motion blur and false detections of similar targets. Simultaneously, the temporal continuity of badminton motion compensates for spatial errors in a single frame image. The limitations of feature extraction allow the corrected heatmap to accurately represent the probability distribution of the actual position of the badminton shuttlecock, providing a highly reliable feature foundation for subsequent shuttlecock contour detection and precise pixel coordinate extraction of the shuttlecock head. This fundamentally improves the accuracy and stability of badminton shuttlecock target localization, and thus lays a crucial temporal correction feature foundation for the accurate construction of the flight trajectory and the accurate prediction of the landing point. This significantly improves the accuracy and robustness of the overall landing point prediction scheme. In summary, based on the above scheme, high-precision localization and tracking of small targets such as high-speed moving badminton shuttlecocks can be achieved to accurately predict the landing point of the badminton shuttlecock. Attached Figure Description

[0017] Figure 1 This is an exemplary flowchart of a badminton shuttlecock landing point prediction method based on error correction, according to some embodiments of this application; Figure 2 This is an exemplary flowchart illustrating the determination of an initial heat map according to some embodiments of this application; Figure 3This is a flowchart illustrating the operation of determining precise pixel coordinates according to some embodiments of this application; Figure 4 This is a schematic diagram of the structure of a badminton shuttlecock landing point prediction system based on error correction, as shown in some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computer device that implements a badminton shuttlecock landing point prediction method based on error correction, according to some embodiments of this application. Detailed Implementation

[0018] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] refer to Figure 1 The figure is an exemplary flowchart of a badminton shuttlecock landing point prediction method based on error correction, according to some embodiments of this application. The figure mainly includes the following steps: In step 101, multiple consecutive video images during the badminton game are acquired, and all video images are preprocessed to obtain a standardized frame image sequence.

[0020] In some embodiments, acquiring multiple consecutive frames of video images during badminton can be achieved as follows: a continuous video stream during badminton can be acquired in real time by an image acquisition device deployed above the badminton court, covering the entire sports field. Then, a video decoder based on an open-source computer vision library is used to analyze the continuous video stream frame by frame, decomposing the continuous video stream into single-frame static images. Each static image obtained is then arranged in the order of acquisition time to obtain multiple consecutive frames of video images. In order to adapt to the high-speed movement of badminton and ensure the real-time processing capability of the subsequent positioning network, the acquisition frame rate is set to no less than 25 frames per second, thereby accurately capturing the positional changes of the badminton in each frame during its rapid flight. The video images are time-series images that record the continuous movement of the badminton and retain the original spatial position information.

[0021] In some embodiments, preprocessing all video images to obtain a standardized sequence of frame images can be achieved using the following steps: Spatial domain denoising is performed on all video images to obtain a smooth frame image sequence with imaging noise removed; The size of each smooth frame image in the smooth frame image sequence is normalized, and then the normalized images are arranged in the order of acquisition time to obtain a standardized frame image sequence.

[0022] In specific implementation, spatial domain denoising processing is performed on all video images to obtain a smooth frame image sequence with imaging noise removed. This can be achieved in the following way: a Gaussian filtering algorithm based on spatial domain filtering can be applied to denoise all video images, suppressing Gaussian noise introduced by the imaging sensor in the images. All video images after removing imaging noise are then recombined in their original time sequence to form a smooth and clean smooth frame image sequence. Here, the smooth frame image sequence refers to a set of continuous frame images that effectively suppresses imaging sensor noise. It can significantly reduce Gaussian noise interference introduced during image acquisition while maintaining the edge contour and motion blur features of the badminton shuttlecock, providing cleaner image data for subsequent precise positioning. In specific implementation, the size normalization of each smooth frame image in the smooth frame image sequence, and then arranging the normalized images in the order of acquisition time to obtain a standardized frame image sequence, can be achieved in the following way: a bilinear interpolation algorithm can be used to normalize the size of each video image in the smooth frame image sequence. For example, the new pixel value is determined by calculating the weighted average of the four neighboring pixels of the target pixel in the original image, thereby scaling the original resolution to a standard size of 288 x 512 pixels. Then, the pixel values ​​of the scaled image are normalized by dividing by 255, so that the range of all pixel values ​​is adjusted to between zero and one, which is convenient for subsequent processing by the neural network model. All images after size normalization and pixel value normalization are obtained, and all images are arranged in the order of the original acquisition time to construct a standardized frame image sequence.

[0023] It should be noted that, in this application, the frame image sequence is a collection of images that record the continuous motion of the badminton shuttlecock and have a uniform pixel standard resolution; it can eliminate size differences caused by different acquisition devices and shooting distances through standardization processing, enabling the neural network to perform batch processing and feature extraction with a fixed input dimension, while retaining the relative position information and spatial distribution characteristics of the badminton shuttlecock through bilinear interpolation.

[0024] In step 102, the spatial features of the frame image sequence are extracted by a small target localization and tracking network based on heatmap contour detection to obtain the initial heatmap of the badminton candidate region in each frame image.

[0025] In some embodiments, reference Figure 2 The figure is an exemplary flowchart illustrating the determination of an initial heatmap according to some embodiments of this application. In this application, the initial heatmap of the badminton candidate region in each frame image is obtained by extracting spatial features from the frame image sequence using a small target localization and tracking network based on heatmap contour detection. This can be achieved through the following steps: In step 1021, for each frame image in the frame image sequence, the image is input into a small target localization and tracking network based on heatmap contour detection for multi-level downsampling convolution, and the spatial features of the badminton shuttlecock target are extracted layer by layer to obtain a multi-scale deep feature map; In step 1022, the deep feature map is upsampled at multiple levels, and then the features of the corresponding layers are connected by skip connections to obtain the recovered feature map; In step 1023, the initial heat map of the badminton candidate region in the image is determined based on the recovered feature map, thereby obtaining the initial heat map of the badminton candidate region in each frame of the frame image sequence.

[0026] It should be noted that the small target localization and tracking network based on heatmap contour detection in this application is a deep learning model that adopts an encoder-decoder architecture and combines spatial attention feature extraction and temporal memory error correction mechanisms. It includes: a U-shaped backbone network constructed using an encoder-decoder architecture, used for multi-level spatial feature extraction and resolution restoration of the input image; a compressed-excitation channel attention and residual feature extraction module deployed in each layer of the encoder and decoder, wherein the compressed-excitation channel attention and residual feature extraction module combines a compressed-excitation channel attention mechanism with a residual connection structure to enhance the foreground response to the small target badminton shuttlecock and suppress the loss of detailed features in the deep network; and a connection between the encoder and decoder. The network employs a skip connection structure to transfer fine-grained spatial information from the encoding layer to the decoding layer to aid feature reconstruction. A temporal memory error correction module, deployed after the spatial feature extraction module, corrects positioning errors based on historical context information by concatenating the current frame input with cached historical frame heatmaps along the channel dimension and performing temporal convolutional inference. A heatmap generation layer at the network output outputs a corrected heatmap representing the probability distribution of the badminton shuttlecock's position. This small target localization and tracking network can convert the input video frame image sequence into a badminton target heatmap sequence corrected by historical context information, serving as the basis for subsequent precise pixel coordinate extraction and landing point prediction.

[0027] In specific implementation, the image is input into a small target localization and tracking network based on heatmap contour detection, and multi-level downsampling convolution is performed to extract the spatial features of the badminton target layer by layer to obtain multi-scale deep feature maps. This can be achieved in the following way: First, the current frame image of standardized size can be passed as input data to the small target localization and tracking network based on heatmap contour detection. The first layer of the small target localization and tracking network encoder performs a convolution operation with a kernel size of 3×3 and a stride of 2 on the input image. While compressing the spatial resolution of the image, the edge and texture features of the badminton target are extracted to obtain the first-level feature map. Then, the first-level feature map is passed sequentially to the subsequent multi-level compression-excitation channel attention and residual feature extraction modules. The feature extraction module expands the receptive field of each module by repeatedly stacking basic convolutional units. At the same time, residual connections are used to add the input features of the module to the output features calculated by convolution element by element, which effectively suppresses the loss of detailed information of small targets layer by layer. After four layers of downsampling operations, a multi-scale deep feature map is obtained with the spatial resolution halved and the number of channels doubled layer by layer. The deep feature map refers to the multi-scale feature representation set with the spatial resolution decreasing layer by layer but the number of channels increasing layer by layer. It is the semantic basis for recovering the detailed information of the target during the upsampling stage of the decoder. It can fully capture the contextual semantic information of the badminton target through multi-level receptive field expansion, and at the same time, the residual connections are used to suppress the loss of detailed features of small targets layer by layer, providing rich deep semantic support for the subsequent accurate recovery of the badminton position.

[0028] In specific implementation, the deep feature map is upsampled at multiple levels, and then the recovered feature map is obtained by skip-connecting the features of the corresponding layers. This can be achieved in the following way: First, the lowest resolution feature map in the deep feature map is convolved by the deepest layer's compression-excitation channel attention and residual feature extraction module in the decoder to initially recover the spatial semantic information of the target. Then, bilinear interpolation is used to upsample the initially recovered feature map by 2 times, so that its spatial resolution is consistent with the feature map size of the corresponding layer in the encoder. Then, the upsampled result is spliced ​​and fused with the corresponding layer's fine-grained feature map saved by the encoder in the channel dimension through skip connections to achieve complementary enhancement of high-level semantic information and shallow edge information. Finally, the spliced ​​fused feature is input into the current layer's compression-excitation channel attention of the decoder. Further convolution calculations are performed with the residual feature extraction module. The compression-excitation channel attention mechanism in this module is used to automatically learn and assign higher weights to the feature channels related to the badminton target, thereby improving the perception ability of blurry small targets. Finally, the above process of upsampling, skip connection fusion and module calculation is repeated three times to gradually restore the spatial resolution to the same as the input image, resulting in the restored feature map. The restored feature map refers to the feature representation that is completely consistent with the spatial size of the input image, integrates multi-scale contextual information and is rich in badminton target position features. It can achieve complementary enhancement of semantic information and edge details through the skip connection fusion mechanism, and improve the perception ability of blurry small targets by using compression-excitation channel attention, thereby ensuring that the restored feature map is rich in accurate badminton target position information.

[0029] In specific implementation, the initial heatmap of the badminton candidate region in the image can be determined based on the recovered feature map in the following way: The recovered feature map can be input into the output layer of the small target localization and tracking network. The feature map is then dimensionality-reduced by a convolution operation with a kernel size of 1×1 and an output channel of 1, generating an original response map with the same spatial size as the input image. The Sigmoid activation function is then used to map the value of each pixel in the original response map to a probability interval between 0 and 1, so that each pixel value represents the confidence that the position belongs to the badminton target region, thereby forming the initial heatmap of the badminton candidate region in the image.

[0030] It should be noted that in this application, the initial heatmap refers to a two-dimensional probability distribution map with the same spatial size as the input image, where each pixel value represents the probability that the location belongs to a badminton target. It can be used as input data for the temporal memory error correction module, transforming the badminton target localization problem into a pixel-level probability prediction problem. By preserving the spatial uncertainty information of the target location in the form of probability distribution, it provides a fusionable continuous probability expression for subsequent error correction based on historical context information.

[0031] In step 103, for each frame image in the frame image sequence, if the current frame image is not the first frame image, the initial heat map of the current frame image is corrected for temporal memory error based on the initial heat map of all historical frame images before the current frame, so as to obtain the corrected heat map of the current frame image.

[0032] In some embodiments, the time-series memory error correction of the initial heatmap of the current frame image based on the initial heatmaps of all historical frame images preceding the current frame, to obtain the corrected heatmap of the current frame image, can be achieved through the following steps: The initial heatmaps of all historical frames before the current frame are concatenated with the initial heatmap of the current frame in the channel dimension to obtain a multi-channel temporal feature map. The temporal feature map is input into the temporal memory error correction module of the small target localization and tracking network and temporal convolution is performed along the channel dimension to extract the motion correlation features between frames; The positioning deviation in the initial heatmap of the current frame is adaptively corrected using a neural network Kalman filter based on the motion correlation features to obtain the corrected heatmap of the current frame image.

[0033] It should be noted that, in this application, the temporal memory error correction module refers to a neural network component deployed after the spatial feature extraction module in the small target localization and tracking network, used to fuse historical frame localization information to optimize the output of the current frame. This module mainly consists of two parts: a circular buffer queue and a multi-layer temporal convolutional layer. The circular buffer queue is responsible for storing and dynamically updating the initial heatmap generated by the previous historical frames, while the multi-layer temporal convolutional layer performs convolution operations on the stitched temporal feature map along the channel dimension to extract inter-frame motion correlation features. By introducing a neural network Kalman filter-style adaptive correction mechanism, it can effectively compensate for the localization deviation of the current frame based on the continuous motion trend when the badminton shuttlecock experiences motion blur or is interfered with by similar targets, thereby significantly improving the accuracy of small target localization and the stability of tracking.

[0034] In specific implementation, the initial heatmaps of all historical frames preceding the current frame are concatenated with the initial heatmap of the current frame in the channel dimension to obtain a multi-channel temporal feature map. This can be achieved as follows: First, a circular buffer queue of a preset length can be constructed, for example, a preset length of seven frames. The queue is maintained using a first-in-first-out update mechanism to continuously store the initial heatmaps of the historical frames preceding the current frame and the corrected heatmaps output after processing by the temporal memory error correction module. Whenever a new frame is processed and its corresponding corrected heatmap is obtained, the new corrected heatmap is stored as the initial heatmap at the end of the queue, while the earliest initial heatmap at the head of the queue is removed to ensure that the buffer always retains the most relevant frame to the current time point. A continuous initial heatmap is generated, and a circular cache queue storing the initial heatmaps of consecutive frames is used as an input component of the temporal memory error correction module. Then, the initial heatmap of the current frame image is aligned and stitched with all the latest historical initial heatmaps stored in the circular cache queue in the channel dimension to form a multi-channel temporal feature map. The temporal feature map refers to the multi-channel data structure formed by stitching the initial heatmap of the current frame with the historical heatmaps of multiple frames in the channel dimension. This temporal feature map integrates the spatial positioning information of the current frame and the historical positioning information of the previous seven frames in the channel dimension, so that each pixel position contains the continuous response change along the time axis, providing a complete data foundation for subsequent analysis of the badminton's motion pattern in a continuous time series.

[0035] In specific implementation, the temporal feature map is input into the temporal memory error correction module of the small target localization and tracking network and temporal convolution is performed along the channel dimension. The extraction of motion correlation features between frames can be achieved in the following way: the temporal feature map can be input into the temporal memory error correction module of the small target localization and tracking network, and temporal convolution operation with a kernel size of 3x3 can be applied sequentially along the channel dimension. The correlation features of the badminton target between each frame are extracted pixel by pixel by calculating the sliding of the convolution kernel in the channel direction. The motion correlation features are features that characterize the displacement trend and velocity change of the badminton in consecutive frames.

[0036] In specific implementation, the positioning deviation in the initial heatmap of the current frame is adaptively corrected by a neural network Kalman filter through the motion association features to obtain the corrected heatmap of the current frame image. This can be achieved in the following way: the temporal memory error correction module performs a neural network Kalman filter adaptive correction on the potential deviation in the initial heatmap of the current frame based on the positioning information of the target in the initial heatmap of the historical frame in the motion association features. For example, when the badminton shuttlecock has a weak response in the current frame due to motion blur, the temporal memory error correction module can enhance the response value of the corresponding position in the current frame based on the continuous motion trend of the previous frame. Finally, the corrected heatmap of the current frame image is output through the output layer.

[0037] It should be noted that in this application, the corrected heatmap refers to the single-channel probability map output by the temporal memory error correction module after fusing motion-related features. It can eliminate motion blur interference and false detection of similar targets in the initial positioning of the current frame by making full use of historical motion information.

[0038] In step 104, the modified heatmap is subjected to badminton shuttlecock contour detection and center point extraction to obtain the precise pixel coordinates of the shuttlecock head in the current frame image, and then the precise pixel coordinates of the shuttlecock head in multiple consecutive frames are obtained.

[0039] In some embodiments, reference Figure 3 The figure is a flowchart illustrating the operation of determining precise pixel coordinates according to some embodiments of this application. In this application, the precise pixel coordinates of the shuttlecock head in the current frame image can be obtained by performing badminton shuttlecock contour detection and center point extraction on the modified heatmap using the following steps: The modified heatmap is binarized and segmented to obtain a binarized heatmap of the badminton shuttlecock candidate region; Based on the binarized heatmap, connected component analysis is performed, and then the contour boundaries of all independent closed regions are extracted to obtain multiple candidate badminton shuttlecock contours. The actual target area for the badminton shuttlecock is determined based on the area of ​​the fitted rectangle of all candidate shuttlecock profiles. Calculate the geometric center coordinates of the target area of ​​the badminton shuttlecock to obtain the precise pixel coordinates of the shuttlecock head in the current frame image.

[0040] In specific implementation, the modified heatmap is binarized and segmented to obtain a binarized heatmap of the badminton candidate region. This can be achieved in the following way: First, the probability value of each pixel in the modified heatmap is traversed, and the probability value of each pixel is compared with a preset confidence threshold. This confidence threshold can be adaptively determined using the Otsu method or fixed at 0.6 based on experience. It is used to distinguish the badminton target region from the background noise region. For pixels with a probability value greater than or equal to the confidence threshold, their pixel value is set to 255. For pixels with a probability value less than the confidence threshold, their pixel value is set to 255. The pixels are set to 0, thus transforming the original continuous probability distribution heatmap into a black-and-white binary image. Then, the isolated noise pixels in the binary image are removed by the opening operation in morphological operations, and the small holes in the target area are closed to obtain a clean binary heatmap as the binary heatmap of the badminton shuttlecock candidate region. The binary heatmap is a black-and-white binary image transformed from the continuous probability distribution in the modified heatmap, which can be used to filter out background noise response and highlight the badminton shuttlecock candidate region, providing clean input data for subsequent contour detection.

[0041] In specific implementation, the connection component analysis based on the binarized heatmap, and the extraction of the contour boundaries of all independent closed regions to obtain multiple candidate badminton shuttlecock contours, can be achieved in the following way: First, the binarized heatmap can be labeled with connected components using the eight-neighborhood connectivity criterion, and a unique label value can be assigned to each labeled connected region using a two-scan algorithm or a seed filling algorithm, transforming pixel-level data into region-level data; then, for each labeled connected region, a boundary tracking algorithm is used to extract its contour point set, which is composed of all boundary pixels within the region arranged in a counterclockwise or clockwise order, forming a closed curve description; and a contour simplification algorithm is used to remove redundant collinear points in the contour, retaining only key vertices to reduce subsequent computational complexity while preserving the geometric features of the contour; finally, all extracted contour point sets are organized in a list to obtain multiple candidate badminton shuttlecock contours, each contour corresponding to a potential target region; wherein, the candidate badminton shuttlecock contour refers to the boundary of an independent closed region that may contain a badminton shuttlecock target, which can be used to exhaustively enumerate all possible badminton shuttlecock target positions in the image, ensuring that no real target is missed.

[0042] In practical implementation, determining the actual target area of ​​the badminton shuttlecock based on the fitted rectangle area of ​​all candidate shuttlecock contours can be achieved as follows: First, traverse each candidate shuttlecock contour, calculate the number of pixels within the area enclosed by the contour to obtain the actual area of ​​the contour, then apply the minimum bounding rectangle algorithm to the contour. Calculate the minimum area rectangle that can contain all points of the contour using the rotating caliper method. This rectangle can be rotated in any direction to most closely fit the contour shape. The width and height of the rectangle reflect the length of the major and minor axes of the contour, respectively. Calculate the geometric area of ​​the minimum bounding rectangle, which is the product of the width and height, as the fitted rectangle area of ​​the contour. The actual area and fitted rectangle area of ​​each candidate badminton shuttlecock contour are obtained. Then, candidate contours that conform to the true size characteristics of a badminton shuttlecock are selected from all candidate badminton shuttlecock contours by using a preset size threshold range (e.g., the minimum actual area is greater than ten pixels and less than five hundred pixels), since the badminton shuttlecock usually appears as the foreground area with the most significant area in the heat map. Then, the contour with the largest fitted rectangle area is selected from all the selected candidate contours as the real badminton shuttlecock target area. The badminton shuttlecock target area refers to the image area that reflects the location of the unique real target of the badminton shuttlecock, which can eliminate the interference of similar objects and accurately locate the actual location of the badminton shuttlecock.

[0043] It should be noted that, in this application, the precise pixel coordinates refer to the pixel-level position of the geometric center of the image region where the unique real target of the badminton shuttlecock is located in the current frame image. This position can be used as the basic data for constructing the flight trajectory sequence and predicting the landing position. Specifically, the precise pixel coordinates of the shuttlecock head in the current frame image can be obtained by calculating the geometric center coordinates of the target region of the badminton shuttlecock in the following way: obtain the coordinates of the four vertices of the smallest bounding rectangle of the target region of the badminton shuttlecock, and obtain the center point coordinates of the target region of the badminton shuttlecock by averaging the coordinates of the diagonal vertices or by directly obtaining the center point coordinates of the rectangle using the formula for the center point of the rectangle. Then, map the center point coordinates back to the original image resolution, and eliminate the coordinate offset caused by the image scaling preprocessing by proportional conversion to obtain the pixel coordinates that are consistent with the spatial position of the original video image as the precise pixel coordinates of the shuttlecock head in the current frame image.

[0044] In step 105, a sequence of the badminton shuttlecock's flight trajectory during motion is constructed based on the precise pixel coordinates of multiple consecutive frames, thereby predicting the landing position of the shuttlecock.

[0045] In some embodiments, constructing a sequence of the shuttlecock's flight trajectory during motion based on precise pixel coordinates across multiple consecutive frames, and then predicting the shuttlecock's landing position, can be achieved through the following steps: Organize the precise pixel coordinates of multiple consecutive frames into a two-dimensional spatiotemporal coordinate sequence in chronological order; The motion state parameters of the spatiotemporal coordinate sequence are calculated to construct the original flight trajectory of the badminton shuttlecock. The original flight trajectory is smoothed and filtered, and then the trajectory end is extrapolated and extended based on the smoothed flight trajectory to obtain the predicted coordinate sequence of the badminton trajectory end. The landing point of the badminton shuttlecock is obtained by performing a spatial intersection operation between the predicted coordinate sequence and the pre-marked boundary line of the badminton court.

[0046] In specific implementation, organizing the precise pixel coordinates of multiple consecutive frames into a two-dimensional spatiotemporal coordinate sequence in chronological order can be achieved in the following way: the precise pixel coordinates of multiple consecutive frames can be arranged sequentially in chronological order through a trajectory construction algorithm to form a spatiotemporal coordinate sequence corresponding to the two-dimensional spatial coordinates and the time dimension; wherein, the spatiotemporal coordinate sequence refers to a structured data set that completely records the planar position information of the badminton shuttlecock at every moment during its flight, which can be used to completely preserve the position evolution law during the badminton shuttlecock's movement and provide basic data for subsequent motion state calculation.

[0047] In specific implementation, the motion state parameters of the spatiotemporal coordinate sequence are calculated to construct the original flight trajectory of the badminton motion process. This can be achieved in the following way: the motion state parameters of the spatiotemporal coordinate sequence can be calculated through a trajectory observation model. For example, a multi-feature fusion trajectory observation model can be used to calculate the velocity vector, acceleration vector, and motion direction angle at each moment. Based on the calculated motion state parameters, the flight trajectory curve of the badminton continuous motion process is constructed as the original flight trajectory. The original flight trajectory refers to a parameterized curve that describes the position evolution of the badminton in the court plane with time as the independent variable. It can be used to truly reflect the kinematic characteristics of badminton flight and provide input basis for subsequent smoothing processing.

[0048] In specific implementation, the original flight trajectory is smoothed and filtered, and then the trajectory end is extrapolated and extended based on the smoothed flight trajectory to obtain the predicted coordinate sequence of the badminton trajectory end. This can be achieved in the following way: the original flight trajectory can be processed by Savitsky-Gory smoothing filtering to filter out high-frequency noise components introduced by factors such as target detection jitter and coordinate quantization errors, while retaining the main motion trend of the badminton flight, resulting in a smoothed flight trajectory curve. Then, a time-series prediction model based on radial basis function interpolation is used to extrapolate and extend the trajectory end of the flight trajectory curve. For example, using historical trajectory points as interpolation nodes, a radial basis function neural network or radial basis interpolation function is constructed to predict the coordinates of multiple consecutive future frames, and the predicted coordinate sequence of the badminton trajectory end is calculated. The predicted coordinate sequence is a set of coordinates representing the possible positions of the badminton at consecutive future moments, describing the possible positions of the badminton at subsequent moments, and can be used to provide spatial basis for landing point determination and improve the continuity and reliability of prediction results.

[0049] In specific implementation, the predicted coordinate sequence is intersected with the pre-calibrated badminton court boundary line to obtain the landing position of the badminton shuttlecock. This can be achieved in the following way: First, the geometric model of the badminton court plane and boundary line can be obtained in advance through spatial calibration. This geometric model is represented by the equation of a straight line or a set of line segments in the court plane coordinate system, accurately defining the boundary between the internal and external areas of the court. Then, through spatial intersection operation, the predicted coordinate sequence at the end of the trajectory is used to determine the point-by-point spatial relationship with the geometric model of the badminton court plane and boundary line. The position attribute of each predicted coordinate relative to the boundary line is calculated, i.e., inside or outside the boundary, and the first coordinate point in the predicted coordinate sequence that comes into contact with the court plane is determined. Finally, the first coordinate point that comes into contact with the court plane is taken as the final predicted landing position of the badminton shuttlecock, and the landing coordinates and the position attribute of the landing point are output.

[0050] Furthermore, in another aspect of this application, in some embodiments, this application provides a badminton shuttlecock landing point prediction system based on error correction, referring to... Figure 4 The figure is a schematic diagram of the structure of a badminton shuttlecock landing point prediction system based on error correction according to some embodiments of this application. The badminton shuttlecock landing point prediction system based on error correction includes: an acquisition module 201, a processing module 202, and an execution module 203, which are described below: The acquisition module 201 in this application is mainly used to acquire multiple consecutive video images during badminton, and to preprocess all video images to obtain a standardized frame image sequence. Processing module 202, in this application, is used to extract spatial features from the frame image sequence through a small target localization and tracking network based on heatmap contour detection, so as to obtain an initial heatmap of the badminton candidate region in each frame image; It should be noted that the processing module 202 is also used for each frame image in the frame image sequence, if the current frame image is not the first frame image, to perform time-series memory error correction on the initial heat map of the current frame image based on the initial heat map of all historical frame images before the current frame, so as to obtain the corrected heat map of the current frame image. Additionally, it should be noted that the processing module 202 is also used to perform badminton shuttlecock contour detection and center point extraction on the modified heatmap to obtain the precise pixel coordinates of the shuttlecock head in the current frame image, and then obtain the precise pixel coordinates of the shuttlecock head in multiple consecutive frames. The execution module 203 in this application is mainly used to construct a sequence of the flight trajectory of the badminton shuttlecock during the motion based on the precise pixel coordinates of multiple consecutive frames, and then predict the landing position of the badminton shuttlecock.

[0051] The foregoing has detailed examples of the badminton shuttlecock landing point prediction method and system based on error correction provided in the embodiments of this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specified application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specified application, but such implementation should not be considered beyond the scope of this application.

[0052] In some embodiments, this application also provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described error-corrected badminton landing point prediction method.

[0053] In some embodiments, reference Figure 5 The dashed lines in the figure indicate that the unit or module is optional. This figure is a schematic diagram of the structure of a computer device implementing a badminton shuttlecock landing point prediction method based on error correction, according to an embodiment of this application. The badminton shuttlecock landing point prediction method based on error correction described in the above embodiments can be... Figure 5 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a memory 302 and at least one communication unit 305. The computer device may be a terminal device, a server or a chip.

[0054] Processor 301 can be a general-purpose processor or a special-purpose processor. For example, processor 301 can be a central processing unit (CPU), which can be used to control computer devices, execute software programs, and process data from software programs. The computer device may also include a communication unit 305 for inputting (receiving) and outputting (transmitting) signals.

[0055] For example, the computer device may be a chip, and the communication unit 305 may be the input and / or output circuit of the chip, or the communication unit 305 may be the communication interface of the chip, which may be a component of a terminal device, network device or other device.

[0056] For example, the computer device may be a terminal device or a server, and the communication unit 305 may be a transceiver of the terminal device or the server, or the communication unit 305 may be a transceiver circuit of the terminal device or the server.

[0057] The computer device may include one or more memories 302 storing a program 304. The program 304 can be executed by a processor 301 to generate instructions 303, causing the processor 301 to execute the method described in the above method embodiments according to the instructions 303. Optionally, the memory 302 may also store data (such as a target audit model). Optionally, the processor 301 may also read data stored in the memory 302, which may be stored at the same storage address as the program 304, or it may be stored at a different storage address than the program 304.

[0058] The processor 301 and memory 302 can be configured separately or integrated together, for example, integrated on the system on chip (SOC) of the terminal device.

[0059] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 301. The processor 301 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0060] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] For example, in some embodiments, this application also provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the above-described error-corrected badminton shuttlecock landing point prediction method.

[0062] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0063] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A badminton shuttlecock landing point prediction method based on error correction, characterized in that, Includes the following steps: Acquire multiple consecutive video images during badminton action, and preprocess all video images to obtain a standardized frame image sequence; Spatial features are extracted from the frame image sequence using a small target localization and tracking network based on heatmap contour detection to obtain an initial heatmap of the badminton shuttlecock candidate region in each frame image; For each frame in the frame image sequence, if the current frame image is not the first frame image, the initial heatmap of the current frame image is corrected for temporal memory error based on the initial heatmap of all historical frame images before the current frame to obtain the corrected heatmap of the current frame image. The modified heatmap is subjected to badminton shuttlecock contour detection and center point extraction to obtain the precise pixel coordinates of the shuttlecock head in the current frame image, and then the precise pixel coordinates of the shuttlecock head in multiple consecutive frames are obtained. The flight trajectory sequence of the badminton shuttlecock during its movement is constructed based on the precise pixel coordinates of multiple consecutive frames, thereby predicting the landing position of the shuttlecock.

2. The method as described in claim 1, characterized in that, Preprocessing all video images to obtain a standardized sequence of frame images specifically includes: Spatial domain denoising is performed on all video images to obtain a smooth frame image sequence with imaging noise removed; The size of each smooth frame image in the smooth frame image sequence is normalized, and then the normalized images are arranged in the order of acquisition time to obtain a standardized frame image sequence.

3. The method as described in claim 1, characterized in that, Spatial feature extraction is performed on the frame image sequence using a small target localization and tracking network based on heatmap contour detection to obtain the initial heatmap of the badminton shuttlecock candidate region in each frame image. Specifically, this includes: For each frame in the frame image sequence, the image is input into a small target localization and tracking network based on heatmap contour detection for multi-level downsampling convolution, and the spatial features of the badminton target are extracted layer by layer to obtain a multi-scale deep feature map. The deep feature map is upsampled at multiple levels, and then the features of the corresponding layers are connected by skip connections to obtain the recovered feature map. The initial heatmap of the badminton candidate region in the image is determined based on the recovered feature map, and then the initial heatmap of the badminton candidate region in each frame of the frame image sequence is obtained.

4. The method as described in claim 1, characterized in that, Based on the initial heatmaps of all historical frames preceding the current frame, the initial heatmap of the current frame image is corrected for temporal memory errors to obtain the corrected heatmap of the current frame image. Specifically, this includes: The initial heatmaps of all historical frames before the current frame are concatenated with the initial heatmap of the current frame in the channel dimension to obtain a multi-channel temporal feature map. The temporal feature map is input into the temporal memory error correction module of the small target localization and tracking network and temporal convolution is performed along the channel dimension to extract the motion correlation features between frames; The positioning deviation in the initial heatmap of the current frame is adaptively corrected using a neural network Kalman filter based on the motion correlation features to obtain the corrected heatmap of the current frame image.

5. The method as described in claim 1, characterized in that, Performing badminton shuttlecock contour detection and center point extraction on the modified heatmap to obtain the precise pixel coordinates of the shuttlecock head in the current frame image specifically includes: The modified heatmap is binarized and segmented to obtain a binarized heatmap of the badminton shuttlecock candidate region; Based on the binarized heatmap, connected component analysis is performed, and then the contour boundaries of all independent closed regions are extracted to obtain multiple candidate badminton shuttlecock contours. The actual target area for the badminton shuttlecock is determined based on the area of ​​the fitted rectangle of all candidate shuttlecock profiles. Calculate the geometric center coordinates of the target area of ​​the badminton shuttlecock to obtain the precise pixel coordinates of the shuttlecock head in the current frame image.

6. The method as described in claim 1, characterized in that, Based on the precise pixel coordinates of multiple consecutive frames, a sequence of the badminton shuttlecock's flight trajectory during motion is constructed, and the landing position of the shuttlecock is predicted. Specifically, this includes: Organize the precise pixel coordinates of multiple consecutive frames into a two-dimensional spatiotemporal coordinate sequence in chronological order; The motion state parameters of the spatiotemporal coordinate sequence are calculated to construct the original flight trajectory of the badminton shuttlecock. The original flight trajectory is smoothed and filtered, and then the trajectory end is extrapolated and extended based on the smoothed flight trajectory to obtain the predicted coordinate sequence of the badminton trajectory end. The landing point of the badminton shuttlecock is obtained by performing a spatial intersection operation between the predicted coordinate sequence and the pre-marked boundary line of the badminton court.

7. The method as described in claim 1, characterized in that, The small target localization and tracking network based on heatmap contour detection is a deep learning model that adopts an encoder-decoder architecture and combines spatial attention feature extraction and temporal memory error correction mechanism.

8. A badminton shuttlecock landing point prediction system based on error correction, used to execute the badminton shuttlecock landing point prediction method based on error correction as described in any one of claims 1 to 7, characterized in that, include: The acquisition module is used to acquire multiple consecutive video images during badminton, and preprocess all video images to obtain a standardized frame image sequence; The processing module is used to extract spatial features from the frame image sequence through a small target localization and tracking network based on heatmap contour detection, so as to obtain an initial heatmap of the badminton shuttlecock candidate region in each frame image; The processing module is further configured to, for each frame image in the frame image sequence, if the current frame image is not the first frame image, perform time-series memory error correction on the initial heatmap of the current frame image based on the initial heatmap of all historical frame images before the current frame, to obtain the corrected heatmap of the current frame image. The processing module is also used to perform badminton shuttlecock contour detection and center point extraction on the modified heatmap to obtain the precise pixel coordinates of the shuttlecock head in the current frame image, and then obtain the precise pixel coordinates of the shuttlecock head in multiple consecutive frames. The execution module is used to construct a sequence of the badminton shuttlecock's flight trajectory during motion based on the precise pixel coordinates of multiple consecutive frames, and then predict the landing position of the badminton shuttlecock.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to retrieve the code and execute the badminton shuttlecock landing point prediction method based on error correction as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the badminton shuttlecock landing point prediction method based on error correction as described in any one of claims 1 to 7.

Images