Multi-target trajectory tracking method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a multi-target trajectory tracking method and device, computer equipment and a storage medium, and the method comprises the steps: extracting a central point of each target object from a high-definition video, mapping each pixel point of each initial video frame to a preset color category, and obtaining a target trajectory; and obtaining an intermediate video frame corresponding to each initial video frame, detecting a bounding box of the target object, generating a single-channel feature map of each target object, determining position information of each target object, and updating track coordinates of each target object based on the position information of each target object. The method has the beneficial effects that the bounding box of the target object can be detected in real time, the single-channel feature map is generated, the target features are effectively extracted, the tracking precision is remarkably improved, and meanwhile, the calculation burden is reduced, so that real-time processing is realized, and the competition requirement in a dynamic environment is met.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a multi-target trajectory tracking method, apparatus, computer device, and storage medium. Background Technology

[0002] With the continuous advancement of intelligent recognition and tracking technologies, multi-target trajectory prediction is increasingly widely used in various sports events. Especially in high-paced ball games, accurately predicting the trajectories of multiple targets is of paramount importance for referee decisions, event analysis, and training improvement. However, current tracking algorithms still face numerous challenges when applied to fast-moving targets (such as billiard balls), leading to frequent target loss.

[0003] Existing tracking algorithms largely rely on highly complex calculations and abundant data support. Many algorithms are slow and consume high computational resources, failing to meet the dynamic demands of competition venues in real time. When faced with rapidly moving targets, traditional region-based tracking methods are prone to losing targets due to brief occlusion or rapid movement, necessitating a multi-target trajectory tracking method. Summary of the Invention

[0004] Therefore, it is necessary to propose a multi-target trajectory tracking method, device, computer equipment, and storage medium to address the existing multi-target trajectory tracking problem.

[0005] A multi-target trajectory tracking method, the method comprising:

[0006] Acquire high-definition videos of multiple target objects, and locate the target in each initial video frame of the high-definition video to obtain the center point of each target object;

[0007] For each initial video frame, a nearest neighbor search is performed, and each pixel of each initial video frame is mapped to a preset color category to obtain the intermediate video frames corresponding to each initial video frame.

[0008] Based on the center point of the target object, detect the bounding box of the target object in each of the intermediate video frames;

[0009] A single-channel feature map of each target object is generated based on the bounding box of each target object;

[0010] The position information of each target object in each intermediate video frame is determined based on each of the single-channel feature maps;

[0011] The orbital coordinates of each target object are updated based on the position information of each target object.

[0012] Furthermore, after the step of updating the orbital coordinates of each target object based on the position information of each target object, the method further includes:

[0013] The target position information of each target object in the last frame of the high-definition video is obtained, and the target motion speed of the target object is calculated according to the preset Kalman filter.

[0014] Based on the target position information and target movement speed of each target object, predict the prediction area where each target object will appear in the next frame image;

[0015] Within the prediction region, normalized cross-correlation and feature map similarity are used to calculate the predicted location information of each target object.

[0016] The orbital coordinates of each target object are updated based on the predicted position information of each target object.

[0017] Furthermore, before the step of updating the orbital coordinates of each of the target objects based on the predicted position information of each of the target objects, the method further includes:

[0018] Obtain the similarity value calculated using normalized cross-correlation and feature map similarity within the predicted region;

[0019] Determine whether the similarity value is lower than a preset similarity threshold;

[0020] If the similarity is below the threshold, the target position information of each target object in the last initial video frame is completed by interpolating the trajectories of adjacent frames.

[0021] Further, the step of predicting the prediction region where each of the target objects will appear in the next frame image based on the target position information and the target motion speed of each target object includes:

[0022] Assuming the target's position is (x, y) and its velocity is (v... x ,v y );

[0023] The time interval for obtaining the next frame is Δt, and the preset radius is used by the formula x′=x+v. x Δt, y′=y+v y Δt is used to calculate the predicted location information;

[0024] Based on the predicted location information (x′, y′) and the preset radius, according to the formula M={(i,j)∣(ix′)2+(jy′)2≤r 2} Calculate the prediction region; where M is the prediction region, (i,j) represents a point in the prediction region, and r is a preset radius.

[0025] Furthermore, the step of acquiring high-definition video of multiple target objects and performing target localization on each initial video frame in the high-definition video to obtain the center point of each target object includes:

[0026] The background region of each initial video frame is segmented and binarized using a preset mean shift algorithm to obtain a binarized video frame corresponding to each initial video frame.

[0027] The center point of the target object in the binarized video frame is located using a preset target object localization method.

[0028] Furthermore, before the step of performing nearest neighbor search on each initial video frame and mapping each pixel of each initial video frame to a preset color category to obtain the intermediate video frames corresponding to each initial video frame, the method further includes:

[0029] Obtain a template image and perform KMeans segmentation on the template image to obtain the segmented color regions;

[0030] The color regions are divided to obtain the color category corresponding to each target object.

[0031] Furthermore, before the step of generating a single-channel feature map of each target object based on the bounding box of each target object, the method further includes:

[0032] The bounding box of each target object is input into a pre-trained CNN classifier to obtain the probability distribution of each target object in each intermediate video frame.

[0033] Based on the probability distribution of each target object in each intermediate video frame, the weighted probability of each target object and its corresponding bounding box is obtained by accumulating the probability according to the time decay weight.

[0034] Obtain the trajectory movement direction of each target object, and determine whether the target object with the highest weighted probability corresponding to the bounding box conflicts with the trajectory movement direction of that target object;

[0035] If the target object's trajectory direction conflicts with the target object's trajectory direction, the target object corresponding to the bounding box will be re-identified.

[0036] A multi-target trajectory tracking device, the device comprising:

[0037] The acquisition module is used to acquire high-definition videos of multiple target objects and perform target localization on each initial video frame in the high-definition video to obtain the center point of each target object.

[0038] The search module is used to perform nearest neighbor search on each of the initial video frames, map each pixel of each initial video frame to a preset color category, and obtain the intermediate video frames corresponding to each initial video frame.

[0039] The detection module is used to detect the bounding box of the target object in each intermediate video frame based on the center point of the target object;

[0040] A generation module is used to generate a single-channel feature map of each target object based on the bounding box of each target object;

[0041] The determination module is used to determine the position information of each target object in each intermediate video frame based on each of the single-channel feature maps;

[0042] An update module is used to update the orbital coordinates of each target object based on the position information of each target object. A computer device includes a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the following steps:

[0043] Acquire high-definition videos of multiple target objects, and locate the target in each initial video frame of the high-definition video to obtain the center point of each target object;

[0044] For each initial video frame, a nearest neighbor search is performed, and each pixel of each initial video frame is mapped to a preset color category to obtain the intermediate video frames corresponding to each initial video frame.

[0045] Based on the center point of the target object, detect the bounding box of the target object in each of the intermediate video frames;

[0046] A single-channel feature map of each target object is generated based on the bounding box of each target object;

[0047] The position information of each target object in each intermediate video frame is determined based on each of the single-channel feature maps;

[0048] The orbital coordinates of each target object are updated based on the position information of each target object. A computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0049] Acquire high-definition videos of multiple target objects, and locate the target in each initial video frame of the high-definition video to obtain the center point of each target object;

[0050] For each initial video frame, a nearest neighbor search is performed, and each pixel of each initial video frame is mapped to a preset color category to obtain the intermediate video frames corresponding to each initial video frame.

[0051] Based on the center point of the target object, detect the bounding box of the target object in each of the intermediate video frames;

[0052] A single-channel feature map of each target object is generated based on the bounding box of each target object;

[0053] The position information of each target object in each intermediate video frame is determined based on each of the single-channel feature maps;

[0054] The orbital coordinates of each target object are updated based on the position information of each target object.

[0055] The beneficial effects of this invention are as follows: By acquiring high-definition video of multiple target objects, effective target localization is performed on each frame, and the center point of the target object is quickly determined. This reduces the risk of target loss in the event of brief occlusion or instantaneous movement. Furthermore, based on a simple and fast nearest neighbor search, each pixel is mapped to a preset color category, making the processing of intermediate video frames more efficient. This enables real-time detection of the bounding box of the target object and generation of a single-channel feature map, effectively extracting target features, significantly improving tracking accuracy, and reducing computational burden. This achieves real-time processing and meets the needs of competitions in dynamic environments. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] in:

[0058] Figure 1 This is an application environment diagram of a multi-target trajectory tracking method in one embodiment;

[0059] Figure 2 This is a flowchart of a multi-target trajectory tracking method in one embodiment;

[0060] Figure 3 This is a structural block diagram of a multi-target trajectory tracking device in one embodiment;

[0061] Figure 4 This is a structural block diagram of a computer device in one embodiment. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Figure 1 This is a diagram illustrating a multi-target trajectory tracking application environment in one embodiment. (Refer to...) Figure 1 This multi-target trajectory tracking method is applied to a multi-target trajectory tracking system. The system includes a terminal 110 and a server 120. The terminal 110 and server 120 are connected via a network. The terminal 110 can be a desktop terminal or a mobile terminal; a mobile terminal can be at least one of a mobile phone, tablet, or laptop. The server 120 can be a standalone server or a server cluster consisting of multiple servers. The terminal 110 is used to acquire high-definition video, and the server 120 is used to analyze the trajectory coordinates of the target objects.

[0064] like Figure 2 As shown, in one embodiment, a multi-target trajectory tracking method is provided. This method can be applied to both terminals and servers; this embodiment uses server application as an example. The multi-target trajectory tracking method specifically includes the following steps:

[0065] S1: Acquire high-definition video of multiple target objects, and locate the target in each initial video frame of the high-definition video to obtain the center point of each target object;

[0066] S2: Perform nearest neighbor search on each initial video frame, and map each pixel of each initial video frame to a preset color category to obtain the intermediate video frames corresponding to each initial video frame.

[0067] S3: Based on the center point of the target object, detect the bounding box of the target object in each of the intermediate video frames;

[0068] S4: Generate a single-channel feature map for each target object based on the bounding box of each target object;

[0069] S5: Determine the position information of each target object in each intermediate video frame based on each of the single-channel feature maps;

[0070] S6: Update the orbital coordinates of each target object based on the position information of each target object.

[0071] As described in step S1 above, a high-definition video sequence containing multiple target objects (such as billiard balls) is acquired. Here, "high-definition video" refers to a continuous image stream with high resolution and strong image quality, capable of clearly capturing moving targets. After acquiring the high-definition video, target detection is performed on each frame to identify and locate multiple target objects in the video. Target localization can utilize various computer vision techniques, such as deep learning-based target detection algorithms (e.g., YOLO, Faster R-CNN, etc.). These algorithms can be trained to identify and select each target object. The bounding box of each target object can be determined using the algorithm's output, and its center point coordinates can be further calculated.

[0072] As described in step S2 above, a nearest neighbor search is performed on each initial video frame, mapping each pixel of each initial video frame to a preset color category, resulting in intermediate video frames corresponding to each initial video frame. Color classification processing is required for each frame to map each pixel to a preset color category. This process can be achieved through nearest neighbor search. Nearest neighbor search is a feature space-based query technique that can effectively find neighboring pixels similar to a given input. Here, we first need to define preset color categories. For example, different color models can be set based on the colors of billiard balls (such as red, yellow, green, etc.). Next, the algorithm traverses each pixel, calculates the similarity to each color category, finds the nearest color category, and assigns it to the pixel. After completion, the output intermediate video frames will display the color classification results for each pixel, forming a clear color map. This simplifies subsequent target recognition; color information makes it easier to determine the position and changes of target objects.

[0073] As described in step S3 above, the bounding box of the target object is detected in each of the intermediate video frames based on the center point of the target object. Based on the previously identified center point of the target object, the bounding box of each target object is further detected more precisely. The bounding box is used to accurately define the position and size of the target object in the video frame; it is represented by a rectangular coordinate system. Object detection methods such as Haar concatenation, HOG features, and deep learning can be used to obtain the bounding box. In this process, the algorithm uses the center point of the target object as a reference to ensure that the generated bounding box does not deviate from the actual position information of the target. Specifically, the detection algorithm searches for the region with the smallest distance to the center point within the intermediate frame and extracts the bounding box containing the target object from it.

[0074] As described in step S4 above, a single-channel feature map for each target object is generated based on its bounding box. Using the obtained bounding boxes, a single-channel feature map is generated for each target object. The feature map extracts local image content of the target object, representing its feature information such as shape, texture, and color. The process of generating a single-channel feature map typically involves extracting the region within the bounding box, and may involve size normalization and color channel processing. To accommodate the algorithm's input constraints, the feature map may be scaled to a fixed size (e.g., 48×48 pixels) to ensure feature uniformity. The generation of single-channel feature maps emphasizes the importance of feature learning and representation. By capturing the key features of the target object, subsequent models can more accurately identify the characteristics of the target object changing in a dynamic environment. Therefore, the feature map not only provides necessary information for target tracking but also improves the accuracy of subsequent classification and prediction.

[0075] As described in step S5 above, the positional information of each target object in each intermediate video frame is determined based on each of the single-channel feature maps. The previously generated single-channel feature maps are used to determine the positional information of each target object in each intermediate video frame. Convolutional neural networks (CNNs) or other deep learning methods can be used to process the single-channel feature maps to learn and predict the dynamic behavior and position of the target object. The neural network extracts deep features from the feature maps, thereby determining the accurate position of the target object. The advantage of this approach is that the deep learning model can learn to capture the changing characteristics of the target object in different environments through training and react quickly to the target's positional information in dynamic scenes. Specifically, the model outputs the bounding box or center coordinates of the target object, forming updated positional information. By storing this positional information in a data structure, a foundation can be provided for subsequent trajectory updates.

[0076] As described in step S6 above, the orbital coordinates of each target object are updated based on its position information. Based on the previously obtained position information of each target object, the system updates the orbital coordinates of each target object. The orbital coordinates are primarily used to record the continuous motion trajectory of the target object in each frame and ensure that the system can maintain real-time tracking of the target object. The update process typically involves adding newly acquired position information (such as the coordinates of the bounding box center point) to the trajectory data of each target object. This process can use interpolation methods to smooth the trajectory and fill in tracking errors that may occur in certain frames. Furthermore, by analyzing historical trajectory data and current position information, the position of the target in future frames can be predicted, thereby achieving predictive trajectory inference. The continuously updated orbital coordinates not only ensure the consistency of tracking but also provide valuable data support for subsequent decision-making and algorithm optimization.

[0077] In one embodiment, after step S6 of updating the orbital coordinates of each target object based on the position information of each target object, the method further includes:

[0078] S701: Obtain the target position information of each target object in the initial video frame of the last frame of the high-definition video, and calculate the target motion speed of the target object according to the preset Kalman filter;

[0079] S702: Based on the target position information and target movement speed of each target object, predict the prediction area where each target object will appear in the next frame image;

[0080] S703: Within the prediction area, normalized cross-correlation and feature map similarity calculations are used to obtain the predicted location information of each target object;

[0081] S704: Update the orbital coordinates of each of the target objects based on the predicted position information of each of the target objects.

[0082] As described in steps S701-S704 above, the orbital coordinates of the target objects are updated. Specifically, it is necessary to extract the position of the target objects in the initial video frame of the last frame, which can be achieved through the bounding box or center point obtained in previous processing steps. This positional information will be used for subsequent motion state analysis. Furthermore, to comprehensively evaluate the dynamic characteristics of the target objects, the system will utilize a Kalman filter to calculate the velocity of each target object. The Kalman filter is a statistically based recursive filtering algorithm capable of predicting and correcting the state of dynamic systems. In this context, the Kalman filter uses the positional information and time intervals of the target objects in previous frames to estimate the current velocity vector of the target. The velocity vector is typically calculated by dividing the change in position by the time interval, including velocity components in the x and y directions. Using the acquired target positional information and velocity, the possible regions where each target object may appear in the next frame are predicted. This prediction process typically employs a simple dynamic model, such as using linear motion assumptions to extend the current position of the target object. A common practice is to calculate the possible changes in the position of the target object in the next few frames based on its current positional information and velocity. For example, using the formula: Predicted position = Current position + Velocity × Δt, where Δt is the time interval between video frames, the system can generate a prediction region centered on the current target position. This region is typically represented as a rectangle or circle, and its size can be dynamically adjusted based on the target's speed and the video's frame rate. This prediction region not only provides a possible location range for subsequent target confirmation but also provides a basis for the tracking algorithm to quickly resume tracking when the target is lost or occluded, thus improving the overall stability and accuracy of the tracking system. Normalized cross-correlation (NCC) and feature map similarity calculation are used to find the specific location of the target object within the prediction region generated in the previous step. Normalized cross-correlation is a commonly used image matching technique that can overcome lighting variations and target shape deformation to a certain extent while still achieving good matching results. First, the system extracts image markers within the prediction region and compares them with the previously generated single-channel feature map, calculating the similarity between the two. This process typically involves a sliding window operation on local image patches, with normalized cross-correlation calculation performed within the window. Through this process, the system can calculate a similarity score for each possible location in the prediction region and then select the location with the highest score as the predicted location information. This method effectively combines spatial location information and feature representation, enabling the system to accurately capture the specific location information of target objects even in fast-moving or complex background conditions, thus providing precise data for subsequent trajectory updates. The obtained predicted location information is used to update the orbital coordinates of each target object, ensuring that the system reflects the latest position of the target object in each frame.This update typically integrates the predicted position information directly into the orbital data structure, forming a continuous trajectory. In some cases, the system may also use interpolation methods to smooth the trajectory and fill in frame skipping errors that may occur due to target occlusion or rapid movement. Simultaneously, this step provides crucial data support for subsequent motion analysis, helping the system identify the target object's motion patterns and paths. By effectively updating the orbital coordinates, the system can ensure real-time target tracking, improving the reliability and accuracy of the tracking system. Furthermore, it provides a foundation for subsequent analysis and decision-making, such as dynamic simulation and competition strategy evaluation, further enhancing the system's application value.

[0083] In one embodiment, before step S704 of updating the orbital coordinates of each of the target objects based on the predicted position information of each target object, the method further includes:

[0084] S7031: Obtain the similarity value calculated using normalized cross-correlation and feature map similarity within the predicted region;

[0085] S7032: Determine whether the similarity value is lower than a preset similarity threshold;

[0086] S7033: If the similarity is lower than the similarity threshold, the target position information of each target object in the last initial video frame is completed by interpolating the trajectory of adjacent frames.

[0087] As described in steps S7031-S7033 above, location information completion is achieved. Specifically, the previously mentioned Normalized Cross-Correlation (NCC) method and feature map similarity calculation are used to obtain similarity values ​​within the prediction region. First, for each possible location within the prediction region, the system extracts image patches within the region and calculates their similarity with the feature map of the target object. This process is generally accomplished using a sliding window method, that is, small patches are extracted window by window within the prediction region with a certain step size, and similarity matching is performed. The calculation formula of Normalized Cross-Correlation can effectively reduce the influence of illumination changes, viewing angle changes, and shape deformation, thereby improving the robustness of matching. The similarity value obtained by calculating the similarity between each image patch and the feature map is used to determine whether the location can be regarded as a new location of the target object. The obtained similarity values ​​not only reflect the degree of matching of information of each possible location within the prediction region, but also provide key data for subsequent judgment and processing. Each similarity value is compared with a preset similarity threshold. This similarity threshold is a predefined standard, usually set based on experimental and optimization results, used to distinguish between matching and non-matching cases. If the similarity value is higher than the threshold, the system considers the predicted location reliable; otherwise, it indicates that the current predicted location may be incorrect or unreliable. Therefore, matches that do not meet expectations can be filtered out, improving the accuracy and stability of tracking. If the similarity value is lower than the preset similarity threshold, the system needs to fill in the target object's position information in the last frame using trajectory data from adjacent frames. This process is achieved through interpolation, which estimates the values ​​of unknown data points. Therefore, based on known trajectory data, the system can infer the target's possible motion path during the period of lost tracking. Specifically, the system analyzes the target object's position in the previous and subsequent frames and uses linear or polynomial interpolation methods to calculate the target's possible position in the current frame. This can be achieved by mathematically interpolating the known coordinates of adjacent frames to generate a smooth trajectory, resulting in the position where the target object should appear in the current frame. The interpolation result effectively reduces the uncertainty of the target's position caused by sudden factors and ensures that a reasonable estimate of the target is maintained even when the video changes rapidly. This completion method not only improves the consistency of tracking but also further enhances the system's adaptability to fast-moving targets, thereby improving the accuracy of motion analysis.

[0088] In one embodiment, step S702, which predicts the prediction region where each of the target objects will appear in the next frame image based on the target position information and the target motion speed of each target object, includes:

[0089] S7021: Assume the target position information is (x, y), and the target's velocity is (v... x ,v y );

[0090] S7022: Obtain the time interval Δt for the next frame and the preset radius using the formula x′=x+v. x Δt, y′=y+v y Δt is used to calculate the predicted location information;

[0091] S7023: Based on the predicted position information (x′, y′) and the preset radius, according to the formula M={(i,j)∣(ix′)2+(jy′)2≤r 2} Calculate the prediction region; where M is the prediction region, (i,j) represents a point in the prediction region, and r is a preset radius.

[0092] As described in steps S7021-S7023 above, the current position and velocity of the target object are first determined. By combining the current position and velocity, the expected position of the target object in the next frame is calculated based on the time interval. Through this simple and effective linear model, the system can quickly estimate the possible coordinates of the target at future points in time. The importance of this step lies in the fact that it establishes the generation of the prediction region based on the actual motion dynamics, ensuring that the predicted position information is consistent with the motion state of the target object, clarifying the potential position range of the target object in future frames, and forming a dynamic prediction region that can flexibly adapt to the movement of the target. This region not only facilitates rapid target localization but also provides basic data support for subsequent applications such as target detection, target matching, and trajectory updates. By defining such a prediction region, the system can effectively handle the dynamic changes of the target object and ensure the continuity and accuracy of the tracking process.

[0093] In one embodiment, step S1, which involves acquiring high-definition video of multiple target objects and performing target localization on each initial video frame in the high-definition video to obtain the center point of each target object, includes:

[0094] S101: The background region of each initial video frame is segmented and binarized using a preset mean shift algorithm to obtain a binarized video frame corresponding to each initial video frame.

[0095] S102: Locate the center point of the target object in the binarized video frame using a preset target object localization method.

[0096] As described in steps S101-S102 above, background segmentation and binarization are performed on each video frame. The mean-shift algorithm is a commonly used non-parametric estimation method, typically used for image segmentation and clustering. It segments an image by finding the local density of pixels. In this process, the algorithm clusters pixels in the image together to form a clearer distinction between the target and the background. During background segmentation, the mean-shift algorithm analyzes the image's color space (such as RGB or HSV), determines the pixel density in different regions, and forms a binary representation of the video frame accordingly, i.e., a clear contrast between the target and the background. After completing the mean-shift processing, the system performs binarization. During binarization, the original pixel values ​​(e.g., grayscale values) are converted to two values, typically 0 and 1. The background region is set to 0 (black), while the foreground (target object) is set to 1 (white). This transformation makes the target object clearly stand out in the image. The result of the binarization process is a binary frame corresponding to each initial video frame, in which each pixel belongs either to the target object or the background. This lays the foundation for the next step of target object detection and localization, reducing unnecessary computation and improving the efficiency of subsequent processing, resulting in more accurate target object extraction. A pre-defined target object localization method will be used to locate the center point of target objects in the generated binarized video frames. Target object localization is achieved by analyzing pixel clusters in the target region of the binarized image. Common methods include centroid calculation, contour detection, and related deep learning models. Furthermore, if the target objects have complex shapes or are numerous, machine learning methods, such as deep learning-based target recognition technology, can be used to analyze the binarized image to more accurately identify and locate the center point of each target object. This successful localization not only establishes basic data for subsequent trajectory tracking but also provides power support for the target recognition system, helping to effectively manage the tracking and analysis of multiple targets in complex environments.

[0097] In one embodiment, before step S2, which involves performing a nearest neighbor search on each initial video frame and mapping each pixel of each initial video frame to a preset color category to obtain the intermediate video frames corresponding to each initial video frame, the method further includes:

[0098] S111: Obtain a template image and perform KMeans segmentation on the template image to obtain the segmented color regions;

[0099] S112: Divide the color regions to obtain the color category corresponding to each target object.

[0100] As described in steps S111-S112 above, a template image is obtained. This image is typically used to guide the subsequent color classification process. The template image represents the color features of the target object and serves as a reference for the diversity and complexity of the target. A labeled image can be selected from the training dataset, or a typical target image can be obtained from a real-world application as the template image.

[0101] Next, the template image is segmented using KMeans. KMeans is a commonly used clustering algorithm that minimizes the variance within each cluster by dividing data points (pixels in this case) into different clusters. First, the system converts the pixels in the template image into points in a feature space, typically using the RGB or HSV color space as a reference to capture color features. Then, the KMeans algorithm performs the following steps: randomly selects K initial centroids (the number of clusters), which are usually random points in the feature space. Each pixel is assigned to its nearest centroid, forming K clusters. The centroids of the new clusters are calculated, generating new centroids for each cluster. These steps are repeated until the centroids no longer change or the change is less than a given threshold. Through KMeans segmentation, the colors in the template image are divided into multiple distinct color regions, allowing the system to effectively identify the differences between these regions. This process lays the foundation for subsequent target object color classification, enabling the system to accurately identify target objects in video frames based on these color features. Each segmented color region is analyzed, including its color histogram, dominant color, and distribution characteristics. Based on pre-defined color categories (such as red, yellow, green, etc.), the system can classify regions by comparing their color features with standard values ​​for these categories (which could be obtained from previously trained samples). For example, Euclidean distance might be used to calculate the similarity between the color features of different regions and the pre-defined color categories, thereby assigning a specific color label to each region.

[0102] In one embodiment, before step S4 of generating a single-channel feature map of each target object based on the bounding box of each target object, the method further includes:

[0103] S301: Input the bounding box of each target object into a pre-trained CNN classifier to obtain the probability distribution of each target object in each intermediate video frame;

[0104] S302: Based on the probability distribution of each target object in each intermediate video frame, the weighted probability of each target object and its corresponding bounding box is obtained by accumulating the probability according to the time decay weight.

[0105] S303: Obtain the trajectory movement direction of each of the target objects, and determine whether the target object with the highest weighted probability corresponding to the bounding box conflicts with the trajectory movement direction of that target object;

[0106] S304: If the target object's trajectory direction conflicts with the target object's trajectory direction, then the target object corresponding to the bounding box is re-identified.

[0107] As described in steps S301-S304 above, the bounding box of each extracted target object is input into a pre-trained convolutional neural network (CNN) classifier. CNNs are widely used in computer vision tasks and have powerful feature extraction and classification capabilities. The region within the bounding box is cropped and preprocessed, typically including resizing to the input size expected by the CNN and normalizing pixel values. After these processes, the image patch corresponding to the bounding box is pushed as input into the CNN model. The CNN model, trained on a large number of samples, can predict the probability distribution of each category based on the content of the input image. The output probability distribution represents the model's confidence in each category, where the possible categories are target objects (e.g., different types of balls, cues, etc.). For example, the vector output by the model might show that a target object in the current frame has a probability of 0.6 for the category "red ball," a probability of 0.3 for the category "green ball," and lower probabilities for other categories. This information provides rich background for subsequent tracking and recognition of target objects, helping the system determine the nature and identity of targets in dynamic scenes, laying a solid foundation for localization and tracking. The system uses the probability distribution of each target object to calculate and update the weighted probability of each target object and its corresponding bounding box. The temporal decay weighting strategy employed here aims to increase the influence of the current classification result while decreasing the weight of historical classification results, ensuring the model's response to the current state is more sensitive. Specifically, the system considers the probability distribution of each target object in the past few frames and assigns a decay weight to each historical result. This weight value is typically based on exponential decay over time (e.g., using a decay factor of 0.9), resulting in a larger weight for more recent results and a gradually decreasing weight for earlier results. A total weighted probability is obtained by summing the weighted probabilities of each historical probability. This weighted probability provides a dynamic confidence assessment for each bounding box, reflecting the temporal persistence and consistency of the target object's motion, laying the foundation for subsequent processing and ensuring the system's stability and reliability. The motion trajectory and direction of each target object are obtained, usually by inferring the motion trajectory through analysis of historical position information. The motion direction can be represented as a velocity vector calculated from the past few frames, or determined through simple difference calculations, i.e., the difference in position information between two consecutive frames. Understanding the movement state of each target object is crucial for subsequent identification and judgment, as the direction of movement provides contextual information about whether the target is in a normal state. Next, the system needs to determine if the target object with the highest weighted probability within the bounding box matches its direction of movement. If there is a discrepancy between the category with the highest weighted probability and the previously recorded trajectory direction, it may indicate that the current bounding box prediction is inaccurate, the target object has changed, or its position has been mistracked. Through this judgment, the system can detect potential target loss or mistracking early, ensuring appropriate corrections are made in subsequent processing strategies.The system re-identifies the location of the target object corresponding to the current bounding box. At this point, the system takes corrective measures to reflect accurate target recognition. Specifically, the system examines the combined information from the current frame and several previous frames, re-evaluating and adjusting the current bounding box. During this process, various methods based on similarity or classification are employed to identify new target objects. These include using normalized cross-correlation (NCC), feature map similarity calculations to confirm new targets in the current image, or reclassification based on deep learning models. Through these techniques, the system can quickly adapt to target changes, re-identify target identities, and reduce latency caused by target loss or mistracking. The key to this step is accurately identifying the true state of the target, rather than passing erroneous target information to subsequent processing, ensuring the system has stronger adaptability and stability in complex and dynamic environments.

[0108] Reference Figure 3 The present invention also provides a multi-target trajectory tracking device, the device comprising:

[0109] The acquisition module 902 is used to acquire high-definition videos of multiple target objects and perform target localization on each initial video frame in the high-definition video to obtain the center point of each target object.

[0110] The search module 904 is used to perform nearest neighbor search on each of the initial video frames, map each pixel of each initial video frame to a preset color category, and obtain the intermediate video frames corresponding to each initial video frame.

[0111] The detection module 906 is used to detect the bounding box of the target object in each intermediate video frame based on the center point of the target object;

[0112] The generation module 908 is used to generate a single-channel feature map of each target object based on the bounding box of each target object;

[0113] The determination module 910 is used to determine the position information of each target object in each intermediate video frame based on each of the single-channel feature maps;

[0114] The update module 912 is used to update the orbital coordinates of each target object based on the position information of each target object.

[0115] In one embodiment, the multi-target trajectory tracking device further includes:

[0116] The target position information acquisition module is used to acquire the target position information of each target object in the initial video frame of the last frame of the high-definition video, and to calculate the target motion speed of the target object according to the preset Kalman filter.

[0117] The prediction region prediction module is used to predict the prediction region where each of the target objects will appear in the next frame image based on the target position information and the target movement speed of each of the target objects.

[0118] The similarity calculation module is used to obtain the predicted location information of each target object within the prediction region by using normalized cross-correlation and feature map similarity calculation.

[0119] The orbit coordinate update module is used to update the orbit coordinates of each of the target objects based on the predicted position information of each target object.

[0120] In one embodiment, the multi-target trajectory tracking device further includes:

[0121] The similarity value acquisition module is used to acquire the similarity value calculated using normalized cross-correlation and feature map similarity within the predicted region;

[0122] A similarity value determination module is used to determine whether the similarity value is lower than a preset similarity threshold;

[0123] The completion module is used to complete the target position information of each target object in the last initial video frame by interpolating the trajectories of adjacent frames if the similarity is lower than the similarity threshold.

[0124] In one embodiment, the prediction region prediction module includes:

[0125] The assumption submodule is used to assume that the position of the target location information is (x, y) and the target's movement speed is (v). x ,v y );

[0126] The preset radius acquisition submodule is used to obtain the time interval Δt for the next frame and the preset radius, using the formula x′=x+v. x Δt, y′=y+v y Δt is used to calculate the predicted location information;

[0127] The prediction region calculation submodule is used to calculate the predicted region based on the predicted location information (x′, y′) and the preset radius, according to the formula M={(i,j)∣(ix′)2+(jy′)2≤r 2} Calculate the prediction region; where M is the prediction region, (i,j) represents a point in the prediction region, and r is a preset radius.

[0128] In one embodiment, the acquisition module 902 includes:

[0129] The binarization submodule is used to segment the background region of each initial video frame using a preset mean shift algorithm and binarize it to obtain the binarized video frame corresponding to each initial video frame.

[0130] The positioning submodule is used to locate the center point of the target object in the binarized video frame using a preset target object positioning method.

[0131] In one embodiment, the multi-target trajectory tracking device further includes:

[0132] The template image acquisition module is used to acquire a template image and perform KMeans segmentation on the template image to obtain the segmented color regions;

[0133] The segmentation module is used to divide the color regions and obtain the color category corresponding to each target object.

[0134] In one embodiment, the multi-target trajectory tracking device further includes:

[0135] The bounding box input module is used to input the bounding box of each target object into a pre-trained CNN classifier to obtain the probability distribution of each target object in each intermediate video frame.

[0136] The weighted probability calculation module is used to obtain the weighted probability of each target object and its corresponding bounding box based on the probability distribution of each target object in each intermediate video frame and the accumulated probability according to the time decay weight.

[0137] The trajectory motion direction acquisition module is used to acquire the trajectory motion direction of each of the target objects and determine whether the target object with the highest weighted probability corresponding to the bounding box conflicts with the trajectory motion direction of the target object.

[0138] The re-identification module is used to re-identify the target object corresponding to the bounding box if it conflicts with the trajectory movement direction of the target object.

[0139] Figure 4 An internal structural diagram of a computer device in one embodiment is shown. This computer device can specifically be a terminal or a server. Figure 4 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and may also store a computer program. When executed by the processor, this computer program enables the processor to implement a multi-target trajectory tracking method. The internal memory may also store a computer program, which, when executed by the processor, enables the processor to implement the multi-target trajectory tracking method. Those skilled in the art will understand that… Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0140] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the following steps:

[0141] Acquire high-definition videos of multiple target objects, and locate the target in each initial video frame of the high-definition video to obtain the center point of each target object;

[0142] For each initial video frame, a nearest neighbor search is performed, and each pixel of each initial video frame is mapped to a preset color category to obtain the intermediate video frames corresponding to each initial video frame.

[0143] Based on the center point of the target object, detect the bounding box of the target object in each of the intermediate video frames;

[0144] A single-channel feature map of each target object is generated based on the bounding box of each target object;

[0145] The position information of each target object in each intermediate video frame is determined based on each of the single-channel feature maps;

[0146] The orbital coordinates of each target object are updated based on the position information of each target object.

[0147] By acquiring high-definition video of multiple target objects, effective target localization is performed on each frame, and the center point of the target object is quickly determined. This reduces the risk of target loss in the event of brief occlusion or instantaneous movement. Furthermore, based on a simple and fast nearest neighbor search, each pixel is mapped to a preset color category, making the processing of intermediate video frames more efficient. This enables real-time detection of the bounding box of the target object and generation of a single-channel feature map, effectively extracting target features and significantly improving tracking accuracy while reducing computational burden. This allows for real-time processing, meeting the needs of competitions in dynamic environments.

[0148] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, causes the processor to perform the following steps:

[0149] Acquire high-definition videos of multiple target objects, and locate the target in each initial video frame of the high-definition video to obtain the center point of each target object;

[0150] For each initial video frame, a nearest neighbor search is performed, and each pixel of each initial video frame is mapped to a preset color category to obtain the intermediate video frames corresponding to each initial video frame.

[0151] Based on the center point of the target object, detect the bounding box of the target object in each of the intermediate video frames;

[0152] A single-channel feature map of each target object is generated based on the bounding box of each target object;

[0153] The position information of each target object in each intermediate video frame is determined based on each of the single-channel feature maps;

[0154] The orbital coordinates of each target object are updated based on the position information of each target object.

[0155] By acquiring high-definition video of multiple target objects, effective target localization is performed on each frame, and the center point of the target object is quickly determined. This reduces the risk of target loss in the event of brief occlusion or instantaneous movement. Furthermore, based on a simple and fast nearest neighbor search, each pixel is mapped to a preset color category, making the processing of intermediate video frames more efficient. This enables real-time detection of the bounding box of the target object and generation of a single-channel feature map, effectively extracting target features and significantly improving tracking accuracy while reducing computational burden. This allows for real-time processing, meeting the needs of competitions in dynamic environments.

[0156] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0157] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0158] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A multi-target trajectory tracking method, characterized in that, The method includes: Acquire high-definition videos of multiple target objects, and locate the target in each initial video frame of the high-definition video to obtain the center point of each target object; For each initial video frame, a nearest neighbor search is performed, and each pixel of each initial video frame is mapped to a preset color category to obtain the intermediate video frames corresponding to each initial video frame. Based on the center point of the target object, detect the bounding box of the target object in each of the intermediate video frames; A single-channel feature map of each target object is generated based on the bounding box of each target object; The position information of each target object in each intermediate video frame is determined based on each of the single-channel feature maps; The orbital coordinates of each target object are updated based on the position information of each target object.

2. The multi-target trajectory tracking method according to claim 1, characterized in that, After the step of updating the orbital coordinates of each target object based on the position information of each target object, the method further includes: The target position information of each target object in the last frame of the high-definition video is obtained, and the target motion speed of the target object is calculated according to the preset Kalman filter. Based on the target position information and target movement speed of each target object, predict the prediction area where each target object will appear in the next frame image; Within the prediction region, normalized cross-correlation and feature map similarity are used to calculate the predicted location information of each target object. The orbital coordinates of each target object are updated based on the predicted position information of each target object.

3. The multi-target trajectory tracking method according to claim 2, characterized in that, Before the step of updating the orbital coordinates of each target object based on the predicted position information of each target object, the method further includes: Obtain the similarity value calculated using normalized cross-correlation and feature map similarity within the predicted region; Determine whether the similarity value is lower than a preset similarity threshold; If the similarity is below the threshold, the target position information of each target object in the last initial video frame is completed by interpolating the trajectories of adjacent frames.

4. The multi-target trajectory tracking method according to claim 2, characterized in that, The step of predicting the prediction region where each of the target objects will appear in the next frame image based on the target position information and the target motion speed of each target object includes: Assuming the target's position is (x, y) and its velocity is (v... x ,v y ); The time interval for obtaining the next frame is Δt, and the preset radius is used by the formula x′=x+v. x Δt, y′=y+v y Δt is used to calculate the predicted location information; Based on the predicted location information (x′, y′) and the preset radius, according to the formula M={(i,j)∣(ix′)2+(jy′)2≤r 2 } Calculate the prediction region; where M is the prediction region, (i,j) represents a point in the prediction region, and r is a preset radius.

5. The multi-target trajectory tracking method according to claim 1, characterized in that, The step of acquiring high-definition video of multiple target objects and locating the target in each initial video frame of the high-definition video to obtain the center point of each target object includes: The background region of each initial video frame is segmented and binarized using a preset mean shift algorithm to obtain a binarized video frame corresponding to each initial video frame. The center point of the target object in the binarized video frame is located using a preset target object localization method.

6. The multi-target trajectory tracking method according to claim 1, characterized in that, Before the step of performing nearest neighbor search on each initial video frame and mapping each pixel of each initial video frame to a preset color category to obtain the intermediate video frames corresponding to each initial video frame, the method further includes: Obtain a template image and perform KMeans segmentation on the template image to obtain the segmented color regions; The color regions are divided to obtain the color category corresponding to each target object.

7. The multi-target trajectory tracking method according to claim 1, characterized in that, Before the step of generating a single-channel feature map of each target object based on the bounding box of each target object, the method further includes: The bounding box of each target object is input into a pre-trained CNN classifier to obtain the probability distribution of each target object in each intermediate video frame. Based on the probability distribution of each target object in each intermediate video frame, the weighted probability of each target object and its corresponding bounding box is obtained by accumulating the probability according to the time decay weight. Obtain the trajectory movement direction of each target object, and determine whether the target object with the highest weighted probability corresponding to the bounding box conflicts with the trajectory movement direction of that target object; If the target object's trajectory direction conflicts with the target object's trajectory direction, the target object corresponding to the bounding box will be re-identified.

8. A multi-target trajectory tracking device, characterized in that, The device includes: The acquisition module is used to acquire high-definition videos of multiple target objects and perform target localization on each initial video frame in the high-definition video to obtain the center point of each target object. The search module is used to perform nearest neighbor search on each of the initial video frames, map each pixel of each initial video frame to a preset color category, and obtain the intermediate video frames corresponding to each initial video frame. The detection module is used to detect the bounding box of the target object in each intermediate video frame based on the center point of the target object; A generation module is used to generate a single-channel feature map of each target object based on the bounding box of each target object; The determination module is used to determine the position information of each target object in each intermediate video frame based on each of the single-channel feature maps; The update module is used to update the orbital coordinates of each target object based on the position information of each target object.

9. A computer-readable storage medium, characterized in that, The system contains a computer program that, when executed by a processor, causes the processor to perform the steps of the multi-target trajectory tracking method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, The device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the multi-target trajectory tracking method as described in any one of claims 1 to 7.

Images