A multi-target tracking method for complex scenes

Abstract

The application relates to the image processing technical field of computer vision, and discloses a multi-target tracking method for a complex scene, which comprises the following steps: acquiring a target image frame; constructing an HND-MOTR model; inputting the target image frame into the HND-MOTR model, introducing a hard negative example denoising training paradigm for training, and generating a trained HND-MOTR model; acquiring a to-be-detected image, inputting the to-be-detected image into the trained HND-MOTR model, and realizing multi-target tracking identification; the application realizes accurate tracking of multi-targets in a complex scene, significantly improves feature discriminability, takes into account new target discovery capability and long-term stability of a track, improves complex scene adaptability, and realizes robust tracking of targets in a complex crowded scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing technology of computer vision, and particularly relates to a multi-target tracking method for a complex scene. BACKGROUND

[0002] Multi-Object Tracking (MOT) is a fundamental task in computer vision, aiming to detect and persistently associate targets of interest across video frames. It plays a key role in a wide range of practical applications such as autonomous driving, video surveillance, smart retail, and sports analysis. Although significant progress has been made in the traditional tracking-by-detection paradigm, their robustness is usually limited under challenging conditions such as occlusion and frequent target entry and exit, mainly due to the inconsistency between the feature requirements of the detection module and the association module.

[0003] In recent years, query-based end-to-end tracking methods have become the mainstream approach. This paradigm unifies target detection and temporal association within one network, enabling joint modeling and thus improving overall optimization efficiency and performance. Existing methods under this framework can be generally divided into two categories. The first category, represented by MeMOTR, is good at long-term temporal modeling. It introduces memory-augmented track queries to support explicit cross-frame propagation, effectively handling occlusions. However, this class of methods usually relies on static query initialization, lacks adaptability to new targets, and is sensitive to initial localization, often leading to tracking failure. The second category, exemplified by MOTRv2, adopts a dynamic, proposal-driven query generation strategy. By leveraging a powerful external detector (e.g., YOLOX5), it improves robustness in target detection and initialization. However, it lacks a structured long-term memory mechanism and instead relies on short-term propagation between adjacent frames, making it prone to identity switches under long-term occlusion or rapid motion.

[0004] On this basis, the prior art also proposes an end-to-end multi-target tracking method, which has made significant progress, but in the pursuit of the unified goal of high-quality target detection and robust long-term temporal association, there are still the following problems:

[0005] (1) Limitations in target initialization and recall rate; for example, the time sequence focus method relies on static, learnable queries to discover new targets. The design of this mechanism is intended to ensure that the model can maintain stable long-term memory, but due to insufficient coupling between the query itself and the image information of the current frame, the detection recall rate and initialization accuracy of new targets are relatively low. In scenarios where targets frequently enter and exit, the model lacks dynamic adaptability and is difficult to effectively capture and initialize new trajectories, affecting the integrity of tracking;

[0006] (2) Vulnerability in long-term identity preservation; for example, the proposal-driven method relies on an external detector to generate dynamic proposals to enhance detection capabilities, but its core architecture lacks a structured long-term memory module. It mainly relies on short-term self-recurrent propagation between adjacent frames for correlation. This design makes it difficult for the model to effectively utilize historical information for trajectory recovery when faced with long-term occlusion, sudden changes in target appearance, or rapid crossing of motion trajectories, easily causing identity switching or trajectory breaks, compromising the long-term robustness of tracking;

[0007] (3) Insufficient feature discriminability in existing training paradigms: current end-to-end trackers (including the above two types) mainly use standard detection losses and basic denoising tasks for training. These training objectives focus on positioning accuracy and rough classification, but are inherently insensitive to discriminability. In crowded, high-similarity scenarios such as multiple dancers or multiple athletes, the visual differences between instances are subtle, and there is a lack of targeted discriminative constraints, making the feature representation learned by the model insufficient to distinguish between spatially adjacent or highly similar individuals, becoming a major bottleneck for maintaining identity consistency in high-density tracking scenarios. SUMMARY

[0008] To overcome the above-mentioned deficiencies in the prior art, the present application provides a multi-target tracking method for complex scenes, which solves the problem of insufficient feature discriminability and insufficient robustness of target tracking in complex scenes in existing multi-target tracking methods.

[0009] To achieve the above-mentioned purposes, the technical scheme adopted by the present application is as follows:

[0010] A multi-target tracking method for complex scenes, comprising the following steps:

[0011] S1, obtaining a target image frame;

[0012] S2, constructing an HND-MOTR model;

[0013] S3, inputting the target image frame with real labels into the HND-MOTR model, introducing a hard negative example denoising training paradigm for training, and generating a trained HND-MOTR model;

[0014] S4. Acquire the image to be detected and input it into the trained HND-MOTR model to achieve multi-target tracking and recognition.

[0015] The present invention has the following beneficial effects:

[0016] 1. The multi-target tracking method proposed in this invention for complex scenes constructs an HND-MOTR model, combines HND training paradigm design, and uses K-NN random sampling for hard negative example generation and multi-level noise injection. It specifically solves the problem of distinguishing similar targets in crowded scenes, effectively reduces the identity switching rate, and significantly improves feature discriminativeness.

[0017] 2. This invention also achieves collaborative optimization of detection and temporal modeling. In the temporal interaction module of the HND-MOTR model, a proposal anchoring update mechanism is introduced to integrate a proposal-driven strategy with high recall, which improves the robustness of the temporal interaction module, balances the ability to discover new targets and the long-term stability of trajectories, and enhances the adaptability to complex scenarios.

[0018] 3. This invention also achieves end-to-end efficiency advantages. By unifying the detection and feature association process, it avoids the error accumulation caused by the separation of detection and association in traditional methods, thereby improving inference efficiency while ensuring accuracy.

[0019] 4. This invention also enhances the generalization ability of the constructed HND-MOTR model. Through multi-source query fusion and adaptive noise injection design, it exhibits excellent performance on complex benchmark datasets and is adaptable to different congestion and occlusion scenarios. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a multi-target tracking method for complex scenarios proposed in this invention.

[0021] Figure 2 This is a schematic diagram of the HND-MOTR model in the embodiment;

[0022] Figure 3 This is a schematic diagram of the K-NN random sampling strategy used for generating hard negative examples in the embodiment;

[0023] Figure 4 This is a schematic diagram of multi-level noise injection in the embodiment;

[0024] Figure 5 This is a schematic diagram of the collaborative mechanism between the proposed query and the timing interaction in the embodiment;

[0025] Figure 6This is a schematic diagram illustrating the ablation study results of the number of neighbors in K-NN random sampling on the performance of the DanceTrack validation set in the example;

[0026] Figure 7 This is a visual illustration of trajectory embedding using the t-SNE method in the embodiment. Detailed Implementation

[0027] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0028] like Figure 1 As shown, a multi-target tracking method for complex scenarios includes the following steps S1-S4:

[0029] S1. Obtain the target image frame.

[0030] In this embodiment, the purpose of obtaining the target image frame is to train the HND-MOTR model constructed in subsequent steps, and the target image frame has a real label, which includes the target ID, bounding box position, size and confidence level.

[0031] S2. Construct the HND-MOTR model.

[0032] In this embodiment, the structure and connection relationships of the HND-MOTR model are as follows: Figure 2 As shown, it includes a feature extraction module, a Transformer decoder, a Temporal Interaction Module (TIM), and a prediction head.

[0033] The feature extraction module includes a backbone network and a Transformer encoder; the feature extraction module is used to extract features, specifically: extracting features from the current image frame. First, the image features are extracted and encoded using a shared backbone network (e.g., ResNet-50) and a Transformer encoder. .

[0034] Furthermore, the HND-MOTR model includes multiple query generation modules, including a proposal query generation module and a learnable detection query generation module. When training with a hard negative example denoising training paradigm, the HND-MOTR model also includes an HND query generation module. The purpose of this is to address the common problem of insufficient feature discriminativeness in existing training paradigms. Current end-to-end trackers primarily use standard detection losses and basic denoising tasks for training, focusing on localization accuracy and coarse classification, but are essentially insensitive to discriminativeness. In crowded, highly similar scenes (such as multiple dancers or athletes), the visual differences between instances are subtle, and the lack of targeted discriminative constraints results in the model's learned feature representations being insufficient to distinguish spatially adjacent or visually highly similar individuals, becoming a major bottleneck in maintaining identity consistency in high-density tracking scenarios. Therefore, based on this, it is necessary to introduce an HND query generation module during model training to improve the model's feature discriminativeness.

[0035] Among them, multiple query generation modules are used to generate queries, specifically: at each time step The query generation integrates four types of queries, including: 1) Tracking queries ( ), from the previous frame Propagated via the time-series interaction module, it encodes the identity and historical state of previous targets; 2) Learnable detection query ( ), used for general object detection, to prevent detectors from missing detections, and the learned detection query is generated in the learnable detection query generation module, specifically: using the nn.Embedding(num_det, dim) function, by randomly initializing the embedding, a learnable detection query is generated so that the ability to find objects in image frames can be trained on data later, where num_det represents the number of detection queries and dim represents the feature dimension; 3) Proposed query ( ), generated by a high-performance external detector (e.g., YOLOX) to maximize the recall of new targets; 4) HND query It is added only during the training phase for discriminative learning; ultimately, the proposed query, learnable detection query, HND query, and tracking query are concatenated to form the joint query set input to the Transformer decoder. .

[0036] The Transformer Decoder is used to unify detection and feature modeling. Specifically, it processes the joint query set... With multi-scale image feature maps The data is fed into a Transformer decoder, which interacts through multi-layered self-attention and cross-attention mechanisms to relocalize existing trajectories, discover new targets, and output decoding results containing accurate bounding boxes and identity embeddings. Finally, the decoding result output by the Transformer decoder is... The localization, classification, and identity embedding information in the image are processed by the corresponding prediction heads, and the target ID, bounding box position, size, and confidence score of the current image frame are directly output. That is, target detection and feature association are completed uniformly in the Transformer decoder.

[0037] S3. Input the target image frame into the HND-MOTR model, introduce the hard negative example denoising training paradigm for training, and generate the trained HND-MOTR model.

[0038] In this embodiment, addressing the lack of discriminative supervision in existing training paradigms, the present invention introduces a hard negative example denoising training paradigm that systematically constructs the most challenging hard negative examples through K-NN random sampling and multi-level noise injection. This design explicitly forces the model to distinguish between highly similar and spatially adjacent distractors and the real target, thereby guiding it to learn fine-grained appearance and contextual features during training. This mechanism directly solves the identity confusion problem caused by feature ambiguity in crowded scenes, forming denser and more separated identity clusters in the feature space, significantly enhancing feature discriminativeness. The specific training process of the HND-MOTR model is as follows:

[0039] Specifically, step S3 includes:

[0040] The current image frame is input into the backbone network and Transformer encoder respectively for feature extraction, generating multi-scale image feature maps. .

[0041] Input the current image frame into the HND query generation module to generate the HND query for the current image frame. Specifically:

[0042] During training, several denoising groups are constructed, and each denoising group includes the same number of positive queries and negative queries. Positive queries are generated by applying a small perturbation to the coordinates of the real label boxes. A set proportion of negative queries are used as hard negative queries, and the remaining proportion of negative queries are used as perturbed false positive queries.

[0043] Then, the K-NN random sampling method is used to generate hard negative example queries. Specifically, for the ground truth bounding box used to generate the hard negative example query, the Euclidean distance between the center coordinates of the ground truth bounding box and the remaining ground truth bounding boxes in the same image frame is calculated and randomly replaced to generate the hard negative example query.

[0044] Specifically, the formula for calculating the Euclidean distance between the center coordinates of the ground truth label box and the remaining ground truth label boxes in the same image frame is as follows:

[0045]

[0046] in, Represents the actual label box To the actual label box The center coordinates of the Euclidean distance , These represent the actual label boxes. center coordinates x-coordinate value, y-coordinate value , These represent the actual label boxes. center coordinates x-coordinate value, y-coordinate value Indicates condition, This indicates the total number of actual label boxes.

[0047] Simultaneously, standard noise is injected into positive queries, maximum noise is injected into disturbed false positive queries, and minimum noise is injected into hard negative queries; finally, the HND query of the current image frame is obtained.

[0048] In this embodiment, the present invention proposes a Hard Negative Denoising (HND) training paradigm to address the common problem of insufficient feature discriminativeness in the training of existing end-to-end multi-object tracking (MOT) models. Since existing methods are typically based on strategies such as Contrast Denoising (CDN), their core idea is to train the decoder to reconstruct the original target from the injected noisy ground truth labels (GT) to stabilize training and accelerate convergence. However, directly applying the original contrastive denoising strategy to multi-object tracking exposes significant limitations. First, contrastive denoising involves bounding box perturbation and label scrambling, but in multi-object tracking MOT scenarios—typically limited to a single object category (e.g., "person")—label corruption becomes largely ineffective. More importantly, negative samples in standard contrastive denoising CDNs are simply ground truth label boxes injected with significant noise. This training encourages the model to correct coarse localization errors rather than distinguish highly similar and spatially adjacent instances—which is precisely the core challenge in crowded multi-object tracking MOT settings.

[0049] Inspired by contrastive learning forms of contrastive denoising, and recognizing its shortcomings in tracking, this invention proposes Hard Negative Example Denoising (HND). The goal of this invention is to redefine denoising from a signal reconstruction task as a powerful engine for discriminative representation learning. Hard Negative Example Denoising is not merely about removing noise, but about explicitly teaching the model to distinguish between the real target and its most confusing imposter, thereby directly addressing the root cause of identity switching in dense tracking scenarios.

[0050] First, hard negative example denoising groups are constructed and mask attention is applied. Hard negative example denoising (HND) operates by constructing dedicated denoising groups during training, specifically: given a set of denoised groups... Frames containing ground truth (GT) objects (ground truth bounding boxes) are used to create multiple denoising groups, each containing... One query, that is, including Positive queries and There are 10 negative queries, all of which are derived from the ground truth label annotations. Positive queries are generated by applying small perturbations to the coordinates of the ground truth label boxes. They are used to reconstruct the original target, thereby encouraging the model to learn robust localization. Negative queries are designed to simulate common MOT failure cases in multi-object tracking. Negative queries are further divided into two types: a portion (e.g., 30%) are replaced with hard negative queries, while the rest are retained as perturbed false positives, i.e., unreplaced negative queries.

[0051] like Figure 3 The diagram illustrates the process of generating hard negative examples using K-NN random sampling. To realistically simulate identity switching scenarios, the bounding box coordinates of the hard negative example query are replaced with the coordinates of neighboring targets. This invention does not rely on deterministic heuristics (e.g., selecting a single nearest neighbor), but instead employs a K-Nearest Neighbor (K-NN) random sampling method to generate more diverse and challenging negative samples; for each ground truth label box selected for generating hard negative examples... (Real label object), first from the remaining ones in the same frame Identify the K nearest neighbors of an object; where the distance is determined by the center coordinates. and The distance between them is measured by the Euclidean distance, that is:

[0052]

[0053] in, Represents the actual label box To the actual label box The center coordinates of the Euclidean distance , These represent the actual label boxes. center coordinates x-coordinate value, y-coordinate value , These represent the actual label boxes. center coordinates x-coordinate value, y-coordinate value Indicates condition, This indicates the total number of actual label boxes.

[0054] Then, a neighbor is randomly sampled from this candidate pool. (Real label box) ), and use its bounding box coordinates to replace the original source (Real label box) The hard negative instance of the query is the coordinates. This design forces the model to face a challenging situation: the query content originates from the target. However, its spatial location corresponds to a semantically different but spatially adjacent target. This setup effectively simulates real-world identity switching and forces the model to learn that such conflicting pairings are incorrect.

[0055] Therefore, the model is encouraged to develop more robust discriminative feature representations to handle appearance similarity and spatial proximity; hence, multi-level noise injection is implemented, meaning the design of the noise injection strategy plays a crucial role in the effectiveness of HND, especially in densely interactive scenarios. Applying a uniform noise scale to all queries has proven to be suboptimal. To address this issue, this invention proposes a multi-level noise injection mechanism, such as... Figure 4As shown, the perturbation strength depends on the query type: 1) Positive queries are injected with a standard level of noise to encourage robust regression; 2) Unreplaced negative queries (i.e., perturbed pseudo-positives) are injected with large perturbations (maximum noise) to simulate simple negative cases; 3) Replaced pseudo-positives (i.e., potential hard negatives) are injected with minimal noise to generate high-fidelity perturbations. Notably, injecting slight noise into replaced pseudo-positive queries is a key design feature. This invention observes that when these pseudo-positives are undisturbed and perfectly aligned with the ground truth bounding boxes (although not a true match), the model tends to exploit this spatial alignment as a shortcut. Specifically, it may implicitly learn a non-discriminatory rule, such as: "A query that is perfectly aligned with the ground truth bounding box (GT box) but not labeled as a positive must be a negative." This behavior bypasses fine-grained appearance modeling, weakening the model's discriminative ability in complex environments. Therefore, by introducing small perturbations, this perfect spatial alignment is intentionally disrupted, transforming pseudo-positives into near-perfect perturbations. This forces the model to rely on contextual and appearance cues for recognition, thus significantly improving robustness under occlusion and crowding conditions. Notably, the precise replacement strategy and noise level can be flexibly adapted to different datasets or tracking frameworks.

[0056] In summary, this step introduces an HND query generation module during the training phase, and improves the model's feature discriminativeness by constructing denoised groups, generating hard negative examples through K-NN random sampling, and employing a multi-level noise injection strategy.

[0057] Input the current image frame into the proposal query generation module to generate a proposal query for the current image frame. .

[0058] Input the previous image frame into the timing interaction module to generate a tracking query for the previous image frame. .

[0059] The learnable detection query generation module generates learnable detection queries by randomly initializing them based on the number of detection queries and feature dimensions. .

[0060] In this embodiment, the learnable detection query generation module generates learnable detection queries by randomly initializing embeddings based on the nn.Embedding(num_det, dim) function. Among them, the nn.Embedding module is naturally suitable for creating a set of learnable vectors with a fixed number and fixed dimensions, which can meet the initialization requirements of detection queries. At the same time, it facilitates subsequent steps to train the query to have the ability to find objects in image frames through data training, which fits the optimization goal of learnable parameters, that is, to adapt the query vector to the target feature matching task through end-to-end training.

[0061] HND query Suggested inquiry Tracking and querying Learning, testing, and query capabilities are available. Concatenate the results to generate a union query set. .

[0062] Union query set Multi-scale image feature maps The input is processed by the Transformer decoder, which interacts with the multi-layer self-attention mechanism and cross-self-attention mechanism to output the decoding result containing bounding boxes and identity embeddings. .

[0063] Decoding results containing bounding boxes and identity embeddings Input the timing interaction module to update the status and generate the tracking query for the next image frame, so as to update the tracking query of the current image frame.

[0064] In this embodiment, after the Transformer encoder performs a unified decoding process, the output corresponding to the active tracking query is the decoding result. The state update is passed through the Temporal Interaction Module (TIM). This update mechanism is applied to the HND-MOTR model constructed in this invention, and the synergy of this hybrid architecture becomes particularly effective at a key stage: when a previously tracked target reappears after occlusion, the proposal-driven mechanism is highly likely to generate a strong detection proposal for it. Due to unified decoding, the corresponding tracking query can directly interact with the rich features of this high-confidence proposal. Therefore, the input fed to the TIM for state update already has accurate localization and rich features. This enables the TIM to refine the trajectory memory and state more accurately and robustly. Therefore, this invention calls the update mechanism of the TIM the Proposal-Anchored Update Mechanism, which transforms the TIM from a passive memory aggregator into an active refiner utilizing strong real-time detection evidence. Compared with autoregressive propagation based purely on tracking queries, this mechanism provides significantly higher robustness—especially when the target undergoes sudden movement or appearance changes.

[0065] like Figure 5 As shown, the decoding result includes bounding boxes and identity embeddings. The process of updating the tracking query of the current image frame by inputting the timing interaction module, performing state updates, and generating the tracking query for the next image frame is as follows:

[0066] Decoding results containing bounding boxes and identity embeddings Tracking query with the previous image frame The images are stitched together to obtain short-term memory of the current image frame, while the trajectory state of the current image frame is used as long-term memory.

[0067] Use the short-term memory of the current image frame as the query vector, and decode the result. As a value vector, the trajectory state of the current image frame is used as a key vector.

[0068] The query vector, value vector, and key vector are input into the attention layer. The relevance weights between the query vector, value vector, and key vector are calculated and then weighted and fused to generate fused features.

[0069] After summing the fused features with the long-term memory of the current image frame pixel by pixel, the result is input into a fully connected layer for feature integration, generating a tracking query for the current image frame.

[0070] The exponential moving average method is used to smooth the parameters of the long-term memory of the current image frame, generating the long-term memory of the next image frame, thus obtaining the tracking query of the next image frame and realizing the update of the tracking query of the current image frame.

[0071] Specifically, the long-term memory of the current image frame is smoothed using the exponential moving average method to generate the long-term memory of the next image frame. Therefore, the formula for tracking and querying the next image frame is:

[0072]

[0073] in, Represents long-term memory of the next image frame. Represents the smoothing coefficient. This represents the long-term memory of the current image frame. This represents the decoding result of the current image frame, including bounding boxes and identity embeddings, output by the Transformer decoder.

[0074] Finally, the decoding result, including the bounding box and identity embedding, will be... The prediction head is input to make predictions, generating the target ID, bounding box position, size, and confidence score of the current image frame, completing the training of the HND-MOTR model, and obtaining the trained HND-MOTR model.

[0075] S4. Acquire the image to be detected and input it into the trained HND-MOTR model to achieve multi-target tracking and recognition.

[0076] In summary, the multi-target tracking method proposed in this invention for complex scenes constructs an HND-MOTR model. By synergistically combining a proposal-anchored temporal interaction architecture with a hard negative example denoising (HND) training paradigm, it effectively addresses the limitations of existing query-based MOT frameworks in simultaneously achieving high recall detection and fine-grained identity discrimination. This results in robust tracking in complex and crowded scenes. Specifically:

[0077] First, at the architectural level, the HND-MOTR model combines a proposal-driven mechanism and a long-term memory module to form a proposal anchoring update mechanism. This mechanism uses high-confidence proposals generated by an external high-performance detector as anchors to calibrate the trajectory state of occluded targets. In inference, it ensures that trajectory recovery and identity re-identification are more robust after long-term occlusion, significantly reduces identity switching (IDs) caused by target re-enactment or abrupt movement, and improves the ability to initialize new targets and the robustness of trajectory recovery.

[0078] Secondly, addressing the lack of discriminative supervision in existing training paradigms, this invention introduces the HND training paradigm, which systematically constructs the most challenging hard negative examples through K-NN random sampling and multi-level noise injection. This design explicitly forces the model to distinguish between highly similar and spatially adjacent interference items and the real target, thereby guiding it to learn fine-grained appearance and contextual features during training. This mechanism directly solves the identity confusion problem caused by feature ambiguity in crowded scenes, forming denser and more separated identity clusters in the feature space. This significantly enhances feature discriminativeness and maintains high tracking accuracy and consistent identity in complex scenes, demonstrating good generalization and application prospects.

[0079] In this embodiment, to verify the effectiveness of the multi-target tracking method for complex scenarios proposed in this invention, the following experiments were conducted:

[0080] I. Experimental Data:

[0081] Extensive experiments were conducted on two large-scale and challenging multi-object tracking datasets: the DanceTrack dataset and the SportsMOT dataset. These benchmarks were chosen because of their inherent difficulty, including densely populated scenes, frequent occlusion, and high inter-object similarity—factors that pose significant challenges to robust tracking and consistent identity preservation.

[0082] The DanceTrack dataset is a large-scale benchmark designed specifically for multi-dancer tracking. This dataset presents unique challenges: high visual similarity between dancers, non-linear and unpredictable motion patterns, and severe occlusion due to complex group choreography. It contains 100 video sequences, with 40 used for training, 25 for validation, and 35 for testing. All videos were filmed at different viewpoints and scales and contain only a single object category: dancers. The high density and frequent interactions of the objects make DanceTrack an ideal benchmark for evaluating a tracker's discriminative feature modeling and identity preservation capabilities.

[0083] The SportsMOT dataset focuses on tracking multiple targets in various sports scenarios, such as basketball and volleyball games. This dataset is characterized by athletes wearing visually similar uniforms within a limited playing space, and their movements are fast-paced and sometimes unstable. It contains 240 video sequences, with 45 used for training, 45 for validation, and 150 for testing. Similar to the DanceTrack dataset, it primarily involves tracking a single object category: athletes. The inherent fast movement and cluttered backgrounds of sports provide a rigorous testing ground for evaluating the tracker's ability to handle appearance variations and motion complexity.

[0084] A comprehensive set of standard multi-target tracking (MOT) evaluation metrics is used to assess the overall performance of the model constructed by the proposed method. The focus is primarily on the HOTA (Higher Order Tracking Accuracy) 10 metric, which provides a more balanced evaluation between detection accuracy, association quality, and localization accuracy. For detailed comparison with previous work, the widely used CLEAR metric and IDF1 score are also reported.

[0085] HOTA, as the primary evaluation metric, provides a unified standard that considers accurate detection, successful association, and precise localization. It is defined as the geometric mean of detection accuracy (DetA) and association accuracy (AssA):

[0086] ;

[0087] Compared to traditional metrics, HOTA provides a more comprehensive and interpretable assessment of tracking performance.

[0088] This invention employs several classic CLEAR metrics for more detailed analysis. Among them, MOTA (Multi-Object Tracking Accuracy) is a key metric, aggregating three types of tracking errors: false positives (FP), false negatives (FN), and identity switching (IDs). It is formally defined as:

[0089]

[0090] Here, GT represents the total number of real tagged objects; special emphasis is placed on the IDs count, because a lower value directly reflects a stronger ability to maintain identity.

[0091] IDF1 is another important metric that highlights the tracker's association quality and long-term identity consistency. It is defined as the harmonic mean of Identity Precision (IDP) and Identity Recall (IDR):

[0092]

[0093] Here, IDTP, IDFP, and IDFN represent the number of true positives, false positives, and false negatives in identity matching, respectively. A high IDF1 score strongly demonstrates the effectiveness of the HND paradigm proposed in this invention in improving identity consistency under complex conditions.

[0094] II. Experimental Procedure:

[0095] Regarding the DanceTrack dataset, the evaluation was first conducted on the highly challenging DanceTrack test set, which is characterized by non-linear motion patterns and severe visual blurring between dancers. These characteristics make it an ideal benchmark for evaluating the effectiveness of the feature discriminative enhancement strategy proposed in this invention. The comparison process is shown in Table 1:

[0096] Table 1. Comparison of the HND-MOTR model with other methods on the DanceTrack test set.

[0097]

[0098] Among them, tracking-by-query is a query-based model, i.e., a transformer-based model. As shown in Table 1, the HND-MOTR model achieved superior performance on all major evaluation metrics. Compared with the strong baseline MOTRv2, the proposed method achieved a significant improvement of +1.9% on HOTA and +2.8% on IDF1, highlighting substantial enhancements in overall tracking accuracy and long-term identity preservation. The performance improvement is further supported by a significant reduction in the number of identity switching (IDs), directly demonstrating the effectiveness of the proposed HND training paradigm in improving feature discriminativity.

[0099] Furthermore, compared to MeMOTR, which also employs a temporal interaction module, HND-MOTR still exhibits significant advantages, particularly in AssA (Association Accuracy) and IDF1. These results demonstrate that, although temporal modeling remains important, the HND strategy proposed in this invention introduces crucial discriminative capabilities lacking in previous methods.

[0100] To further evaluate the robustness and generalization ability of the proposed method on the SportsMOT dataset, HND-MOTR was evaluated on the SportsMOT test set, which features fast-moving athletes and cluttered backgrounds. The comparison process is shown in Table 2.

[0101] Table 2. Comparison of the HND-MOTR model with other methods on the SportsMOT test set.

[0102]

[0103] Table 2 further confirms the effectiveness of the proposed method under dynamic environments and various conditions. The proposed HND-MOTR model consistently achieves competitive or superior performance compared to other leading methods. Specifically, compared to the strong baseline method MeMOTR, the proposed model shows a significant improvement of 1.7% in HOTA and 1.0% in MOTA. Most importantly, HND-MOTR achieves a highly competitive IDF1 score of 73.4%. In sports scenarios, athletes wearing similar uniforms frequently cross paths, making excellent performance on identity center metrics (such as IDF1) crucial. The proposed model excels in this regard, demonstrating its superior ability to handle identity preservation in highly dynamic environments. These results indicate that the advantages of the HND training paradigm are not limited to a single domain but provide a generalizable solution for enhancing feature discriminability in challenging MOT scenarios.

[0104] III. Ablation Experiment:

[0105] A series of ablation experiments were conducted to thoroughly investigate the independent and synergistic effects of the components of the proposed HND-MOTR model. First, the impact of architecture fusion and the HND training paradigm under different data configurations was evaluated. Then, key hyperparameters related to HND were analyzed in depth. Unless otherwise stated, all experiments were performed on the DanceTrack validation set. To systematically evaluate the contribution of the proposed method, two sets of control experiments were designed to decouple the effects of architecture fusion and HND training. To maintain clarity in the ablation study, the augmented architecture (MeMOTR+Proposals) trained using the standard strategy is referred to as Hybrid-MOTR; the results are summarized in Tables 3 and 4.

[0106] Table 3. Ablation experiments of the proposed architecture fusion on the DanceTrack dataset.

[0107]

[0108] As shown in Table 3, structural augmentation analysis was performed using MeMOTR as the baseline model. When only the YOLOX-based proposal generation mechanism was introduced to construct Hybrid-MOTR, the model performance was significantly improved. Compared to the baseline model, the HOTA score of Hybrid-MOTR increased from 68.5 to 69.7, and the MOTA score increased to 91.3. This indicates that integrating a high-recall proposal mechanism into a long-term temporal framework can effectively improve the initialization accuracy of new targets and provide more reliable feature anchors for the re-identification of occluded targets, validating the significant benefits of structural fusion for tracking stability.

[0109] Table 4 shows the main results and ablation studies on the DanceTrack+CrowdHuman combined dataset.

[0110]

[0111] To fully validate the effectiveness of HND, a combined dataset of DanceTrack and CrowdHuman was used for training. As shown in Table 4, MOTRv2 (HOTA 69.9) was used as a strong baseline. When using the proposed Hybrid-MOTR architecture (i.e., without HND), the HOTA was 69.0. When the HND training strategy was further introduced, the performance of the HND-MOTR model was significantly improved, with the HOTA score jumping to 71.8 and the IDF1 score reaching 74.5.

[0112] The HND-MOTR model achieved a 2.8 percentage point improvement in HOTA and a 3.4 percentage point improvement in IDF1 compared to Hybrid-MOTR (without HND). This performance strongly demonstrates that the HND paradigm is the core driving force behind the performance enhancement of this invention, as it significantly enhances the model's discriminability in the feature space by introducing explicit discriminative supervision.

[0113] like Figure 6 As shown, Figure 6Curves of HOTA, MOTA, and IDF1 as a function of k were plotted. As k increases from 1 to 5, all core metrics show a steady improvement. This validates the design philosophy of the proposed method: random sampling from a larger spatially adjacent candidate pool provides the model with more diverse and challenging hard negative examples, effectively promoting discriminative learning. Experimental results show that all three key metrics reach peak performance at k=5. Performance begins to decline as k increases further. It is inferred that excessively large k values ​​introduce spurious negative examples with weak competition to the current target, interfering with the model's identification of the most critical neighboring interference items, thus impairing learning performance. Therefore, k=5 was determined to be the optimal balance setting on the DanceTrack dataset.

[0114] In summary, these ablation experiments collectively demonstrate that there is a high degree of complementarity between the architecture enhancement and discriminative training of this invention: the former provides stable, high-quality observation features, while the latter enhances the inherent distinguishability of the features. Together, they construct a robust and high-performance multi-target tracking system.

[0115] IV. Specific Implementation Process:

[0116] To validate the effectiveness of the Hard Negative Example Denoising (HND) paradigm at the feature level, the t-SNE method was used to perform dimensionality reduction visualization of the track embeddings learned by the model. For example... Figure 7 As shown, the trajectory embeddings of the first 100 frames of two challenging sequences, dancetrack0010 and dancetrack0058, are visualized. Figure 7 In this model, embeddings corresponding to different tracking identities (IDs) are marked with different colors and shapes. Ideally, embeddings of the same identity should form tight clusters, while clusters of different identities should remain clearly separated. Figure 7 Figure (a) and Figure 7 Figure (c) shows the embedding results of the baseline architecture (Hybrid-MOTR) trained without HND. It can be observed that embedding points of different identities exhibit significant overlap and mixing in the feature space, with blurred boundaries between clusters. This directly indicates insufficient discriminative power and identity confusion at the feature level. In stark contrast, Figure 7 Figure (b) in the middle and ​Figure (d) in the figure illustrates the embedding results of the final model HND-MOTR trained using the HND paradigm proposed in this invention. By systematically constructing challenging hard negative examples and introducing explicit discriminative supervision, the model is forced to learn feature representations with higher discriminative power. The results show that embedding points of the same identity form more compact and cohesive clusters, while clusters of different identities maintain clear and good separation. This transformation from a chaotic to an ordered feature space strongly demonstrates that the HND paradigm successfully enhances the model's feature discriminative ability, thereby fundamentally solving the identity preservation problem in crowded scenes.

[0117] In summary, the proposed method of this invention constructs a novel HND-MOTR model, which is an ensemble model for multi-target tracking in complex and crowded scenes. This model integrates a discriminative training paradigm with an enhanced temporal architecture. The core of this model lies in the introduction of a Hard Negative Example Denoising (HND) training paradigm, which for the first time reconstructs the denoising task into a discriminative feature learning engine for MOT identity preservation. Specifically, HND utilizes a K-NN-based random sampling method to systematically construct the most challenging and highly similar hard negative examples from spatially neighboring targets. It also breaks spatial alignment through a multi-level noise injection mechanism, thereby forcing the model to learn fine-grained, highly discriminative feature representations, directly addressing the problem of insufficient feature discriminativity in crowded scenes. At the architectural level, the model adopts a proposal-anchored temporal interaction architecture. By integrating proposal queries from external detectors during the unified query generation phase, it ensures high-precision initialization of new targets. Simultaneously, it innovatively introduces a proposal-anchored update mechanism, utilizing real-time high-confidence proposals to calibrate and anchor the state and long-term memory (TIM) of trajectory queries, enhancing the robustness of trajectory recovery and identity re-identification after long-term occlusion. Finally, the entire system uses a Transformer decoder to achieve joint processing and end-to-end optimization of four different types of queries (i.e., proposal queries, learnable queries, tracking queries, and HND queries), thereby achieving synergistic effects between architectural and training advantages, working together to achieve high robustness and high identity consistency tracking of targets in complex and crowded scenarios.

[0118] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0119] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A multi-target tracking method for complex scenarios, characterized in that, Includes the following steps: S1. Obtain the target image frame; S2. Construct the HND-MOTR model; S3. Input the target image frame into the HND-MOTR model, introduce the hard negative example denoising training paradigm for training, and generate the trained HND-MOTR model, specifically: The current image frame is input into the backbone network and Transformer encoder respectively for feature extraction, generating multi-scale image feature maps. ; Input the current image frame into the HND query generation module to generate the HND query for the current image frame. Specifically: During training, several denoising groups are constructed, and each denoising group includes the same number of positive queries and negative queries. Positive queries are generated by applying a small perturbation to the coordinates of the true label boxes. A set proportion of negative queries are used as hard negative queries, and the remaining proportion of negative queries are used as perturbed false positive queries. Then, the K-NN random sampling method is used to generate hard negative example queries. Specifically, for the ground truth label box that generates the hard negative example query, the Euclidean distance between the center coordinates of the ground truth label box and the remaining ground truth label boxes in the same image frame is calculated to generate the hard negative example query. Simultaneously, standard noise is injected into positive queries, maximum noise is injected into disturbed false positive queries, and minimum noise is injected into hard negative queries; finally, the HND query of the current image frame is obtained. Input the current image frame into the proposal query generation module to generate a proposal query for the current image frame. ; Input the previous image frame into the timing interaction module to generate a tracking query for the previous image frame. ; The learnable detection query generation module generates learnable detection queries by randomly initializing them based on the number of detection queries and feature dimensions. ; HND query Suggested inquiry Tracking and querying Learning, testing, and query capabilities are available. Concatenate the results to generate a union query set. ; Union query set Multi-scale image feature maps The input is processed by the Transformer decoder, which interacts with the multi-layer self-attention mechanism and cross-self-attention mechanism to output the decoding result containing bounding boxes and identity embeddings. ; Decoding results containing bounding boxes and identity embeddings Input the timing interaction module to update the status and generate the tracking query for the next image frame, so as to update the tracking query of the current image frame; Finally, the decoding result, including the bounding box and identity embedding, will be... Input the prediction head to make predictions, generate the target ID, bounding box position, size and confidence of the current image frame, complete the training of the HND-MOTR model, and obtain the trained HND-MOTR model; S4. Acquire the image to be detected and input it into the trained HND-MOTR model to achieve multi-target tracking and recognition.

2. The multi-target tracking method for complex scenarios according to claim 1, characterized in that, The HND-MOTR model includes a feature extraction module, a Transformer decoder, a temporal interaction module, and a prediction head; The feature extraction module includes a backbone network and a Transformer encoder.

3. The multi-target tracking method for complex scenarios according to claim 2, characterized in that, The HND-MOTR model also includes various query generation modules, including a proposed query generation module and a learnable detection query generation module.

4. The multi-target tracking method for complex scenes according to claim 3, characterized in that, When training using a hard negative example denoising training paradigm, the HND-MOTR model's various query generation modules also include an HND query generation module.

5. The multi-target tracking method for complex scenarios according to claim 1, characterized in that, Decoding results containing bounding boxes and identity embeddings The input timing interaction module updates the state and generates the tracking query for the next image frame. The update process for the tracking query of the current image frame is as follows: Decoding results containing bounding boxes and identity embeddings Tracking query with the previous image frame The images are stitched together to obtain the short-term memory of the current image frame, while the trajectory state of the current image frame is used as the long-term memory. Use the short-term memory of the current image frame as the query vector, and decode the result. As a value vector, the trajectory state of the current image frame is used as the key vector; The query vector, value vector, and key vector are input into the attention layer. The relevance weights between the query vector, value vector, and key vector are calculated and then fused to generate fused features. After summing the fused features with the long-term memory of the current image frame pixel by pixel, the result is input into a fully connected layer for feature integration, generating a tracking query for the current image frame. The long-term memory of the current image frame is smoothed by using the exponential moving average method to generate the long-term memory of the next image frame, which is the tracking query of the next image frame, and realizes the update of the tracking query of the current image frame.

6. The multi-target tracking method for complex scenes according to claim 5, characterized in that, The exponential moving average method is used to smooth the parameters of the long-term memory of the current image frame, generating the long-term memory of the next image frame. Therefore, the formula for tracking and querying the next image frame is: in, Represents long-term memory of the next image frame. Represents the smoothing coefficient. Represents the long-term memory of the current image frame. This represents the decoding result of the current image frame, including bounding boxes and identity embeddings, output by the Transformer decoder.

7. The multi-target tracking method for complex scenes according to claim 1, characterized in that, The formula for calculating the Euclidean distance between the center coordinates of the ground truth label box and the remaining ground truth label boxes in the same image frame is: in, Represents the actual label box To the actual label box The center coordinates of the Euclidean distance , These represent the actual label boxes. center coordinates x-coordinate value, y-coordinate value , These represent the actual label boxes. center coordinates x-coordinate value, y-coordinate value Indicates condition, This indicates the total number of actual label boxes.

Images