A multi-target tracking method

Abstract

The application belongs to the technical field of computer vision, and particularly relates to a multi-target tracking method; on the basis of a classic Kalman filter + Hungarian algorithm, a unique PoDe-SORT framework is proposed, element-wise fusion of a width-height ratio representing the posture of a target with a standard IoU, construction of a deep iterative layered correlation process from fine granularity to coarse granularity based on pseudo-depth, and the like, so that a lightweight PoDe-SORT based on 2D sensor data operation with smaller calculation amount is realized while the robustness is improved. Overall, the application can realize the effect of target tracking based on 3D sensor information with small calculation amount, and has high robustness, strong operation rationality, small deformation error and high scene applicability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to a multi-target tracking method. BACKGROUND

[0002] Multi-Object Tracking (MOT) aims to localize multiple targets of interest and maintain their identity consistency in video sequences. This technology plays an important role in the field of computer vision and has become the basis for key applications such as video surveillance, autonomous driving, and human-computer interaction. Although related research has made significant progress, the robustness of tracking in complex scenes such as irregular motion and crowdedness is still a major bottleneck that needs to be broken through. Currently, Tracking by Detection (TBD) is the most mainstream paradigm in the MOT field, which divides the problem into two tasks: target detection and data association. The association stage usually relies on the appearance and motion cues of the target to calculate similarity. However, in complex scenes such as irregular motion and crowdedness, these cues often become unreliable, such as Figure 1 The upper part shows. In order to achieve robust association, existing methods either introduce additional cues such as velocity direction and confidence score, or set association priorities. However, they still ignore the deformation effect in the motion process. We found that this neglected effect is the key reason for the unreliability of cues in complex scenes. In addition, some work attempts to estimate pseudo-depth information from 2D observations to compensate for the lack of cues. However, they fail to consider the impact of bounding box deformation. This leads to noise in pseudo-depth, making it difficult to accurately reflect the true spatial position of the target. As Figure 1 shown, although Intersection over Union (IoU) can effectively capture the translational motion of the target, it is difficult to deal with the geometric changes caused by deformation. Both non-uniform scaling and uniform scaling types of deformation will significantly change the overlap of the bounding box. This effect is coupled with the real target displacement effect, resulting in a large deviation in the IoU similarity score. At the same time, affected by deformation, target features are more susceptible to interference from other targets. In crowded scenes, these deviations will be further amplified.

[0003] With the improvement of detector accuracy, the research focus of the tracking by detection (TBD) paradigm shifts to enhancing the robustness of the association stage. At the same time, three-dimensional scenes provide richer location information than two-dimensional images, which helps to improve the performance of two-dimensional MOT. In two-dimensional MOT, most TBD methods follow the motion consistency assumption. Early simple online and real-time tracking (SORT) is based on Kalman filter (KF) to build a linear motion model, combined with the Hungarian algorithm to achieve efficient association, but it is sensitive to non-linear motion and occlusion. Therefore, the prior art provides various target tracking methods and tracking models to solve this problem, but each has the problems of large computational overhead, task conflict, difficulty in dealing with crowded scenes, continuous amplification of bounding box matching error in complex scenes, deformation error, etc., and cannot balance robustness, task rationality, deformation error and scene applicability. Overall, it is difficult to achieve a reasonable method based on two-dimensional MOT to match the effect of three-dimensional target tracking. SUMMARY

[0004] Therefore, the present application provides a target tracking method based on the classic Kalman filter + Hungarian algorithm, which adopts a unique PoDe-SORT framework, uses aspect ratio to represent the posture of the target, and then elementarily fuses with standard IoU, constructs a depth iterative hierarchical association process from fine to coarse based on pseudo-depth, etc. While improving robustness, a lightweight PoDe-SORT based on 2D sensor data operation with smaller computational load is achieved. Overall, the present application can achieve the effect of target tracking based on 3D sensor information with very small computational load, while being highly robust, computationally reasonable, having small deformation error and having high scene applicability.

[0005] The specific technical solutions adopted by the present application are as follows:

[0006] A multi-target tracking method for locating multiple targets simultaneously in each image frame of a video sequence and maintaining the identity consistency of each target; the multi-target tracking method comprises:

[0007] motion estimation of each target trajectory in the current frame of image based on Kalman filter, and association operation of detected targets and predicted trajectories in the next frame based on Hungarian algorithm;

[0008] When performing the association operation:

[0009] suppressing the association interference caused by target deformation based on the posture perception intersection-over-union method and the depth perception cascade matching method;

[0010] wherein:

[0011] The posture-aware intersection-over-union method is based on the aspect ratio to represent the posture of the target, and then is fused with the standard IoU element by element to improve the adaptability to posture distortion and reduce the correlation interference caused by non-equal ratio deformation. The multi-target tracking method is based on diagonal overlap calculation and adaptive weighting mechanism to dynamically strengthen the discriminative effect of height or width under different postures of the target.

[0012] The deep perception cascade matching method is based on pseudo-depth to construct a depth hierarchy, adopts a cascade matching process from fine granularity to coarse granularity, gradually relaxes the depth constraint on the target, limits the correlation search within a similar depth interval, and reduces the correlation interference caused by equal ratio scaling.

[0013] Further, the diagonal overlap calculation method specifically includes:

[0014] The posture-aware diagonal intersection-over-union is calculated, i.e., the overlap ratio of the frame along the diagonal direction of the detection frame and the track frame is calculated to represent the geometric similarity:

[0015]

[0016] In the formula, DiIoU is the posture-aware diagonal intersection-over-union, and respectively represent the width and height of the intersection region of the detection frame and the track frame, and respectively represent the width and height of the union region of the detection frame and the track frame.

[0017] Further, the adaptive weighting mechanism dynamically strengthens the discriminative effect of width and height under different postures by adjusting the weight of width and height in DiIoU, specifically:

[0018] The optimized posture-aware diagonal intersection-over-union is defined as Therefore:

[0019]

[0020]

[0021] In the formula, r is the aspect ratio; the threshold value , and are used to divide four posture intervals, wherein represents the standard posture, represents the wide posture, represents the extremely wide posture, and represents the extreme posture.

[0022] Furthermore, when performing the association operation, if the pose-aware intersection-union ratio is defined based on element-wise multiplication, then:

[0023]

[0024]

[0025] In the formula, and These represent the bounding box areas of the two detected targets; Indicates the pose-aware intersection-union ratio; This indicates the standard intersection-union ratio.

[0026] Furthermore, the depth-sensing cascaded matching method includes the following steps:

[0027] S101: Calculate the pseudo-depth of all detected targets in the image frame to determine the detection range. The detection range is evenly divided into L sub-intervals. The detection targets are assigned to corresponding sub-intervals according to their pseudo-depth, forming detection subsets. ;

[0028] Calculate the pseudo-depth of all trajectories in the image frame to determine the trajectory range interval, and uniformly divide the trajectory range interval into L sub-intervals; assign the trajectories to the corresponding sub-intervals according to their pseudo-depth to form the obtained trajectory subset. ;

[0029] S102: For each subinterval... and The association operation is performed in between; the detected targets and trajectories that fail to match are saved respectively.

[0030] Furthermore, the cascaded matching process from fine-grained to coarse-grained is as follows: the image frame is divided into fine-grained segments, targets with similar depth variations are aggregated, and then the depth constraints are gradually relaxed to facilitate the re-association of mis-layered targets.

[0031] Furthermore, the depth-aware cascaded matching method also includes the following steps:

[0032] S103: Take the unmatched set as input, reduce the number of layers L (coarser granularity), and re-execute S101-S102 within each set range;

[0033] S104: Repeat S103 until the termination condition of the associated operation is met.

[0034] Furthermore, the specific method for gradually relaxing the depth constraint is as follows: the number of layers in the layering is continuously reduced in the iterative association, and the matching within the layer adopts a preset association threshold, which helps to prevent matching failure caused by layering errors.

[0035] Furthermore, the depth-aware cascaded matching method also adaptively corrects mis-layering by expanding the search domain when there is noise in the pseudo-depth, thereby alleviating matching failures caused by mis-layering.

[0036] Furthermore, when performing the association operation, the following rules are followed: first, the association operation is performed on high-confidence targets; second, low-confidence targets are processed; and finally, the lost trajectory is recovered using historical detection results.

[0037] The present invention can bring the following beneficial effects:

[0038] The target tracking method of this invention achieves excellent performance on three benchmarks: MOT17 (Multi-target tracking benchmark [J]. arXiv preprint arXiv:1603.00831, 2016), MOT20 (Multi-target tracking benchmark in crowded scenes [J]. arXiv preprint arXiv:2003.09003, 2020), and DanceTrack (Multi-target tracking in uniform appearance and diverse motion scenes [C] / / IEEE / CVF Conference Proceedings on Computer Vision and Pattern Recognition. 2022: 20993-21002). It has a wide range of applications, can bring stable and significant performance improvement to various trackers, and has extremely low computational overhead. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram illustrating the operation of a model for improving the robustness of multi-target tracking in the background art.

[0041] Figure 2 This is a schematic diagram of the operation of the model (PoDe-SORT) used in a multi-target tracking method according to a specific embodiment of the present invention;

[0042] Figure 3 This is the pseudocode of the associated pipeline of the model used in a multi-target tracking method according to a specific embodiment of the present invention;

[0043] Figure 4 This diagram illustrates the comparison of the correlation mechanism between the standard Intersection over Union (IoU) and the Pose-Aware Intersection over Union (PAIoU) method in pose-changing scenarios.

[0044] Figure 5This is pseudocode for the depth-aware cascaded matching method in a specific embodiment of the present invention. Detailed Implementation

[0045] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0046] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0047] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0048] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The illustrations only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0049] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0050] In existing technologies, to address the issues of nonlinear motion and occlusion sensitivity, deep online tracking (Simple online and realtime tracking with a dEEp associationmetric, DEEpSORT) and deep person tracking (Real-time multiple people tracking with dEEply learned candidate selection and person re-identification, MOTDT) introduce independent re-identification (ReID) models to improve object discrimination capabilities, but they suffer from high computational costs. FairMOT (On the fairness of detection and re-identification in multiple object tracking) and Quasi-densesimilarity learning for multiple object tracking (QDTrack) reduce computational costs by jointly training detection and ReID, but they suffer from task conflicts. Although Relation-aware multiple object tracking with decoupled representation (RelationTrack) and Decoupling features to solve the contradictory dilemma of feature requirements (FDTrack) alleviate this conflict by using feature decoupling, their performance is still limited by this limitation. Some studies have reduced the reliance on appearance features by optimizing motion modeling. Center-point-based tracking (Tracking objects as points, CenterTrack) constructs a correlation metric by regressing the target displacement, but it struggles to handle crowded scenarios.Robust associations multi-pedestrian tracking (BoT-SORT) and occlusion-robust multi-object tracking using pseudo-depth cues (PD-SORT) expand the KF state space and introduce camera motion compensation (CMC) to improve motion prediction accuracy, but increase computational burden. Inspired by this, the end-to-end paradigm achieves trajectory time propagation through attention mechanisms, performing well in complex scenes, but the computational overhead of self-attention and feedforward networks is still not negligible.

[0051] Recent research emphasizes improving adaptability in complex scenarios while maintaining efficiency. ByteTrack (Multi-object tracking by associating every detection box) proposes a cascaded association strategy, using low-confidence detections and unmatched trajectories for secondary matching, effectively mitigating mismatches caused by occlusion. Observation-centric sort: Rethinking sort for robust multi-object tracking (OC-SORT) employs an observation-centric approach, improving association accuracy through the speed consistency between historical observations and new detections, and correcting the KF state by fitting a virtual linear trajectory after trajectory interruption, but it does not explicitly model observation bias caused by deformation. Weak cues matter for online multi-object tracking (Hybrid-SORT) introduces weak cues to compensate for strong cues, using confidence, height, and velocity direction to improve tracking performance in complex scenarios. These methods achieve a good balance between efficiency and performance by introducing low-cost additional cues to replace complex feature extraction. However, these methods still mainly rely on shallow fusion of two-dimensional observation features and fail to explicitly model the geometric deformation effects of bounding boxes caused by target motion. Existing methods, which employ the assumption of uniform motion and static appearance modeling, struggle to capture the effects of geometric deformation caused by changes in target pose or depth displacement. This leads to a continuous amplification of bounding box matching errors in complex scenes, becoming a key obstacle to improving the robustness of 2D MOT.

[0052] 3D information in MOT:

[0053] In Motion Targeting (MOT) research, 3D scenes provide richer location information than 2D images, which helps in dealing with complex scenarios such as congestion. In recent years, several works have utilized 3D sensor data, such as point clouds, to build high-precision 3D MOT systems. However, these methods rely on depth sensors and have high computational costs, making them difficult to deploy in resource-constrained scenarios.

[0054] To reduce system costs, some studies have attempted to improve association performance by directly estimating the 3D information of targets from 2D images. Sparse object tracking algorithms (Multi-object tracking by performing scene decomposition based on pseudo-depth, SparseTrack) construct pseudo-depth by measuring the distance from the bottom of the bounding box to the bottom edge of the image, thereby layering the image to alleviate occlusion problems. Occlusion-Robust Multi-Object Tracking Using Pseudo-Depth Cues (PD-SORT) introduces pseudo-depth into motion modeling, extending the IoU to 3D space and enhancing association discriminative power by utilizing the depth difference between detection and trajectory.

[0055] While the aforementioned methods improve matching robustness without relying on true 3D information, their performance is still affected by target deformation. First, due to the non-linear relationship between proportional scaling and depth, the pseudo-depth calculated from the bounding box position varies at different depths, making the target prone to being assigned to the wrong layer. Second, existing motion models based on linear assumptions fail to consider the impact of deformation when predicting the target's current position, leading to errors in the pseudo-depth between the predicted and detected boxes. Treating these errors as true depth variations during the association process weakens the matching robustness.

[0056] Therefore, in one embodiment of the present invention, a multi-target tracking method is provided for simultaneously locating multiple targets in each image frame of a video sequence while maintaining the consistency of the identities of each target; the multi-target tracking method includes:

[0057] Motion estimation is performed on the trajectory of each target in the current frame image based on the Kalman filter. In the next frame, the detected targets and the predicted trajectories are associated based on the Hungarian algorithm.

[0058] When performing the associated operation:

[0059] Suppress correlation interference caused by target deformation based on the attitude-aware cross-union ratio (PAIoU) and depth-aware cascaded matching (DACM) method;

[0060] in:

[0061] The posture-aware intersection-union method is based on the aspect ratio to characterize the target's posture and then fused element-wise with the standard IoU to improve adaptability to posture distortion and reduce correlation interference caused by non-uniform deformation. The method is based on diagonal overlap calculation and adaptive weighting mechanism to dynamically enhance the discrimination role of height or width under different target postures.

[0062] The depth-aware cascaded matching method constructs a depth hierarchy based on pseudo-depth and adopts a cascaded matching process from fine-grained to coarse-grained. It gradually relaxes the depth constraints on the target, restricts the association search to a similar depth range, and reduces association interference caused by proportional scaling.

[0063] In some embodiments, the multi-target tracking method of the present invention is implemented based on a lightweight online tracker (PoDe-SORT), the overall framework of which is as follows: Figure 2 As shown in the figure, this method is based on the classic SORT paradigm, uses a Kalman filter for motion estimation, and employs the Hungarian algorithm to correlate the detected target with its trajectory. To effectively suppress correlation interference caused by target deformation, we introduce two lightweight modeling mechanisms: PAIoU and DAM.

[0064] This embodiment proposes PAIoU, a pose-aware geometric similarity measure. Its core lies in using aspect ratio to represent pose and integrating diagonal overlap calculation with an adaptive weighting mechanism to dynamically enhance the discriminative role of width or height under different poses. Finally, it is fused element-wise with the standard IoU to improve adaptability to pose distortion and reduce association interference caused by non-uniform deformation. DAM proposes a robust hierarchical association strategy. This strategy constructs depth hierarchies based on pseudo-depth and employs a cascaded matching process from fine-grained to coarse-grained, gradually relaxing depth constraints to effectively reduce association interference caused by proportional scaling. Furthermore, this mechanism can adaptively correct erroneous hierarchies by expanding the search domain when there is noise in the pseudo-depth, effectively mitigating matching failures caused by erroneous hierarchies. To achieve the core idea of ​​explicitly modeling bounding box deformation, this embodiment integrates PAIoU and DAM into the data association process.

[0065] This embodiment of PoDe-SORT constructs the similarity between the detected trajectory and the target position, direction of motion, and pseudo-depth. The consistency of velocity is measured using OCM. Bounding box deformation modeling is introduced into the similarity calculation: PAIoU modulates the standard IoU based on pose changes, forming pose-aware geometric similarity; DAM is based on pseudo-depth layering, restricting the association search to similar depth ranges. The two work together to effectively suppress deformation-induced association interference.

[0066] This embodiment of PoDe-SORT extends the association process of OC-SORT, integrating PAIoU and DAM. The association of each frame is divided into three stages: first, associating high-confidence targets ( Figure 3 (Pseudocode lines 3-4), followed by handling low-confidence targets (using the ByteTrack strategy). Figure 3 (Pseudocode lines 5-6) Finally, historical observations are used to recover the lost trajectory ( Figure 3 Lines 7-9 of the pseudocode (OC-SORT) represent the Observation-Centric Recovery (OCR) module. After association is complete, new trajectories are initialized and long-term inactive trajectories are removed. Figure 3 (Pseudocode lines 10-11). The pseudocode for the associated pipeline of PoDe-SORT in this embodiment is as follows: Figure 3 As shown.

[0067] In real-world scenarios, changes in the pose of a target cause anisotropic deformation of the bounding box, manifested as asymmetric changes in width and height. However, traditional IoU similarity relies solely on the overlapping area of ​​the bounding boxes, making it difficult to effectively capture the geometric deformation caused by pose changes. This is an inherent limitation of 2DMOT.

[0068] To effectively model non-uniform deformation, this embodiment employs diagonal IoU (DiIoU). DiIoU enhances the discriminative power of overall geometric changes in the bounding box, more accurately characterizing anisotropic deformation caused by pose changes, and overcoming the limitation of IoU relying on area overlap. DiIoU characterizes geometric similarity by calculating the overlap ratio of two boxes along the diagonal direction:

[0069]

[0070] in and These represent the width and height of the area where the two frames intersect. and These represent the width and height of the joined area of ​​the two frames, respectively.

[0071] To further decouple attitude change characteristics, this embodiment uses an aspect ratio of As a direct visual representation of posture, changes in aspect ratio effectively reflect posture changes and are independent of the proportional scaling effect caused by changes in target depth. Based on this, this embodiment provides a dynamic weighting strategy, which dynamically strengthens the discriminative role of width and height under different postures by adjusting the weights of width and height in DiIoU. The posture-aware diagonal intersection-union ratio is defined. :

[0072]

[0073]

[0074] Among them, threshold , and Divide the attitude range into four categories: It is a standard posture, close to the proportion of a typical pedestrian. It is a wide-body posture, usually caused by small changes in posture. It is an extremely wide posture, usually caused by drastic changes in posture. This indicates an extreme posture, which may be caused by severe occlusion, target splicing, or rare posture.

[0075] Ultimately, we utilized The attitude change information modulation standard IoU is used to significantly improve the discriminative power of target association in geometric matching and its adaptability to attitude changes, such as... Figure 4 As shown. PAIoU is defined by element-wise multiplication:

[0076]

[0077]

[0078] in, and These represent the areas of the two bounding boxes, respectively.

[0079] Figure 4 This section compares the association mechanisms of IoU and PAIoU in pose-changing scenarios. The calculation formulas for IoU and PAIoU are given above; the comparison below shows their association effects under pose changes (in the figure, I represents IoU similarity, D represents pose similarity, and S1 / S2 represents two candidate targets). When the pose changes (represented by the aspect ratio r), PAIoU dynamically strengthens the more discriminative dimension (width or height) based on the pose, thus more accurately completing target association.

[0080] In real-world scenarios, the depth displacement of a target triggers proportional scaling of the bounding box, which can severely interfere with association in crowded scenes. Existing methods typically derive pseudo-depth based on perspective geometry and employ fixed-layer hierarchical matching to reduce interference, but they ignore the impact of bounding box deformation. This is mainly reflected in two aspects: First, proportional scaling has a non-linear relationship with depth, causing the pseudo-depth calculated based on the bounding box position to vary at different depths, thus rendering the fixed-layer strategy based on pseudo-depth ineffective. Second, constrained by the assumption of uniform motion, the pseudo-depth estimated by the motion model cannot accurately reflect the true spatial position of the target under deformation. These two factors together lead to noise in the pseudo-depth, making targets prone to mis-layering and causing matching failures. To address this, this embodiment proposes DAM, which aims to model the impact of pseudo-depth changes on the geometric properties of the bounding box. It adopts an iterative hierarchical association process from fine to coarse: first, fine-grained partitioning is performed to aggregate targets with similar depth changes; then, the depth constraints are gradually relaxed to provide re-association opportunities for mis-layered targets, thereby significantly reducing association interference caused by proportional scaling.

[0081] For each frame of the image, first calculate the pseudo-depth of all detections to determine the detection range. And evenly divided into L sub-intervals. The detected targets are assigned to corresponding sub-intervals based on pseudo-depth, forming detection subsets. The same process is applied to the trajectory to obtain a subset of the trajectory. Subsequently, within each sub-interval and Inter-execution data association ( Figure 5 (Pseudocode lines 8-15); unmatched detections and trajectories are stored separately. and After each round, the unmatched set is used as the input for the next round. The number of layers L is reduced (for coarser granularity), and layers are re-layered and associated within their respective ranges until the termination condition is met. Figure 5 (See pseudocode lines 5-18). To prevent matching failures due to layering errors, a stricter IoU threshold is used within each layer. See the complete process below. Figure 5 .

[0082] In some embodiments, the PAIoU and DAM provided by the multi-target tracking method described above can be applied to various trackers, including SORT, DEEpSORT, MOTDT, and ByteTrack, which cover a variety of mainstream data association strategies. For example, SORT relies on motion information to achieve efficient tracking; DEEpSORT and MOTDT fuse motion and appearance information to enhance robustness; and ByteTrack utilizes low-confidence bounding boxes to assist motion information. The PAIoU and DAM of this invention have excellent adaptability. At the same time, these two deformation mechanisms can bring stable and significant performance improvements to various trackers with extremely low computational overhead. This embodiment is the first systematic solution proposed from the perspective of bounding box deformation modeling for robust tracking problems in complex scenes.

[0083] This embodiment constructs a robust geometric deformation mechanism by modeling the differentiated effects of pose changes and depth displacement on bounding box deformation, fundamentally reducing the interference of deformation on association. This targeted geometric modeling mechanism does not depend on a specific tracking framework or association strategy, and therefore has excellent universality.

[0084] The following describes the experimental results of applying the multi-target tracking method of this embodiment to various trackers.

[0085] Benchmark experiment:

[0086] This embodiment was evaluated on the MOT17, MOT20, and DanceTrack test sets, and compared with several state-of-the-art (SOTA) trackers. In Tables 1-3, all methods using the same YOLOX (You Only Look Once) detector are italicized, and the best results for each metric are shown in bold. All evaluation results are based on proprietary detector schemes. On all datasets, PoDe-SORT outperforms the baseline OC-SORT, especially on the DanceTrack dataset, which features nonlinear motion and highly dense interactions, where the improvement is particularly significant. Meanwhile, the introduced additional computational overhead is almost negligible (as shown in Table 4).

[0087] Compared to other state-of-the-art trackers, this embodiment of PoDe-SORT achieves near-optimal performance, even slightly outperforming GeneralTrack, which employs a more complex model, on MOT17. It should be noted that the MOT17 and MOT20 datasets primarily consist of simple linear motions with relatively small target pose changes and depth displacement amplitudes. Because these datasets contain a large amount of linear motion, linear interpolation is widely used in post-processing to improve correlation smoothness. To further highlight the effectiveness of PoDe-SORT itself, we additionally provide comparison results for PoDe-SORT and the baseline OC-SORT without linear interpolation (marked *) on MOT17 and MOT20.

[0088] The comparison results of each model are explained below.

[0089] DanceTrack: As shown in Table 1, PoDe-SORT achieved state-of-the-art results of 59.5%, 43.6%, and 60.5% in HOTA, AssA, and IDF1, respectively, representing improvements of 4.9%, 3.4%, and 5.9% compared to the baseline method OC-SORT. Specifically, in complex scenes like DanceTrack, characterized by irregular motion and dense crowds, bounding box deformation is a key factor affecting correlation performance. Our method effectively addresses this challenge by explicitly modeling bounding box deformation during motion. This advantage is fully reflected in core metrics such as AssA and IDF1, demonstrating that our method significantly improves the robustness of the tracker in complex scenes.

[0090] Table 1 Results of the DanceTrack test set

[0091]

[0092] In Table 1:

[0093] Attention - an attention-based method; Embedding - an appearance-based embedding method; Motion - a motion model-based method; HOTA - Higher-Order Tracking Accuracy; AssA - Association Accuracy; IDF1 - ID-F1 Score; DetA - Detection Accuracy; MOTA - Multiple Object Tracking Accuracy.

[0094] The names of the models in the table are proprietary terms in this field and are generally left untranslated; the following are some exemplary translations made to facilitate reading by those not skilled in the art:

[0095] QDTrack - Quasi-dense tracking (dense sampling); BoT-SORT - Improved SORT (integrates appearance information and camera motion compensation); StrongSORT - Strong SORT (an enhanced version of DEEpSORT); DiffusionTrack - Diffusion tracking; GeneralTrack - General tracking; CenterTrack - Center point tracking; TraDes - Joint detection and tracking; SparseTrack - Sparse tracking; MambaMOT - Mamba multi-target tracking (based on the Mamba architecture).

[0096] MOT17: As shown in Table 2, PoDe-SORT achieved excellent performance scores of 63.7, 63.5, and 78.3 on HOTA, AssA, and IDF1, respectively, with overall performance second only to GeneralTrack. To eliminate post-processing interference, we further compared the performance under the condition of no linear interpolation (marked with *), and found that PoDe-SORT improved upon OC-SORT by 1.3, 1.1, and 1.8 on HOTA, AssA, and IDF1, respectively; even after adding linear interpolation, PoDe-SORT still maintained a performance advantage of 0.5, 0.1, and 0.8. This proves that in scenarios dominated by linear motion, this method can still stably improve tracking robustness.

[0097] Table 2 Results of the MOT17 test set

[0098]

[0099] MOT20: As shown in Table 3, MOT20 has a higher target density and occlusion level compared to MOT17, significantly increasing the tracking difficulty. Despite this, PoDe-SORT achieved excellent performance scores of 62.6, 62.5, and 76.9 on HOTA, AssA, and IDF1, respectively, with overall performance significantly outperforming many state-of-the-art (SOTA) methods. Under the condition of no linear interpolation (*-marked), PoDe-SORT improved upon OC-SORT by 1.5, 1.2, and 2.1 on the three metrics, respectively, further validating that the proposed method can effectively reduce the interference of deformation on correlation and exhibits good robustness and generalization ability in dense target scenes.

[0100] Table 3 Results of the MOT20 test set

[0101]

[0102] To verify the effectiveness of each component of PoDe-SORT in complex scenarios, ablation experiments were conducted on the DanceTrack validation set (see Table 4). The results show that after constructing PAIoU, the association performance was significantly improved (AssA: +5.0, IDF1: +3.9), and the overall tracking performance also improved (HOTA: +3.6). Introducing DAM to restrict the association search to similar depth ranges further enhanced the association robustness (AssA: +2.3, IDF1: +3.4), and the overall performance continued to improve (HOTA: +1.8). When jointly modeling the effects of pose changes and depth displacement on bounding box deformation, the association performance achieved a breakthrough improvement (AssA: +6.8, IDF1: +7.0), with a final HOTA improvement of 5.1. Experimental results indicate that PAIoU and DAM jointly construct a robust bounding box deformation mechanism, effectively reducing the interference of deformation on association. The 2.1 MOTA improvement is mainly due to the effective utilization of low-confidence detection boxes, further leveraging the advantages of the bounding box deformation mechanism. Under the same hardware environment, PAIoU and DAM introduce only slight delays of -0.3 FPS and -1.0 FPS respectively, resulting in an overall system reduction of only 1.5 FPS, while maintaining the same high real-time efficiency as the baseline method.

[0103] Table 4 shows the results of the component ablation experiment.

[0104]

[0105] In Table 4: Method; Baseline

[0106] As can be seen, the embodiments of the present invention systematically ablate the internal components of PAIoU, gradually verifying the effectiveness of pose modeling (see Table 5). First, by fusing DiIoU with the standard IoU (second row in Table 5), the HOTA, AssA, and IDF1 metrics improved by 1.6, 2.1, and 1.4, respectively. This gain mainly stems from DiIoU's characterization of geometric similarity by calculating the overlap ratio of two bounding boxes along the diagonal direction, thus enhancing its sensitivity to changes in target pose. Second, the introduction of a dynamic weighting mechanism (third row) dynamically strengthens the discriminative role of width and height under different poses, enabling the metrics to more flexibly reflect the impact of pose changes on bounding box deformation. This results in significant improvements of 1.5, 2.0, and 1.4 in HOTA, AssA, and IDF1, respectively, indicating that PAIoU constructs a geometric metric that is sensitive to pose changes and possesses adaptive capabilities.

[0107] Furthermore, thanks to OC-SORT's mechanism of correcting KF states through virtual trajectories, directly predicting the bounding box width (rather than relying on indirect representations of area and aspect ratio, see line 4) further improves performance (HOTA +0.5, AssA +0.9, IDF1 +1.1). This is because width has more complex motion patterns than height. OC-SORT's superior KF state correction mechanism allows direct width prediction to more accurately describe the target's pose changes.

[0108] Table 5 Ablation test results of PAIoU components

[0109]

[0110] This embodiment further conducts ablation experiments on the dynamic weighting mechanism in PAIoU to verify the rationality of the weight selection. Specifically, PAIoU automatically classifies targets into four poses—standard, wide-body, extra-wide, and extreme—based on the aspect ratio of the target bounding box, and selects the most representative weight α within each pose range for testing (non-global parameter traversal), focusing on the settings adapted to the pose change characteristics. For example, in the standard pose, the target height is dominant, so a higher α is assigned (e.g., α=1.9); while in the extra-wide or extreme poses, the target width increases significantly, and the height decreases relatively, so α gradually decreases accordingly (e.g., α=0.6, 0.4) to strengthen the discriminative role of width. During the experiment, to ensure variable control, when exploring the influence of α on the current pose, the other poses are fixed to their respective optimal weights. In the standard, wide-body, extra-wide, and extreme poses, the tracker in this embodiment achieves optimal performance when α is 1.9, 1.5, 0.6, and 0.4, respectively.

[0111] To verify the effectiveness of the iterative layering strategy in the DAM of this invention, the performance under different configurations (DanceTrack validation set) is compared in Table 6. In the single-layer matching setting (rows 1-5 of Table 6), the HOTA shows a trend of first increasing and then decreasing as the number of layers increases from 1 to 8. Further analysis revealed that overly fine-grained division (e.g., 8 layers) leads to sparse distribution of targets in each layer. Simultaneously, the nonlinear relationship between proportional deformation and depth, and the sensitivity of pseudo-depth estimation to bounding box deformation, have a more significant impact under fine-grained layering, causing targets to be incorrectly layered, resulting in matching failures and a drop in HOTA to 52.0. Based on this, this invention designs an iterative layering mechanism from fine to coarse (rows 6-10 of Table 6), where [8,6] represents the fine-grained layer and [4,2,1] represents the coarse-grained layer. Among all combinations, the iterative layering of [6,4,1] achieves 54.1, 37.7, and 55.3 on HOTA, AssA, and IDF1, respectively. Although slightly inferior to the optimal [6,4,2,1], this layering strategy, when combined with PAIoU, produces a good synergistic effect (see the last row of Table 4), indicating that this setting can effectively mitigate the interference caused by pose changes on pseudo-depth estimation. Therefore, [6,4,1] can be ultimately selected as the default layering strategy for DAM.

[0112] Table 6. Effectiveness of different stratification strategies in DAM

[0113]

[0114] To verify the generalization ability of the proposed multi-target tracking method, PAIoU and DAM modules were integrated into four representative trackers: SORT, DeepSORT, MOTDT, and ByteTrack. The performance comparisons after integration are shown in Table 7-8 (DanceTrack validation set). This paper models the differential effects of pose changes and depth displacement on bounding box deformation and constructs a robust geometric deformation mechanism to reduce the interference of deformation on the correlation. Experimental results show that the introduction of either PAIoU or DAM significantly improves various metrics, especially correlation-related metrics such as IDF1 and AssA (e.g., PAIoU improves AssA by +3.8 and IDF1 by +3.2 on ByteTrack; DAM improves AssA by +2.0 and IDF1 by +3.7 on DeepSORT). Although the improvement was limited in individual cases (e.g., the AssA of PAIoU on MOTDT) or there were slight fluctuations (e.g., a slight decrease in AssA of DAM on ByteTrack), overall, considerable and consistent performance gains were achieved across the four different tracker frameworks. The results demonstrate that PAIoU and DAM in the embodiments of this invention have good generalization capabilities, can be easily integrated into mainstream trackers, and deliver stable performance improvements.

[0115] Table 7 Performance of PAIoU on Representative Trackers

[0116]

[0117] Table 8 Performance of DAM in Representative Trackers

[0118]

[0119] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A multi-target tracking method, characterized in that, Used to simultaneously locate multiple targets in each image frame of a video sequence while maintaining the consistency of the target's identity; The multi-target tracking method includes: Motion estimation is performed on the trajectory of each target in the current frame image based on the Kalman filter. In the next frame, the detected targets and the predicted trajectories are associated based on the Hungarian algorithm. When performing the associated operation: Suppress correlation interference caused by target deformation based on attitude-aware cross-union comparison method and depth-aware cascaded matching method; in: The attitude-aware intersection-union method is based on the aspect ratio to characterize the target's attitude and then fused with the standard IoU element-wise to improve adaptability to attitude distortion and reduce correlation interference caused by non-uniform deformation; the multi-target tracking method is based on diagonal overlap calculation and adaptive weighting mechanism to dynamically enhance the discrimination role of height or width under different target attitudes. The depth-aware cascaded matching method constructs a depth hierarchy based on pseudo-depth and adopts a cascaded matching process from fine-grained to coarse-grained. It gradually relaxes the depth constraints on the target, restricts the association search to a similar depth range, and reduces association interference caused by proportional scaling.

2. The multi-target tracking method according to claim 1, characterized in that, The method for calculating diagonal overlap is as follows: Geometric similarity is characterized by calculating the pose-aware diagonal intersection-over-union ratio, which is the proportion of overlap between the detection box and the trajectory box along the diagonal direction. In the formula, DiIoU is the intersection-union ratio of the attitude-aware diagonals. and These represent the width and height of the intersection region between the detection box and the trajectory box, respectively. and These represent the width and height of the union region of the detection box and the trajectory box, respectively.

3. The multi-target tracking method according to claim 2, characterized in that, The adaptive weighting mechanism dynamically enhances the discrimination effect of width and height under different postures by adjusting the weights of width and height in DiIoU, specifically as follows: Define the optimized pose-aware diagonal intersection-union ratio as: Then we have: In the formula, r is the aspect ratio; threshold , and Used to divide four types of attitude ranges, among which Indicates the standard posture. Indicates a wide-body stance. Indicates an extremely wide stance. It indicates an extreme stance.

4. The multi-target tracking method according to claim 3, characterized in that, When performing the association operation, if the pose-aware intersection-union ratio is defined based on element-wise multiplication, then: In the formula, and These represent the bounding box areas of the two detected targets; Indicates the pose-aware intersection-union ratio; This indicates the standard intersection-union ratio.

5. The multi-target tracking method according to claim 1, characterized in that, The depth-aware cascaded matching method includes the following steps: S101: Calculate the pseudo-depth of all detected targets in the image frame to determine the detection range. The detection range is evenly divided into L sub-intervals. The detection targets are assigned to corresponding sub-intervals according to their pseudo-depth, forming detection subsets. ; Calculate the pseudo-depth of all trajectories in the image frame to determine the trajectory range interval, and uniformly divide the trajectory range interval into L sub-intervals; assign the trajectories to the corresponding sub-intervals according to their pseudo-depth to form the obtained trajectory subset. ; S102: For each subinterval... and The association operation is performed in between; the detected targets and trajectories that fail to match are saved respectively.

6. The multi-target tracking method according to claim 5, characterized in that, The cascaded matching process from fine-grained to coarse-grained is as follows: the image frame is divided into fine-grained segments, targets with similar depth variations are aggregated, and then the depth constraints are gradually relaxed to facilitate the re-association of mis-layered targets.

7. The multi-target tracking method according to claim 6, characterized in that, The depth-aware cascaded matching method further includes the following steps: S103: Take the unmatched set as input, reduce the number of layers L, and re-execute S101-S102 within each set range; S104: Repeat S103 until the termination condition of the associated operation is met.

8. The multi-target tracking method according to claim 1, characterized in that, The specific method for gradually relaxing the depth constraint is as follows: the number of layers in the layering is continuously reduced in the iterative association, and the matching within the layer adopts a preset association threshold, which helps to prevent matching failure caused by layering error.

9. The multi-target tracking method according to claim 1, characterized in that, The depth-aware cascaded matching method further includes: when there is noise in the pseudo-depth, adaptively correcting the erroneous layering by expanding the search domain to alleviate the matching failure caused by the erroneous layering.

10. The multi-target tracking method according to claim 1, characterized in that, When performing the association operation, the following rules are followed: first, the association operation is performed on high-confidence targets; second, low-confidence targets are processed; and finally, the lost trajectory is recovered using historical detection results.

Images