A cascade multi-target tracking method and system for point tracking verification

Abstract

This invention discloses a cascaded multi-target tracking method and system with point tracking verification, relating to the fields of computer vision and target tracking technology. The method includes the following steps: Step 1, frame detection and trajectory initialization; Step 2, construction and execution of the point tracking verification module; Step 3, dynamic two-stage cascaded matching based on verification results; and Step 4, trajectory filtering by the short trajectory suppression module. This invention employs the aforementioned cascaded multi-target tracking method and system with point tracking verification. It achieves refined verification of trajectory-detection matching through feature point extraction and optical flow tracking, dynamically optimizes the matching cost matrix based on the verification results, and designs a dual-threshold short trajectory filtering mechanism to accurately eliminate pseudo-trajectories. Supplemented by a lightweight computational optimization strategy, it improves tracking robustness and accuracy while ensuring real-time performance. The overall solution can be non-intrusively integrated into existing tracking frameworks.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and target tracking technology, and in particular to a cascaded multi-target tracking method and system for point tracking verification. Background Technology

[0002] Multi-target tracking is one of the core research directions in computer vision. Its core lies in accurately locating and identifying multiple targets within a continuous video frame sequence while maintaining the continuity of their trajectories. It is a key technological support for intelligent applications such as autonomous driving environmental perception, robot autonomous navigation, intelligent security monitoring, and traffic scene analysis, and has extremely high research and application value in both industry and academia. Existing detection-based multi-target tracking technologies, especially two-stage cascaded tracking methods, still suffer from numerous technical shortcomings in practical engineering applications, particularly in complex scenarios such as occlusion, rapid movement, dense crowds, and sudden changes in lighting. Tracking robustness, accuracy, and continuity all decrease significantly, failing to meet the actual needs of high-end intelligent applications such as autonomous driving and robotics. Specific problems include: in scenarios with occlusion and rapid movement, trajectory fragmentation, target identity switching, and low matching accuracy due to reliance solely on bounding box IoU matching; and the inability to effectively filter short pseudo-trajectories formed by detector false detections in complex scenarios, leading to a high number of false detections and a decline in core indicators. Improving overall tracking accuracy is crucial. Summary of the Invention

[0003] The purpose of this invention is to provide a cascaded multi-target tracking method and system with point tracking verification, which integrates the advantages of point tracking and feature detection, optimizes the matching mechanism, effectively filters pseudo trajectories, and takes into account both real-time performance and engineering implementation.

[0004] To achieve the above objectives, this invention provides a cascaded multi-target tracking method with point tracking verification, comprising the following steps: Step 1: Frame detection and trajectory initialization, obtaining a high-confidence detection result set through the target detector. D high and low confidence test result set D low It initializes active trajectories and generates an active trajectory set containing target motion information and feature point information. T ; Step 2: Construction and execution of the point tracking verification module. Design a multi-layered point tracking verification module to complete the fine-grained verification of the matching relationship between the trajectory and the detection results. Step 3: Dynamic two-stage cascaded matching based on verification results. Combine point tracking verification results to dynamically adjust the IoU fusion distance matrix, optimize the matching strategy, and improve the matching accuracy in complex scenarios. Step 4: Trajectory selection by the short trajectory suppression module. The short trajectory suppression module selects effective trajectories based on a dual threshold decision function that considers trajectory length and current frame detection confidence.

[0005] Preferably, step two includes the following steps: S21. Feature point extraction for active trajectory sets. The historical bounding boxes of each trajectory and the current frame D high and D low All detected bounding boxes In the image region corresponding to the bounding box Inside, the Shi-Tomasi corner detector is used to extract the feature point set: ; S22, Optical Flow Tracking and Multi-Layer Verification, for the trajectory of the previous frame. T feature point set The Lucas-Kanade pyramid optical flow algorithm is used to track and obtain the feature point set of the current frame. The forward-backward consistency check, RANSAC affine transformation verification, and interior point count and region verification are performed sequentially to achieve accurate determination of the matching relationship through multi-layer verification. S23. Output the verification results and construct the verification matrix: ; in, Representing the trajectory With detection Verification passed. This indicates that the verification failed.

[0006] Preferably, in S21, the maximum number of corner points to be extracted is set. Corner quality threshold Minimum pixel distance of feature points Pixels, filtering out low-quality and invalid feature points.

[0007] Preferably, step three includes the following steps: S31. First-stage high-confidence matching: Calculate the active trajectory set T and the high-confidence detection set. D high IoU distance matrix D IoU The fusion distance matrix is ​​obtained by fusing detection confidence; based on the validation matrix... V Dynamically adjust the distance matrix and set coefficients for verified matching pairs. Reduce the matching distance for verification failures and fusion distances below a certain threshold. Set coefficients for matching pairs Increase the matching distance, and keep the original distance unchanged in all other cases to obtain the adjusted distance matrix. D adj The Hungarian algorithm was used to... D adj Perform matching to obtain matched trajectory-detection pairs, and retain the set of unmatched trajectories. T remain ; S32, Second Stage Low-Confidence Matching: Matching the Unmatched Trajectory Set T remain With low confidence detection set D low Repeat the distance matrix calculation, dynamic adjustment, and Hungarian matching operations described above to complete the second matching; update the active trajectory set based on the results of the two matchings. T Successfully matched trajectories update their bounding boxes, feature points, and other information, while unmatched trajectories are marked as lost.

[0008] Preferably, step four includes the following steps: S41. Trajectory Feature Statistics: Statistical analysis of each trajectory. T i The core features include trajectory length L i and the detection confidence of the current frame S i ; S42. Dual threshold decision filtering: setting a trajectory length threshold. Frame, short trajectory confidence threshold Through decision function ; Determine the validity of the trajectory; S43. Output the valid trajectories. Retain the trajectories with a decision function result of 1, and discard the short and low-confidence pseudo-trajectories with a result of 0 to obtain the final set of trajectories to be output: .

[0009] A cascaded multi-target tracking system for point tracking verification includes a data acquisition module, a data processing module, a trajectory decision module, and a result output module. The data acquisition module is used to acquire video frames and detection boxes; the data processing module is used to perform confidence filtering and trajectory caching; the trajectory decision module is used to perform correlation calculations based on point trajectory constraints and temporal stability strategies; and the result output module is used to generate trajectory files and output evaluation results.

[0010] Therefore, this invention employs a cascaded multi-target tracking method and system with point tracking verification, significantly improving tracking robustness and accuracy in complex scenarios. By innovatively combining point trajectory tracking with traditional detection box appearance features, it effectively solves the association error problem that easily arises from relying solely on appearance similarity or motion models in challenging scenarios such as target occlusion, rapid movement, and appearance changes. The point tracking verification module utilizes optical flow to track local feature points within the target, providing continuous motion consistency evidence independent of the overall appearance. It achieves efficient and accurate false trajectory filtering, improving output quality: the Short Trajectory Suppression (STS) module performs post-processing filtering on activated trajectories using simple rules (based on trajectory length and average confidence), effectively identifying and removing transient low-confidence false trajectories caused by detector noise or transient erroneous associations. The module is lightweight and flexibly integrated, improving performance without significantly increasing computational overhead: the point tracking verification module is triggered only when there is ambiguity in traditional IoU association (controlled by a preset IoU range threshold) and employs a feature point caching and update strategy to avoid repeated calculations per frame. The short trajectory suppression module is just a simple logical judgment with extremely low computational cost.

[0011] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0012] Fig. 1 This is a flowchart of a cascaded multi-target tracking method for point tracking verification according to the present invention; Fig. 2 This is a diagram of the device of the present invention. Detailed Implementation

[0013] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0014] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0015] Example Please see Figs. 1-2 This invention provides a cascaded multi-target tracking method with point tracking verification, comprising the following steps: Step 1: Frame Detection and Trajectory Initialization. For each frame of the video sequence, YOLOX-X is used as the basic object detector to obtain a high-confidence detection result set. D high Detection result sets with confidence levels higher than a preset threshold and with low confidence levels. D low If the confidence level is lower than a preset threshold, each detection result includes the target bounding box coordinates and a confidence score, and an active trajectory set containing target motion information and feature point information is initialized. T This provides a foundation for subsequent tracking and matching.

[0016] Step 2: Construction and execution of the point tracking verification module. Design a multi-layered point tracking verification module to complete the fine-grained verification of the matching relationship between the trajectory and the detection results.

[0017] S21. Feature point extraction for active trajectory sets. The historical bounding boxes of each trajectory and the current frame D high and D low All detected bounding boxes In the image region corresponding to the bounding box Inside, the Shi-Tomasi corner detector is used to extract the feature point set: ; Set the maximum number of corner points to extract. Corner quality threshold Minimum pixel distance of feature points Pixels, filtering out low-quality and invalid feature points.

[0018] S22, Optical Flow Tracking and Multi-Layer Verification, for the trajectory of the previous frame. T feature point set The Lucas-Kanade pyramid optical flow algorithm is used to track and obtain the feature point set of the current frame. Perform forward-backward consistency checks sequentially (filtering for Euclidean distances greater than 100%). Pixel feature points), RANSAC affine transformation verification (setting reprojection error threshold) Pixel inlier feature extraction), inlier count and region validation (minimum inlier count required) (And at least 42% of the inlier features fall within the current detection bounding box), achieving accurate determination of matching relationships through multi-layer verification.

[0019] S23. Output the verification results and construct the verification matrix: ; in, Representing the trajectory With detection Verification passed. This indicates that the verification failed, providing a basis for subsequent adjustments to the matching cost matrix.

[0020] Step 3: Dynamic two-stage cascaded matching based on verification results. Combine point tracking verification results to dynamically adjust the IoU fusion distance matrix, optimize the matching strategy, and improve matching accuracy in complex scenarios.

[0021] S31. First-stage high-confidence matching: Calculate the active trajectory set T and the high-confidence detection set. D high IoU distance matrix D IoU（ ） The fusion distance matrix is ​​obtained by fusing detection confidence. ( For detection box (Confidence level); based on the verification matrix V Dynamically adjust the distance matrix and set coefficients for verified matching pairs. Reduce the matching distance for verification failures and fusion distances below a certain threshold. Set coefficients for matching pairs Increase the matching distance, and keep the original distance unchanged in all other cases to obtain the adjusted distance matrix. D adj The Hungarian algorithm was used to... D adj Perform matching to obtain matched trajectory-detection pairs, and retain the set of unmatched trajectories. T remain ; S32, Second Stage Low-Confidence Matching: Matching the Unmatched Trajectory Set T remain With low confidence detection set D low Repeat the distance matrix calculation, dynamic adjustment, and Hungarian matching operations described above to complete the second matching; update the active trajectory set based on the results of the two matchings. T Successfully matched trajectories update their bounding boxes, feature points, and other information, while unmatched trajectories are marked as lost.

[0022] Step 4: Trajectory selection by the short trajectory suppression module. The short trajectory suppression module selects effective trajectories based on a dual threshold decision function that considers trajectory length and current frame detection confidence.

[0023] S41. Trajectory Feature Statistics: Statistical analysis of each trajectory. T i The core features include trajectory length L i and the detection confidence of the current frame S i .

[0024] S42. Dual threshold decision filtering: setting a trajectory length threshold. Frame, short trajectory confidence threshold Through the decision function: ; Determine the validity of the trajectory.

[0025] S43. Output the valid trajectories. Retain the trajectories with a decision function result of 1, and discard the short and low-confidence pseudo-trajectories with a result of 0 to obtain the final set of trajectories to be output: .

[0026] Ablation experiments on the MOT17 benchmark dataset fully demonstrate the effectiveness of this invention. The experimental results are shown in Table 1. The results indicate that: Using the point tracking verification (+pt) module alone can improve the HOTA of the baseline method from 60.21% to 60.32%, MOTA from 73.57% to 73.87%, and IDF1 from 72.86% to 73.00%, while significantly reducing identity switching (IDSW) (from 6111 times to 5919 times).

[0027] Using the short trajectory suppression (+sts) module alone also improved all major metrics (HOTA: 60.21%→60.31%, MOTA: 73.57%→73.68%, IDF1: 72.86%→72.98%) and reduced trajectory breakage (Frag: 3738→3714).

[0028] Combining the two modules (baseline + pt + sts) achieved optimal performance, with HOTA reaching 60.40%, MOTA reaching 73.95%, and IDF1 reaching 73.11%, fully demonstrating the complementarity between the modules and achieving comprehensive optimization of tracking accuracy, correlation consistency, and output purity.

[0029] Table 1 shows the ablation experiment results on the MOT17 dataset. A cascaded multi-target tracking system with point tracking verification includes a data acquisition module, a data processing module, a trajectory decision module, and a result output module. The data acquisition module is used to acquire video frames and detection boxes; the data processing module is used to perform confidence filtering and trajectory caching; the trajectory decision module is used to perform correlation calculations based on point trajectory constraints and temporal stability strategies; and the result output module is used to generate trajectory files and output evaluation results.

[0030] The method in this embodiment includes: S1, acquiring the original detection data D; S2, normalizing D to obtain D1; S3, constructing an association cost function and completing matching under the association threshold to obtain the initial trajectory T0; S4, calculating the trajectory stability score based on the consistency of displacement, velocity and direction, and performing reassociation or termination on abnormal trajectories; S5, using a sliding window smoothing and short-term occlusion keep-alive mechanism to obtain the optimized trajectory T1; S6, outputting the final tracking result.

[0031] The parameters can be selected from the following range: detection threshold Association threshold Sliding window length The input video resolution is ,in , Typical values ​​are 1920×1080 or 3840×2160. This embodiment is deployed on a computing device containing an NVIDIA Titan V graphics card, and the parameters can be adaptively adjusted according to scene density and resolution.

[0032] The method can be implemented by a computer program, which is stored in a computer-readable storage medium and executed by a processor.

[0033] Therefore, this invention employs a cascaded multi-target tracking method and system with point tracking verification, significantly improving tracking robustness and accuracy in complex scenarios. By innovatively combining point trajectory tracking with traditional detection box appearance features, this invention effectively solves the association error problem that easily arises from relying solely on appearance similarity or motion models in challenging scenarios such as target occlusion, rapid movement, and appearance changes. The point tracking verification module utilizes optical flow to track local feature points within the target, providing continuous motion consistency evidence independent of the overall appearance. This allows the system to make more reliable association decisions based on the spatiotemporal continuity of point trajectories even in frames where the target is partially or completely occluded, or where the detection box is incomplete or unreliable, thereby significantly reducing identity switching and trajectory breakage.

[0034] This invention achieves efficient and accurate false trajectory filtering, improving output quality: The Short Trajectory Suppression (STS) module uses simple rules (based on trajectory length and average confidence) to post-process and filter activated trajectories, effectively identifying and removing transient low-confidence false trajectories caused by detector noise or transient erroneous associations. This mechanism directly reduces false positives at the output end, improving the purity and reliability of the final trajectory set, while also reducing the burden on subsequent processing modules.

[0035] The modules are lightweight and flexibly integrated, improving performance without significantly increasing computational overhead: the point tracking verification module is triggered only when there is ambiguity in the traditional IoU association (controlled by a preset IoU range threshold), and it employs a feature point caching and update strategy to avoid repeated calculations in each frame. The short trajectory suppression module is merely a simple logical judgment with extremely low computational cost. Therefore, the two core enhancement modules of this invention can be integrated into existing two-stage tracking frameworks (such as ByteTrack) in a non-intrusive and low-overhead manner, achieving stable performance improvements while maintaining the original system efficiency framework essentially unchanged.

[0036] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A cascaded multi-target tracking method for point tracking verification, characterized in that, Includes the following steps: Step 1: Frame detection and trajectory initialization, obtaining a high-confidence detection result set through the target detector. D high and low confidence test result set D low It initializes active trajectories and generates an active trajectory set containing target motion information and feature point information. T ; Step 2: Construction and execution of the point tracking verification module. Design a multi-layered point tracking verification module to complete the fine-grained verification of the matching relationship between the trajectory and the detection results. Step 3: Dynamic two-stage cascaded matching based on verification results. Combine point tracking verification results to dynamically adjust the IoU fusion distance matrix, optimize the matching strategy, and improve the matching accuracy in complex scenarios. Step 4: Trajectory selection by the short trajectory suppression module. The short trajectory suppression module selects effective trajectories based on a dual threshold decision function that considers trajectory length and current frame detection confidence.

2. The cascaded multi-target tracking method for point tracking verification according to claim 1, characterized in that, Step two includes the following steps: S21. Feature point extraction for active trajectory sets. The historical bounding boxes of each trajectory and the current frame D high and D low All detected bounding boxes In the image region corresponding to the bounding box Inside, the Shi-Tomasi corner detector is used to extract the feature point set: ； S22, Optical Flow Tracking and Multi-Layer Verification, for the trajectory of the previous frame. T feature point set The Lucas-Kanade pyramid optical flow algorithm is used to track and obtain the feature point set of the current frame. The forward-backward consistency check, RANSAC affine transformation verification, and interior point count and region verification are performed sequentially to achieve accurate determination of the matching relationship through multi-layer verification. S23. Output the verification results and construct the verification matrix: ； in, Representing the trajectory With detection Verification passed. This indicates that the verification failed.

3. The cascaded multi-target tracking method for point tracking verification according to claim 2, characterized in that: Set the maximum number of corner points to extract in S21. Corner quality threshold Minimum pixel distance of feature points Pixels, filtering out low-quality and invalid feature points.

4. The cascaded multi-target tracking method for point tracking verification according to claim 3, characterized in that, Step three includes the following steps: S31. First-stage high-confidence matching: Calculate the active trajectory set T and the high-confidence detection set. D high IoU distance matrix D IoU The fusion distance matrix is ​​obtained by fusing detection confidence; based on the validation matrix... V Dynamically adjust the distance matrix and set coefficients for verified matching pairs. Reduce the matching distance for verification failures and fusion distances below a certain threshold. Set coefficients for matching pairs Increase the matching distance, and keep the original distance unchanged in all other cases to obtain the adjusted distance matrix. D adj The Hungarian algorithm was used to... D adj Perform matching to obtain matched trajectory-detection pairs, and retain the set of unmatched trajectories. T remain ; S32, Second Stage Low-Confidence Matching: Matching the Unmatched Trajectory Set T remain With low confidence detection set D low Repeat the distance matrix calculation, dynamic adjustment, and Hungarian matching operations described above to complete the second matching; update the active trajectory set based on the results of the two matchings. T Successfully matched trajectories update their bounding boxes, feature points, and other information, while unmatched trajectories are marked as lost.

5. The cascaded multi-target tracking method for point tracking verification according to claim 4, characterized in that, Step four includes the following steps: S41. Trajectory feature statistics: Statistical analysis of each trajectory. T i The core features include trajectory length L i and the detection confidence of the current frame S i ; S42. Dual threshold decision filtering: setting a trajectory length threshold. Frame, short trajectory confidence threshold Through decision function ； Determine the validity of the trajectory; S43. Output the valid trajectories. Retain the trajectories with a decision function result of 1, and discard the short and low-confidence pseudo-trajectories with a result of 0 to obtain the final set of trajectories to be output: 。 6. A cascaded multi-target tracking system with point tracking verification, applied to the cascaded multi-target tracking method with point tracking verification as described in any one of claims 1-5, characterized in that: It includes a data acquisition module, a data processing module, a trajectory decision module, and a result output module. The data acquisition module is used to acquire video frames and detection boxes; the data processing module is used to perform confidence filtering and trajectory caching; the trajectory decision module is used to perform correlation calculations based on point trajectory constraints and temporal stability strategies; and the result output module is used to generate trajectory files and output evaluation results.

Images