Contrastive Loss Training for Unsupervised Multi-Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-object tracking algorithms face challenges in real-time performance due to labor-intensive trajectory annotation and imbalance in trajectory lengths, leading to inefficiencies in the Joint Detection and Embedding (JDE) paradigm, especially when dealing with large numbers of trajectories and inconsistent lengths.
Innovation Solution
A contrastive loss based training strategy for unsupervised multi-object tracking is introduced, utilizing a Self-Supervised Contrastive ID (SSCI) module that constrains embedded features through self-supervised contrastive loss, enhances cross-frame expression ability via forward and reverse matching, and employs the Hungarian algorithm for consistent matching pairs, eliminating the need for trajectory annotation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If classification-based embedding training is used, then tracking accuracy is improved for small datasets, but model fitting difficulty increases and performance limits occur when trajectory number is large
Solution Approach 1:
The patent changes the training objective from classification-based to contrastive loss-based. Instead of classifying trajectories into categories with a fully connected layer, the method uses contrastive loss to directly optimize embedding features based on spatial relationships between objects in adjacent frames. This parameter change eliminates the need for trajectory category classification and avoids the fitting difficulties associated with large numbers of trajectories.
2Adaptability or versatility
If JDE paradigm with common backbone network is used, then unified feature extraction is achieved, but sub-task conflict occurs leading to lack of effect
Solution Approach 1:
The patent segments the feature extraction process into separate branches: a detection branch and an embedding branch. The embedding branch is specifically designed for tracking tasks with contrastive loss optimization, while the detection branch handles object detection. This segmentation allows each branch to be optimized for its specific sub-task without conflict, resolving the effectiveness issue in the JDE paradigm.
3Measurement precision
If trajectory annotation is used for training, then tracking performance is improved, but annotation labor intensity becomes extremely high
Solution Approach 1:
The patent implements self-supervised learning where the model automatically generates training data from video frames without human annotation. The contrastive loss function uses spatial relationships between objects in adjacent frames to create positive and negative sample pairs, allowing the model to learn tracking features autonomously. This self-service approach eliminates the need for manual trajectory annotation while maintaining tracking performance.
Data Source
AI summary
The present invention relates to unsupervised tracking technology, specifically an unsupervised tracking model training strategy based on contrastive loss. The method comprises: S1: forming a constrained SSCI module using the relation between objects within a video frame and between adjacent video frames; S2: setting features of different objects in each frame as negative samples, and similar adjacent frame objects as positive sample pairs, constructing contrastive loss; S3: constraining embedded features (E_t) by variable loss based on self-supervised contrastive loss. This invention provides a contrastive loss-based training strategy for unsupervised multi-object tracking, leveraging the prior that objects in a frame must be different to enhance object similarity, and using self-supervised learning to match similar objects in short-interval frames as positive samples to boost cross-frame feature expression. Finally, it further improves cross-frame feature expression by ensuring consistent forward and reverse matching.