Graph-Based Object Tracking with Lifted Edges for Long-Distance Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-person tracking systems face challenges in accurately tracking individuals across long distances and distinguishing between similar-looking persons in crowded scenes, often leading to confusion and propagation of errors due to the complexity of motion under perspective projection and the asymptotic uncorrelation of motion models.
Innovation Solution
A system and method that utilize a graph-based approach with two types of edges (regular and lifted edges) to model the tracking problem as a minimum cost lifted multicut problem, incorporating deep networks for re-identification and incorporating long-range information to verify and update tracks, while avoiding the confusion of similar-looking individuals by penalizing long-term false joints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Duration of action of moving object
If motion models are used for tracking, then tracking can be performed in crowded scenes, but the models become asymptotically uncorrelated over long distances leading to tracking failures
Solution Approach 1:
The patent combines motion models with appearance models into a unified tracking framework. While motion models provide short-term prediction, appearance models provide long-term identification consistency, merging the strengths of both approaches to maintain reliable tracking across long distances without the asymptotic uncorrelation problem.
Solution Approach 2:
The patent implements feedback mechanisms where tracking results are continuously refined by comparing predicted positions with actual detections and updating appearance representations. This feedback loop maintains correlation over long distances by constantly reinforcing the connection between tracklets through appearance consistency checks.
2Duration of action of moving object
If appearance-based re-identification is used to track persons across long distances, then long-range tracking is improved, but similar-looking persons are incorrectly identified as identical
Solution Approach 1:
The patent applies local quality by using multiple localized appearance features (e.g., body parts, clothing regions) rather than relying on a single global appearance representation. This allows the system to distinguish between similar-looking persons by comparing specific local features that may differ even when overall appearance is similar.
Solution Approach 2:
The patent combines multiple types of information (motion characteristics, localized appearance features, spatial-temporal context) into a composite tracking representation. This composite approach allows the system to disambiguate between similar-looking persons by considering the full multi-modal evidence rather than relying solely on appearance.
3Device complexity
If tracklets are introduced to reduce state space, then computational complexity is reduced, but mistakes in tracklet generation are propagated to the final solution
Solution Approach 1:
The patent prepares for potential tracklet generation errors by implementing verification mechanisms that check tracklet consistency before final assignment. This cushioning approach allows the system to detect and correct mistakes before they propagate to the final solution, maintaining reliability while still benefiting from the computational efficiency of tracklet-based approaches.
4Reliability
If CNN-based feature representations are used to model relations between detections, then appearance-based tracking is improved, but computational cost increases
Solution Approach 1:
The patent segments the computational task by processing only relevant detection pairs using CNN-based appearance comparison. Rather than computing all pairwise comparisons, the system identifies and processes only those pairs that are spatially and temporally relevant, significantly reducing computational energy while maintaining tracking reliability through targeted appearance analysis.
Data Source
AI summary
A system for tracking objects in a temporal sequence of digital images is configured to: detect potential objects in the images, the detected potential objects being indicated as nodes, identify pairs of neighboring nodes, such that for each pair the nodes of said pair potentially represent an identical object and their spatial and/or temporal relationship with each other is within a predetermined range, connect each pair of neighboring nodes with a first type edge, identify at least one supplementary pair of distant nodes whose spatial and/or temporal relationship with each other exceeds the predetermined range, connect the pair of distant nodes with a supplementary second type edge, each of the first and second type edges being assigned a cost value, and determine a track of an object in the temporal sequence of digital images based on a set of connected first type edges and at least one second type edge.