Occlusion-Aware Multi-Object Tracking With Periodic Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object tracking systems struggle with partial or total occlusion of objects in crowded scenes, leading to tracking errors and high resource and time requirements, particularly in real-time applications.
Innovation Solution
A two-stage process combining periodic object detection and single-object tracking, leveraging a correlation filter tracker (CFNet) for real-time multi-object tracking, with a novel tracking loss process that utilizes temporal context to enhance performance and reduce computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multi-object tracking is performed using traditional detection-based methods, then object identities can be maintained, but computational resources and processing time increase significantly
Solution Approach 1:
The patent segments the tracking process into two distinct modes: detection mode (performed periodically on select frames) and tracking mode (performed on all frames). This segmentation allows the computationally intensive detection algorithm to run less frequently while a lighter tracking algorithm handles intermediate frames, resolving the contradiction between tracking accuracy and processing speed.
Solution Approach 2:
The system implements periodic object detection at intervals rather than on every frame. Detection mode is activated based on conditions such as frame interval thresholds or tracking confidence levels, allowing the system to switch between heavy detection and light tracking operations periodically, thereby reducing overall computational load while maintaining accuracy.
2Measurement precision
If object detection is performed on every frame, then detection accuracy is maintained, but resource requirements and processing time increase
Solution Approach 1:
Object detection is performed periodically rather than on every frame. The system determines whether to execute detection based on frame intervals, tracking confidence thresholds, or occlusion events, significantly reducing the number of detection operations while maintaining overall detection accuracy through strategic sampling.
Solution Approach 2:
The system performs preliminary tracking on intermediate frames to maintain object states and predictions before periodic detection updates. This preliminary action keeps tracking data current between detection events, reducing the need for frequent full detection passes and minimizing processing time while preserving detection accuracy.
3Reliability
If detection-based tracking is used to handle occlusions, then object identities are maintained during occlusion, but computational complexity increases
Solution Approach 1:
The system dynamically switches between detection mode and tracking mode based on real-time conditions such as occlusion detection, frame intervals, and tracking confidence levels. This dynamic adaptation allows the system to use heavy detection only when necessary (e.g., during occlusion events) while relying on lighter tracking during normal conditions, reducing overall system complexity.
Solution Approach 2:
The patent introduces an intermediary tracking process that operates between periodic detection events. This intermediary tracker maintains object states, predictions, and occlusion handling during intervals when full detection is not performed, serving as a mediator that reduces the need for continuous complex detection while maintaining tracking robustness during occlusions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosure herein enables tracking of multiple objects in a real-time video stream. For each individual frame received from the video stream, a frame type of the frame is determined. Based on the individual frame being an object detection frame type, a set of object proposals is detected in the individual frame, associations between the set of object proposals and a set of object tracks are assigned, and statuses of the set of object tracks are updated based on the assigned associations. Based on the individual frame being an object tracking frame type, single-object tracking is performed on the frame based on each object track of the set of object tracks and the set of object tracks is updated based on the performed single-object tracking. For each frame received, a real-time object location data stream is provided based on the set of object tracks.