Correlation Filter Tracking with Occlusion Maps for Crowded Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-object tracking systems face challenges in computational efficiency, storage requirements, and difficulty in tracking objects in crowded or occluded environments, particularly when using correlation filters for localization and data association.
Innovation Solution
Implementing batch processing of image areas using Graphics Processing Units (GPUs) for parallel localization and filter updates, applying focused windowing and occlusion maps to enhance correlation filter learning, and associating locations based on correlation response values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional single-object trackers are used independently on full video frames, then object localization can be achieved, but computational requirements and storage needs increase significantly when tracking multiple objects concurrently
Solution Approach 1:
The patent divides the video frame into multiple regions of interest (ROIs) and processes each ROI independently with dedicated correlation filters. This segmentation allows parallel processing of multiple objects simultaneously, reducing the computational burden compared to processing entire frames while maintaining localization accuracy for each object.
Solution Approach 2:
The patent combines multiple single-object trackers into a unified multi-object tracking system that shares common resources such as correlation filter banks and processing infrastructure. This merging enables efficient concurrent tracking of multiple objects by eliminating redundant computations and storage requirements.
2Adaptability or versatility
If multiple single-object trackers run concurrently to track multiple objects, then multi-object tracking capability is achieved, but storage requirements limit the number of objects that can be tracked simultaneously
Solution Approach 1:
The patent implements a universal correlation filter bank that serves all object trackers simultaneously. Each filter in the bank can be applied to multiple different objects and video streams, eliminating the need for separate filters for each tracker. This multi-functionality dramatically reduces storage requirements while maintaining the ability to track any number of objects concurrently.
3Ease of operation
If conventional approaches process non-homogeneous tracker data serially, then data association can be performed, but processing time increases and efficiency decreases
Solution Approach 1:
The patent transforms diverse tracker outputs into a homogeneous data structure with consistent formatting, dimensions, and feature representations. This homogenization enables all trackers to be processed in parallel batches rather than serially, dramatically reducing processing time while maintaining data association accuracy through unified correlation-based matching.
4Reliability
If correlation filters are learned from image areas of various sizes and shapes, then object-specific features are captured, but processing complexity increases and parallel processing becomes difficult
Solution Approach 1:
The patent dynamically adjusts correlation filter parameters including kernel size, sigma values, and sampling rates based on the specific characteristics of each object and its search region. This parameter adaptation allows the system to maintain high feature representation accuracy for objects of varying sizes and shapes while optimizing computational efficiency for parallel GPU processing.
Data Source
AI summary
In various examples, correlation filters may be learned for object tracking based at least on occlusion maps. When learning a correlation filter from an image area, an occlusion map may be applied to the image area that masks, excludes, and/or blurs occlusions of the target object. The correlation filter may be learned from a modified image, thereby reducing or eliminating learning from occlusions while still allowing for learning the target object from exposed portions. The occlusion maps may be generated using a machine learning model, such as a Gaussian Mixture Model (GMM) that is trained (e.g., using the image areas used to learn the correlation filter) using the target object as a background so that occlusions are detected as foreground.