Video Object Tracking via Depth-Aware Appearance Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video surveillance systems face challenges in robustly tracking multiple moving objects, particularly due to occlusions caused by stationary or dynamic objects, which can lead to loss of object identity and inefficient appearance-based tracking methods that fail under sudden changes in object appearance.
Innovation Solution
A method that identifies and classifies pixels in merged objects using appearance models and weighting factors based on depth order, employing normalized color histograms and edge density information to maintain object tracking identities during occlusions, and utilizing a combination of blob-based and appearance-based tracking to handle complex occlusion scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If appearance-based tracking is used to track objects, then object identity can be maintained, but the method fails under sudden changes in object appearance and has high computational complexity
Solution Approach 1:
The merged blob is segmented into multiple regions using graph cut optimization, where each region is assigned to a specific tracked object. This segmentation allows the system to handle occlusions by dividing the overlapping area into distinct object regions based on appearance model compatibility, thereby maintaining object identity even when objects merge or occlude each other.
Solution Approach 2:
The patent changes the parameter space by using normalized color histograms and depth affinity metrics instead of raw pixel values. The graph cut optimization uses energy minimization with data terms (appearance compatibility) and smoothness terms (spatial continuity) to dynamically adjust segmentation parameters, enabling robust tracking under appearance variations.
2Measurement precision
If graph cut optimization is used for blob segmentation, then segmentation accuracy is improved, but computational complexity increases
Solution Approach 1:
The graph cut optimization applies different weighting to different regions of the blob based on local appearance characteristics. The data term in the energy function compares local patch appearances with stored object templates, allowing accurate segmentation while focusing computational effort on discriminative regions rather than processing the entire blob uniformly.
Solution Approach 2:
The patent performs graph cut optimization only on merged blobs that require segmentation, rather than on all blobs in every frame. This selective application reduces overall computational complexity while maintaining segmentation accuracy where needed. The system identifies merged blobs through overlap detection and applies segmentation only to those cases.
3Productivity
If multiple objects are tracked simultaneously, then surveillance coverage is improved, but occlusions cause loss of object identity
Solution Approach 1:
The system performs preliminary actions by detecting object merges before they cause tracking failure. When blobs overlap in the current frame, the system proactively segments them using graph cut optimization and reassigns pixels to appropriate object identities. This preliminary segmentation prevents identity loss that would otherwise occur during occlusion events.
Solution Approach 2:
The tracking system uses feedback from appearance model matching scores to detect when objects have merged or occluded each other. The graph cut optimization uses feedback from the energy minimization process to iteratively refine segment boundaries, ensuring that object identities are correctly maintained even during complex occlusion scenarios involving multiple objects.
Data Source
AI summary
A video surveillance system (10) comprises a camera (25), a personal computer (PC) (27) and a video monitor (29). Video processing software is provided on the hard disk drive of the PC (27). The software is arranged to perform a number of processing operations on video data received from the camera, the video data representing individual frames of captured video. In particular, the software is arranged to identify one or more foreground blobs in a current frame, to match the or each blob with an object identified in one or more previous frames, and to track the motion of the or each object as more frames are received. In order to maintain the identity of objects during an occlusion event, an appearance model is generated for blobs that are close to one another in terms of image position. Once occlusion takes place, the respective appearance models are used, in combination with a depth factor representing the depth order of the occluded objects, to segment the resulting group blob into regions which are classified as representing one or other of the merged objects.