Object Tracking via Sub-Part Segmentation and Local Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image processing technologies face challenges in accurately tracking feature points of non-rigid objects, such as faces, under occlusion, as they struggle to capture shape changes and require prior knowledge of occluded feature points, and are not robust enough to distinguish between occluded and unoccluded features.
Innovation Solution
An image processing device that extracts features based on a shape with sub-parts, applies global and local models to estimate shapes, detects occluded sub-parts, and restricts their motion using selected local models to generate a final output, allowing for accurate tracking even under occlusion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If robust estimators are used to truncate part responses below a threshold, then occlusion handling is improved, but feature points consistent with the shape model but inconsistent with ground truth are incorrectly treated as outliers
Solution Approach 1:
The invention segments the shape into multiple sub-parts (e.g., left face, right face, top, bottom) and creates separate local models for each sub-part. This segmentation allows the system to independently handle occlusion in different regions without incorrectly treating all deviating feature points as outliers, thereby resolving the contradiction between occlusion handling and alignment accuracy.
Solution Approach 2:
The invention applies different motion restriction strategies to different sub-parts of the shape. Local models are created for each sub-part with specific motion restrictions tailored to that region. This local quality approach allows robust handling of occlusion in affected sub-parts while maintaining accurate alignment in unoccluded sub-parts, preventing the erroneous treatment of all inconsistent feature points as outliers.
2Reliability
If exemplars are used to provide shape constraint and solve graph matching problem, then alignment under occlusion is improved, but the method assumes changes in object shape can be captured through limited number of exemplars which is not feasible for objects with lot of changes in shape, pose and parts
Solution Approach 1:
The invention creates local models for each sub-part that can independently adapt to shape changes, pose variations, and occlusion patterns. Instead of relying on a limited set of static exemplars, the dynamic local models can handle diverse shape changes by focusing on local geometric relationships within each sub-part, thereby achieving both robust alignment under occlusion and high adaptability to shape variations.
3Reliability
If occlusion annotations are used during training where each layer predicts both part locations and occlusion states, then occlusion detection is improved, but the method requires knowledge of occluded feature points beforehand which is not available in real-time tracking
Solution Approach 1:
The invention enables the tracking system to automatically detect and handle occlusion without requiring pre-annotated occlusion data. By segmenting the shape into sub-parts and creating local models, the system can self-adapt to occlusion patterns during tracking by identifying which sub-parts exhibit abnormal motion patterns, thereby achieving robust occlusion detection without the complexity of pre-labeled training data.
4Measurement precision
If global model is applied to estimate shape, then overall shape tracking is improved, but occluded sub-parts cause distortion in un-occluded parts as the tracker deviates towards boundary between tracked and occluding objects
Solution Approach 1:
The invention segments the global shape model into multiple local sub-part models. Each local model independently estimates the position and orientation of its corresponding sub-part, preventing occluded sub-parts from distorting the estimation of unoccluded parts. This segmentation isolates the impact of occlusion to only the affected local regions while maintaining accurate tracking in unoccluded regions.
Solution Approach 2:
The invention applies different estimation strategies to different sub-parts based on their occlusion status. Local models allow the system to maintain high measurement precision in unoccluded sub-parts while robustly handling occlusion in affected sub-parts, thereby preventing the propagation of distortion from occluded regions to unoccluded regions that occurs with global models.
Data Source
AI summary
An image-processing device performing: extracting features from a target image based on a shape, the shape including sub parts each being a group of feature points each being correlated with a position within an object; deriving an estimated shape by applying a global model to an initial shape, the global model representing a relation between one of the features extracted and a motion of the initial shape; detecting, in the sub parts, an occluded sub part by comparing an estimated shape derived from a previous image and an estimated shape derived from a current image; and generating a final output shape by combining estimated shapes each derived by applying local models to the initial shape, the local models each representing a relation between the feature and a motion of a sub part in the sub parts and each restricting a motion of at least one of the sub parts.