Articulated Object Tracking via Super-pixel Connectivity Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object tracking technologies face challenges in accurately tracking articulated objects, such as humans, in video sequences due to their changing geometrical shape and motion states, leading to issues like 'fading-out' when parts of the object stop moving, as they rely solely on temporal derivatives without considering the object's complete structure.
Innovation Solution
An image-processing apparatus that segments each frame into super-pixels, constructs connectivity graphs, and uses temporal derivative maps to track articulated objects by associating non-zero derivatives with super-pixels, updating classifications based on previous frames, and restoring missing parts based on connectivity graph comparisons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If temporal derivatives are used to track articulated objects, then moving objects can be detected, but objects fade out when parts stop moving
Solution Approach 1:
The patent segments the articulated object into multiple parts and tracks each part independently using connectivity graphs. This allows the system to maintain tracking of stationary parts even when temporal derivatives fail to detect them, preventing the fading-out problem while preserving accurate detection of moving parts.
Solution Approach 2:
The patent performs preliminary segmentation of the articulated object into parts before tracking begins. By pre-establishing the structural relationships between parts through connectivity graphs, the system can maintain continuous tracking of all parts regardless of their motion state, avoiding the reliability issue of temporal derivative-based methods.
2Ease of manufacture
If background subtraction is used to locate foreground objects, then simple segmentation is achieved, but articulated objects with changing shapes cannot be accurately tracked
Solution Approach 1:
The patent divides the articulated object into multiple parts and uses connectivity graphs to represent spatial relationships between these parts. This segmentation approach maintains simple processing while accurately capturing the changing boundaries of articulated objects as they move and deform.
Solution Approach 2:
The patent employs dynamic connectivity graphs that are updated across video frames to adapt to changing object shapes and configurations. This allows the system to maintain accurate object boundaries even as articulated objects deform, while keeping the segmentation process computationally efficient.
3Ease of operation
If average movement from center of mass is used to track objects, then simple tracking is achieved, but articulated objects with changing geometrical shapes cannot be accurately tracked
Solution Approach 1:
The patent segments the articulated object into multiple parts and tracks each part's position and motion independently. This maintains operational simplicity while significantly improving position accuracy by accounting for the changing geometrical shape and differential motion of different body parts.
Solution Approach 2:
The patent extends tracking from a single center-of-mass point to multiple dimensional components (individual parts) of the articulated object. This multi-dimensional approach preserves the simplicity of point-tracking operations while achieving accurate representation of complex articulated motion patterns.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Various aspects of an image-processing apparatus and method to track and retain an articulated object in a sequence of image frames are disclosed. The image-processing apparatus is configured to segment each image frame in the sequence of image frames into different segmented regions that corresponds to different super-pixels. An articulated object in a first motion state is detected by non-zero temporal derivatives between a first image frame and a second image frame. A first connectivity graph of a first set of super-pixels of the first image frame, is constructed. A second connectivity graph of a second set of super-pixels of the second image frame, is further constructed. A complete object mask of the articulated object in a second motion state is generated based on the first connectivity graph and the second connectivity graph, where at least a portion of the articulated object is stationary in the second motion state.