Temporally Consistent Disparity Estimation Using Texture and Motion Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer stereo vision methods fail to produce temporally-consistent disparity maps, leading to unwanted temporal artifacts due to inconsistent disparity calculations across frames.
Innovation Solution
An image processing apparatus and method that enforces temporal consistency by classifying pixels into texture, non-texture, motion, and non-motion areas, and uses a temporal constraint unit to determine whether to enforce consistency based on these classifications, with a disparity estimator calculating disparities based on corresponding pixels in previous frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If disparity is calculated independently for each frame, then calculation speed is improved, but temporal consistency deteriorates causing artifacts
Solution Approach 1:
The patent applies preliminary action by pre-calculating disparities for pixels in texture areas and non-motion areas before the main disparity estimation process. This allows these pixels to inherit disparities from previous frames without recalculating, thereby maintaining temporal consistency while improving calculation speed. The temporal constraint unit identifies such pixels in advance and enforces temporal consistency on them.
2Stability of the object's composition
If temporal consistency is enforced for all pixels, then temporal consistency is improved, but calculation complexity increases
Solution Approach 1:
The patent applies local quality by differentiating pixel treatment based on their local characteristics. Pixels are classified into texture areas, non-texture areas, motion areas, and non-motion areas. Temporal consistency is enforced selectively on texture areas and non-motion areas where it is beneficial, while motion areas and non-texture areas are processed independently. This localized approach reduces overall calculation complexity while maintaining temporal consistency where needed.
3Measurement precision
If disparity estimation is performed for every pixel, then measurement precision is improved, but processing time increases
Solution Approach 1:
The patent applies partial action by performing full disparity estimation only for necessary pixels (motion areas and non-texture areas) while using temporal constraints for other pixels (texture areas and non-motion areas). This selective approach reduces processing time while maintaining sufficient disparity estimation accuracy for the overall image, as the constrained pixels inherit reliable disparities from previous frames.
Data Source
AI summary
An apparatus and method estimate disparities of pixels in a frame. In order to acquire temporally-consistent disparity maps it is determined whether to enforce a temporal consistency with respect to pixels. In order to determine whether to enforce the temporal consistency, a texture, motion, or a matched motion in the frame may be detected or estimated. A disparity of a pixel where the temporal consistency is enforced may be calculated based on a disparity of a corresponding pixel in a previous frame.