Array Camera Depth Estimation with Visibility-Aware Parallax Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for parallax detection and correction in images captured using array cameras face challenges with partially occluded regions and variations in light intensity, making it difficult to accurately estimate depth and visibility.
Innovation Solution
A method for parallax detection and correction that involves selecting a reference viewpoint, normalizing images, determining initial depth estimates, and using subsets of images to refine depth estimates, while considering visibility and photometric similarity criteria to handle occlusions and variations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-view stereo methods are used to estimate depth from images captured by array cameras, then depth information can be obtained, but accuracy deteriorates in partially occluded regions and areas with light intensity variations
Solution Approach 1:
The patent segments the set of captured images into multiple subsets, where each subset contains images that provide complementary visibility information for different regions of the scene. By processing each subset separately and combining results, the system achieves more reliable depth estimation in occluded regions compared to using all images together.
Solution Approach 2:
The patent changes the parameter of image selection by dynamically choosing different subsets of images based on visibility criteria and photometric similarity. This allows the system to adapt to local scene conditions, selecting only those images that provide reliable correspondence information for each pixel location, thereby improving depth estimation accuracy in challenging regions.
2Quantity of substance
If all captured images are used for depth estimation, then more data is available for calculation, but processing complexity and computational load increase
Solution Approach 1:
The patent extracts and uses only the necessary subset of images for each pixel location rather than processing all captured images. By identifying and removing images that do not contribute useful information (due to occlusions or poor photometric similarity), the system reduces computational complexity while maintaining depth estimation accuracy.
Solution Approach 2:
The patent applies partial action by using only the minimum necessary number of images from each subset required to achieve reliable depth estimation. Instead of exhaustively processing all possible image combinations, the system selects sufficient subsets that provide the needed visibility coverage, reducing computational burden while maintaining adequate accuracy.
3Measurement precision
If traditional disparity search methods are used, then depth can be calculated from pixel shift, but accuracy deteriorates when corresponding pixels cannot be reliably identified due to occlusions
Solution Approach 1:
The patent implements feedback by using visibility information and photometric similarity measurements to guide the selection of image subsets for depth estimation. This feedback mechanism allows the system to identify and exclude images where corresponding pixels are likely to be unreliable, thereby improving the accuracy of pixel correspondence identification and subsequent depth measurement.
Data Source
AI summary
Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.