Array Camera Depth Estimation for Occlusion-Aware Parallax Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for parallax detection and correction in images captured using array cameras face challenges with partially occluded regions and photometric variations, making it difficult to accurately estimate depth and visibility from multiple viewpoints.
Innovation Solution
A method involving selecting a reference viewpoint, normalizing images to enhance pixel similarity, determining initial depth estimates through pixel correspondence, and refining depth estimates using candidate subsets to handle occlusions and photometric variations, while utilizing calibration and geometric corrections to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-view stereo methods are used to estimate depth from multiple camera images, then depth information can be obtained, but accuracy deteriorates in regions with occlusions and photometric variations
Solution Approach 1:
The patent segments the set of captured images into multiple subsets, where each subset contains images that are visible from different viewpoints. By processing each subset separately and combining results, the system handles occlusions more effectively - regions occluded in one subset can be observed in other subsets, improving overall depth estimation reliability in previously problematic areas.
Solution Approach 2:
The patent changes the parameter of image selection by dynamically choosing different subsets of images based on visibility conditions. Instead of using all images uniformly, the system adapts which images are used for depth estimation at different spatial locations, selecting subsets where the target region is visible, thereby maintaining accuracy despite photometric variations and occlusions.
2Measurement precision
If images are captured from multiple viewpoints to improve depth estimation, then more depth information is available, but processing complexity increases due to photometric variations and occlusions
Solution Approach 1:
The patent divides the complex task of processing all images into smaller sub-tasks by segmenting images into subsets. Each subset is processed independently for depth estimation, which simplifies the processing at each step while still utilizing information from multiple viewpoints. This segmentation reduces the overall computational complexity compared to processing all images together.
Solution Approach 2:
The patent applies partial action by using only the necessary subset of images for each depth estimation task rather than processing all available images. By selecting and processing only relevant image subsets where the target region is visible, the system achieves sufficient depth precision without the excessive computational burden of processing every captured image.
3Quantity of substance
If all captured images are used for depth estimation, then more data is available for calculation, but accuracy decreases due to mismatched pixels from occlusions and photometric variations
Solution Approach 1:
The patent extracts and removes problematic images or image regions from the processing set - specifically, it excludes images where the target region is occluded or where photometric variations would cause mismatched pixels. By taking out only the necessary subset of images where the target is clearly visible, the system maintains high depth estimation accuracy while still utilizing multiple viewpoints.
Solution Approach 2:
The patent changes the parameter of image selection by dynamically adjusting which images are included in the depth estimation based on visibility conditions. Instead of using a fixed set of all images, the system adapts the image set parameter to include only those images where the target region is visible and photometric conditions are suitable, thereby maintaining precision while utilizing sufficient data quantity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances the accuracy of depth estimation and visibility determination in images captured by array cameras, effectively addressing issues of occlusions and photometric variations, resulting in improved depth maps and fused images with higher resolution.
Implementation Method 1
The amount an object shifts between different camera views is called the disparity, which is inversely proportional to the distance to the object. A disparity search that detects the shift of an object in multiple images can be used to calculate the distance to the object based upon the baseline distance between the cameras and the focal length of the cameras involved.
Data Source
AI summary
Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.