Depth Estimation from Light Fields Using Multi-Resolution EPI Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current scene reconstruction techniques, such as laser scanning and passive image-based stereo methods, face challenges in handling occlusions and producing accurate reconstructions, especially in highly detailed environments like movie sets, due to inefficiencies and inaccuracies in processing high-resolution images and handling moving elements.
Innovation Solution
A computer-implemented method for estimating depth in a scene using fine-to-coarse depth estimation from high spatio-angular resolution light fields, which computes reliable depth estimates around object boundaries first, then in more homogeneous regions, without global optimization, to retain precise object contours and ensure smooth reconstructions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If multiple laser scans are employed to capture complete scene data, then coverage of occluded regions is improved, but processing time and manual cleanup effort increase significantly
Solution Approach 1:
The system performs depth estimation at multiple resolution levels starting from the finest resolution, making preliminary depth decisions early in the processing pipeline. This preliminary action allows the system to identify reliable depth estimates before full processing, reducing the need for extensive manual cleanup of merged scans while maintaining complete scene coverage.
2Stability of the object's composition
If global optimization is applied to obtain smooth reconstruction results, then surface continuity is improved, but processing efficiency decreases and high resolution details are lost
Solution Approach 1:
The processing is segmented into multiple resolution levels, with depth estimation performed separately at each level. This segmentation allows the system to maintain high resolution details without applying global optimization that would blur edges, while still achieving local smoothness in homogeneous regions through the multi-resolution approach.
Solution Approach 2:
The system applies different processing strategies to different regions based on local characteristics. Edge regions maintain high resolution and sharp contours, while homogeneous regions benefit from smoothed depth estimates. This local quality approach ensures surface continuity where needed without sacrificing overall processing efficiency or high resolution details.
3Device complexity
If coarse-to-fine resolution processing is used, then computational complexity is reduced, but reconstruction accuracy at object silhouettes deteriorates
Solution Approach 1:
The system inverts the traditional coarse-to-fine processing approach by performing depth estimation from the finest resolution level first, then progressively moving to coarser resolutions. This inversion ensures that high resolution depth accuracy is captured at object silhouettes and edges before any downsampling occurs, while still benefiting from reduced computational complexity at coarser levels for filling in homogeneous regions.
4Manufacturing precision
If high resolution images are processed to capture detailed contours, then object boundary precision is improved, but processing time increases due to the large data volume
Solution Approach 1:
The high resolution image processing is segmented into multiple resolution levels. Depth estimation is performed at the finest resolution to capture precise object boundaries and contours, then the results are propagated to coarser resolution levels. This segmentation allows the system to maintain high contour precision where needed while reducing processing time for large data volumes through the multi-resolution hierarchy.
Solution Approach 2:
The system performs depth estimation partially at each resolution level, focusing computational effort on regions where high precision is needed (edge regions at fine resolution) while using simpler processing for homogeneous regions. This partial action approach maintains contour precision without requiring excessive processing of the entire high resolution dataset.
Data Source
AI summary
The disclosure provides an approach for estimating depth in a scene. According to one aspect, regions where the depth estimation is expected to perform well may first be identified in full-resolution epipolar-plane images (EPIs) generated from a plurality of images of the scene. Depth estimates for EPI-pixels with high edge confidence are determined by testing a number of discrete depth hypotheses and picking depths that lead to highest color density of sampled EPI-pixels. The depth estimate may also be propagated throughout the EPIs. This process of depth estimation and propagation may be iterated until all EPI-pixels with high edge confidence have been processed, and all EPIs may also be processed in this manner. The EPIs are then iteratively downsampled to coarser resolutions, at which edge confidence for EPI-pixels not yet processed are determined, depth estimates of EPI-pixels with high edge confidence made, and depth estimates propagated throughout the EPIs.